Commentators explain that when the verse says "He is thy praise," it means God is the sole source of Israel's honor and glory. As John Calvin notes, apart from God, they would have nothing to boast of and would be left with shame. Our identity and worth are found exclusively in our relationship with Him.