The Rabshakeh's list of cities (Hamath, Arpad, etc.) was not random. Commentators explain these were recent, well-known conquests by the Assyrian empire. By asking "Where are their gods?", he was making a powerful psychological argument: if the gods of these other nations couldn't save them from Assyria, what hope does Jerusalem have? This was a calculated taunt rooted in historical victories.