John Gill Commentary 2 Kings 3:2

"And he did that which was evil in the sight of Jehovah, but not like his father, and like his mother; for he put away the pillar of Baal that his father had made." — 2 Kings 3:2 (ASV)

And he did evil in the sight of the Lord Was guilty of idolatry:

but not like his father, and like his mother ; his father Ahab, and his mother Jezebel:

for he put away the image of Baal that his father had made ;

He did not destroy it, only removed it from the temple of Baal where it was set, that it might not be worshipped, at least publicly, see (1 Kings 16:31 1 Kings 16:32).

This he did, either moved to it by his own conscience, observing the sudden deaths of his father and brother, which he might suppose was for their idolatry; or in order to obtain success in his war with Moab he was entering into; or being instigated by Jehoshaphat to do it, or otherwise he might refuse to join him.