Commentators note that the psalmist's grief is deepened by remembering the joy of past corporate worship. Recalling how he led festive processions to God's house makes his current isolation more painful. John Calvin calls this a "godly sorrow," as it's focused on the loss of God's presence, not just social gatherings.