Charles Ellicott Commentary 1 Corinthians 5:10

"not at all [meaning] with the fornicators of this world, or with the covetous and extortioners, or with idolaters; for then must ye needs go out of the world:" — 1 Corinthians 5:10 (ASV)

Yet not altogether with the fornicators of this world.—This is a limitation and explanation of the command given not to associate with fornicators. It would have been almost impossible for the command to be literally obeyed without the Christian withdrawing altogether from the business of life, so the Apostle explains that it is the good reputation and purity of the Church that he is concerned with preserving. There are so many fornicators, and the covetous, and idolaters in this world (i.e., the non-Christian world) that people must encounter them. But Christians must tolerate no such sins among their own number; they must exclude from their social circle any brother who, while bearing the name of Christ, indulges in the vices of the non-Christian world. The Church is to be the light of the world, and not the recipient of the world’s darkness.