John Gill Commentary Genesis 43:5

"but if thou wilt not send him, we will not go down; for the man said unto us, Ye shall not see my face, except your brother be with you." — Genesis 43:5 (ASV)

But if you will not send [him], we will not go down
This they said not as undutiful, and from a spirit of rebellion and disobedience to their father, or of stubbornness and obstinacy, but because they dared not go down, nor could they with any safety; they might expect to be taken up as spies, and put to death as they were threatened; and besides, it would be in vain, and to no purpose, since there was no likelihood of succeeding, or of getting any provision.

for the man said to us, you shall not see my face, except your
brother [be] with you ;
which they repeat both for the confirmation of it, and as an apology for themselves, to clear them from any charge of unfaithfulness.