Charles Ellicott Commentary Numbers 4:20

"but they shall not go in to see the sanctuary even for a moment, lest they die." — Numbers 4:20 (ASV)

But they shall not go in to see when the holy things are covered. —Better, But they shall not go in to see the holy things (or, the sanctuary) for a moment.

The Levites were not permitted to enter the holy place in which the priests ministered, much less the Holy of Holies. Whether this rule was or was not relaxed at the time of the removal of the Tabernacle, as the prohibition against entrance into the Holy of Holies must have been in regard to the priests, the Levites were not permitted in any case to look upon the Ark and the other holy things until they were covered.