I don't think you have to worry about it. You're not their only candidate and if they're not conscious of all the strangers they're seeing you shouldn't work there.
I went through 280 resumes two months ago. That generated maybe 20 interviews for my wife, all of which were Zoom/phone. If that interview went well, we had an in-person interview with the whole team, or as many as were in the office and not doing telemedicine (we've had everyone in the office exactly once since February). If I recall correctly, we had three in-person interviews.