There's no physical space everyone wants to be in either. VRML and X3D were closer to something that could be generally useful to whatever degree VR is generally useful, but both came along too soon and were dead by the time actually doing something with them was viable.
It's an interesting distinction. The thing about virtual spaces is they're easier to separate and propagate than real spaces. And if you've got a protocol (kind of like how ReDEF really wants this to be about Unreal Engine) then you can shift from space to space.
Which was what VRML was designed to support, being a 3D analog of HTML. It was modeled on Open Inventor rather than a game engine, and never got good editors, but the idea was I make a space on my server and you make one on yours and we can link them together. Something like that is a much more likely to yield science fictiony ubiquitous 3d interfaces than the Unreal engine, even though the first try didn't work out so well.