Still good topic for late night conversation today. I still think on this from time to time.
IMO the crux of the issue may revolve around 'perception' (physical relationship with consciousness); that is, perception is reality rather than a proof of it. Schrödinger has some interesting related conclusions about consciousness in his book What is Life?. He basically concludes that a plurality of consciousnesses must be an illusion.