Exactly, I have spent a little time playing around with what type of information can be gathered from a machine loading a webpage. When you combine fonts, resolutions, versions, flash, cookies, language etc to produce a machine fingerprint you can get pretty close to uniquely identifying a single user within a restricted subset. TOR doesn't protect from this.
Now if you add in the fact that you can be identified by your browsing pattern as well (all of us will have a large number of sites we visit regularly and those follow us even if we change device, my list will be different to yours and we can be identified by it if someone is familiar enough with our habits) then you get an idea of what you are up against in order to actually acquire anonymity.
As you say, assume you are not anonymous. Even if you are right now there is nothing to say that some future time will see a consolidation of various databases containing historic browsing records allowing some future snooper to figure out what sites you visited today.