Yeah Battletech offends me deeply. It's got an incredible amount of lore, all of it rendered as dully as humanly imaginable. It has incredible industrial design and miserable gameplay. That's the amazing thing to me - No attempts whatsoever have been made to increase the playability, starting with Fasa, passing through Wizards of the Coast and now with Piranha - fucking Mechwarrior is the most boring game imaginable where every scenario is a revelation in "this would be much easier and more pleasant if I weren't trapped in a giant robot."
Here's the thing: All those games you list were designed by college kids with no friends to be played by college kids with no friends. Those college kids? They played them. And they interacted with the guys who wrote them. Cyberpunk is an excellent example. Mike Pondsmith so loved Walter Jon Williams' Hardwired that he invited WJW to playtest Cyberpunk before it was published. And between Cyberpunk and Cyberpunk 2020, they made quality-of-life changes like "youth bonuses" to discourage players from just running through the military and college as character generation buffs; Cyberpunk was entirely full of 26-year-old mercenaries hot off the GI bill.
Frankly, I think Magic The Gathering fucked it up for everyone. Instead of giving you something to think about it gave you bridge for incels. Now those incels could collect baseball cards, too. Pretty soon there was a giant pile of money and people came to D&D through fucking Pokemon.
And I think computers fucked it up for everyone. The massive milsim tabletop shit went right the fuck away as soon as you could give a computer the overhead. Ever play Car Wars? I reckon not. Turns are divided into ten phases, each turn takes a second, entirely so that the physics of vehicular combat can be properly addressed. A good and proper autoduel takes an hour and a half, two hours to play out but represents ten to fifteen seconds of real time; actual Car Wars racing rules allow you to condense all non-mediated turns by 10x or 100x because if you aren't cutting someone off or shooting at them the results don't really matter. Car Wars hit its peak about 1992, 1993... fast forward to 1995: