So I'm going to take your same post and twist it for optimism instead of pessimism. I'm going to do this despite the fact that I'm deeply cynical about it all. The difference is "why?"
My motorcycle is going stupider and stupider fast because engineering technology is trickling down from aerospace to consumer use. I tune that bike with a laptop... that I bought for $40. Think about that: I have control (to the percentage point) over fuel and spark for every region of that engine's run. When I was in high school you couldn't do that in Formula 1. So that's a need being met by available technology.
Contrast with the Concorde: I have this picture, framed. Once I've restored it, it'll go up next to one of the Mattes a buddy of mine made for the Total Recall remake. The styles are similar. That's a Boeing 2707, first iteration, back when Boeing thought they could economically produce a supersonic swing-wing SST every bit as big as a 747. When they rolled it out, the SR-71/A-12 was still secret; their best guess at an engine was the YJ93, which the XB-70 used. Still, it was within the limits of reasonable technology and we didn't know that a Mach 2.7 swing-wing jumboliner was crazy talk.
And it might not have been, if it weren't for OPEC.
I've got all sorts of data on the 2707. Its fuel curves were, in a word, optimistic. That's why they ditched the swing wing; they just didn't have the power and they didn't have the efficiency to push that much heavy that fast. Then the oil crisis hit, gas shot through the roof, the project was cancelled and some wag put a sign up on I-5 South saying
The Concorde, meanwhile, was a national project for France and England. No way they're going to let a little thing like "pragmatism" get in the way of that! Which is why the Concorde continued to fly, at $15k a ticket (and more), until 2003... losing $10,000 per ticket.
It comes back around to "WHY?"
When you're fighting a proxy war with the Soviet Union, it makes sense to build Saturn V rockets to send people to the moon first. That's a PR battle for hearts and minds. But once you're there, you're no longer making victories. The Space Shuttle exists entirely for one object:
That's a KH-9 Hexagon, the biggest spy satellite we ever flew. Problem was, the NRO really didn't want to launch things in space shuttles. They thought shuttles were a terrible idea. But since half the justification for the shuttle was military, the NRO dared NASA to build something so preposterously large that it'd get killed in committee. Unfortunately NASA called their bluff and we ended up with a turkey nobody wanted. Why did we ground the shuttle with no replacement? Because the NRO had evolved beyond the needs of anything that large and because what shuttles we had left were dangerous as fuck.
See, nowhere in there is there an organic need for a space shuttle. It was every bit as political as the Saturn V, every bit as political as the Concorde. Take away the Soviets and you take away your space race. Meanwhile, the Chinese are 30 years and more behind so even if you wanted to drum up another costly, inefficient proxy battle it's gonna be a while before it makes any sense.
There was a time, though, when it made no sense to cross the ocean. There was a time when only crazy experimenters tried to fly. And there was a time when the raw juice in fossil fuels was insanely dangerous and explosive, and now I buy them by the gallon with the flexpass on my keychain so that I can chase Ferrari Scuderias down Sepulveda Pass at midnight at 120mph.
There's no "WHY" for space yet. There was, then there wasn't. I suspect there will be again. I doubt it'll look the way the sci fi writers say and I doubt it'll be any time soon, but I don't doubt its eventuality.
The Vorkosigan Saga kicks the shit out of the Hyperion Cantos, by the way. I couldn't stand Dan Simmons. The audiobook, in particular, was grating as hell.