Kelly Johnson was fond of saying "If it looks pretty, it'll fly pretty." Ben Rich points out in his book "Skunk Works" that his career encompassed twenty seven aircraft while observing that a modern engineer will be lucky with one. He also points out that in order to get the go-ahead on OXCART (which he calls the SR-71 as OXCART hadn't been declassified yet) took Kelly Johnson, one guy at DS&T and two senators while there were over 2500 auditors on the B-2 project alone. It was just a different world. Stakes were lower, costs were lower, innovation flowed like river water. Interestingly enough, though, in poking around the Web for this article, I decided to look back on the RQ-3. So here's the thing. We've had two new fighters since I was a kid - the F-22 and the F-35. They're expensive, they're big, they're ridiculous. But we've had nine new UAVs since 1994, and that doesn't include off-the-books beasties like the RQ-170. That's just MQ-1 through MQ-9. If I were an aero and astro guy, I'd be designing drones. Hell, I just started messing around with a Blade MSRX last week and I'm hooked.
While there would be arguments regarding having only a limited number of people on a TS (and above) project (such as OXCART), you limit the potential of information being leaked, and reduce the amount of tracing required if it is leaked. Using your example, 4 people (SR-71) is a lot easier to maintain confidentiality with vs 2500 (B2); otherwise there's an inceease risk of a repeating of the Thomas Cavanaugh or Noshir Gowadia affairs. No matter your stance on drones, they are the current trend and will be for some time, I imagine. From a cost and risk-assessment basis, they are a lot more appeasing than pilot-based aircraft. I've not had a crack at playing with consumer-level drones or remote control helicopers yet, the extremely limited flight time (5-10mins) kinda puts me off. That said, it will only be a matter of the time until the consumer grade ones weigh less, have further ranges etc. It will be even more intriguing when they start operating in autonomous union, too.