Funding has become outcome oriented and science has always worked to some degree by accident. I think what is being described in this article is a symptom of the calcification of institutions. It's a general rule that institutions established for the best of intentions inevitably become their own worst enemy because the institution becomes more important than whatever its founding goal was.

Schools, in particular, are great examples of this but so are government structures of all sorts, prisons, hospitals, banks. The institutions ultimately outlive and become contrary to their original purposes when the health of the institutions becomes the priority over time. When was the last time we had a teacher's strike over compulsory standardized testing rather than over wages? Sorry to say so but I think this is a damning fact.

Educational institutions have become progressively specialized over time with the specific intention of pushing scientific progress with this enormous blind spot for the serendipitous nature of discovery and invention. It's precisely the emphasis on STEM that is causing this failure. The harder you push the less progress you make.

It's not that mysterious why this should happen. When people come to believe that "science" is for specialists and not up to each of us no matter what our background is then there is a tendency to avoid becoming an active inventor because people come to believe they are unqualified to pursue novel ideas and that those well-paid specialists will inevitably do a better job than any individual can ever do. This is also why the GNU/Linux desktop never took off with the masses despite being quite usable and freely available. People feel that it's not their business and they are better served by waiting for some corporate solution to their problems.

This is, unfortunately, a very dangerous path which leads to tyranny and sadly that seems to be what we're sowing from our harvest of institutional specialization. If you look around at Humanities programs in universities today you will see a great decline in their influence, size and funding precisely because the students think the purpose of education is to get a job and that the only jobs that pay well are in technology.

This becomes a self-fulfilling prophecy that leads to a form of tyranny.