At a certain level, this is what a test-driven development is meant to accomplish. You write all the tests that show your software is working within specification, then you write the implementation to pass the tests. In practice? You've got the halting problem, which says that you can never be 100% sure that your process won't freeze unless you severely constrain how you program. You've got a workforce primarily composed of arrogant kids out of college who would rather make a name for themselves working on "sexy" projects than write documentation and audit code all day. And you've got the managers above who prioritize "it works" over an elegant solution.
Right, this is what I had in mind. There are definitely better ways to develop software than others, TDD is one. That's what the idea of SE means to me. Halting problem says that there doesn't exist a terminating decision procedure for all programs. You can still prove that a program terminates using Hoare logic or something similar.