Good Software

For some time now, I've been working on the theory that people are different.  Go ahead pause to consider if you really want to continue reading.  

Seriously?  People are different ... and you've been working on this theory "for some time"?  Maybe if I left now I would have enough time to finish that sudoku.

Well, there's a little bit more to it than just that.  So let me set the scene a bit and I think you'll see that there's something here to consider.  Also there could be some horrifying consequences that will render the Software Engineering Industry asunder.  Destroying the lives of hard working men and women whose only crime was doing their jobs.  Meanwhile these same consequences will facilitate the first megalomaniac sociopath who realizes the implications and has the means and ambition to act on them to rebuild the Software Engineering Industry in their own image.  You know of what I speak.  A barren wasteland where Software Engineers form primitive tribes to survive.  And tire armor.  Yeah!  Tire armor for everybody!  TIRE ARMOR FOREVER!!!

Okay, maybe not that.  But it might explain *some* of the reasons why everyone yells at each other so much about programming languages.  And … it could provide someone who's really clever (and owns a software engineering company) to build more efficient teams and ensure better code base quality over time.

The situation today

Is that software kind of sucks.  The best in the world are able to just barely create software that does what we want without emailing our social security number straight identity thieves.  

Right now your best bet for getting software that works is to find a small group of highly motivated smart people who get along (but who also secretly hate each other, just a little bit) and put them into a room under the guidance of "that guy".  You know.  That guy who pesters all of your stake holders for every minor detail of what they want until the stake holders decide that they were better off doing it manually.  That guy who refuses to sign off on the release until you get 200% code coverage, user studies successfully completed by a homeless man from a different country who doesn't speak the language the application displays and who is completely illiterate, and finally all of the integration tests are successfully run by a blind monkey.  No not a code monkey like a real monkey.

Then you give that team complete autonomy and a budget that is two hundred percent of what the estimates tell you, and they still fail to deliver anything useful half of the time.

All of this assumes that you don't need any security in your application.  If it needs to be secure, then you need the same thing with some minor changes.  Your highly motivated team of smart people also need to have PhDs.  Like each one needs to have multiple PhDs.  Also they each need no less than five (5) and no more than eight (8) graduate students all of whom were child prodigies.  Finally, they each need to have been doing computer security work for at least 30 years (and it wouldn't hurt if they were spies in the cold war).

You still need "that guy", but he also needs some upgrades.  And unfortunately, you'll never find him.  He has to find you.  Just assemble your team and put out a press release that you're interested in maybe creating a secure application some day.  Pay the team to do research or something.  Perhaps they can create prototypes of what the final secure application could be like.  Just don't get jittery and use one of the prototypes because it's "good enough".  Because it's not good enough.  

Eventually, that guy will show up.  You'll probably find him in your house one day after work.  He will ask you questions and you will answer them.  Don't worry about getting them correct because you cannot get them correct.  He is only interested in how you fail.  

Once "that guy" finds you and he leads your team to construct your secure application, do not forget the final step.  It is vital that you have a plan for when some bored Russian undergrad proves that the Chinese have been putting hardware viruses in the sand that is used to create the computer chips that your application runs on and *that* allows the NSA to trigger the backdoor that they hid in number theory, which devastates the entire security model of your application causing it to immediately email your social security number to every identity thief on the planet.

The right way to write software

Is almost definitely a myth.  Which is actually kind of the point that I wanted to start with.  People are different.  This has consequences for how they create software.  Unfortunately, the completely true picture I drew above concerning the state of software development today gives a very big incentive to charlatans who will try to sell you "the right way to write software".  Try not to be too hard on them, there's a large chance they don't realize that they are charlatans.  Their methods might even work … for them.  And maybe their methods even work for people sufficiently like them.  But their methods don't work for everyone.  Also if you try to cheat by only employing people who are all sufficiently similar, you'll get destroyed when they encounter a problem that they can't get their heads wrapped around, but which literally any other developer on the planet could solve in twenty minutes (but not if they have to code like *that*).

At one point in time I think I really wanted to find "the one way" to program.  It was going to be the right way to program, it would let me work faster, it would eliminate bugs, and I could teach it to everyone.  Then one day when I was knee deep in type theory, trying to decide if domain theory or category theory should be the next topic that I venture into, when I happened to consider the economic side of perfect software.  Can a company actually bill it's clients enough or can an internal software division procure a sufficient budget to pay for the software engineer who is also a computer science category theorist?  The only reason that I spend time trying to decide whether I like dependent types or general algebraic data types with polymorphic kinds better is because I'm lonely and weird.  Normal people have lives.

And then there's the evidence.  I kept on bumping into problems that I couldn't solve without much frustration, but normal software engineers seem to be able to tackle without any problems.  Can trying to figure out Sequent Calculus in your free time ruin you as a software engineer?  Well, no that doesn't make sense either because I also kept bumping into problems that normal software engineers found incomprehensible, but that I could solve trivially.  A suspicious pattern that suggests that there's no best way of thinking that allows you to solve all problems.  Let's take a look at an example.

TDD vs Parametric Polymorphism (ie type theory stuff)

There are two claims that have been floating around on the internet.  They sound very similar to each other, but you wouldn't want to assert that they are the same.  At least don't if you want to stay out of trouble.  The first is the TDD claim:  "I write my tests first and that allows me to write correct code."  The second is the Hindley-Milner type inference claim:  "When my program type checks, then I know it's correct."

To me, both of these claims sound like magical thinking.  I do this thing, which has a totally intrinsic and completely unexplained and unexplored ability to manifest "correct" software regardless of the constraints of physical reality, and my software just somehow works completely correct.  The first time.  Every time.  

I don't buy that TDD is *the* way to write software because I have rarely found anyone who can actually explain to me what it is or why it works.  The people who do give me an actionable definition have always given me a different actionable definition than the previous person who gave me an actionable definition.  I suppose I wouldn't be opposed to trying TDD in practice, however I would first need to actually understand what it is I'm suppose to be doing.  Write the test first and then correct software is emitted isn't a well defined algorithm.

I don't buy that Hindley-Milner type inference is *the* way to write software because I know about dependent types.  And universal quantification, and polymorphic kinds, and general algebraic data types.  Even assuming you upgrade your Parametric Polymorphism, popularized in languages that use Hindley-Milner like inference such as ML or Haskell, to a dependently typed system and then go on to prove everything you can think of about every last function, then you still have to worry about an impedance mismatch between the problem you're solving and the actual code.  A perfectly proven algorithm doesn't protect you against misidentifying the problem you were trying to solve.

However!  Both types of people write software that works!  It isn't perfect, but it does mostly solve the problem and allows you to get on with your life.

So my theory at least for now is that people are different.  They don't think the same, they don't work the same, and they don't write software the same.  And that's just something we're going to have to deal with moving forward in the Software Engineering Industry.  I'm not sure exactly what this means that we should do, but I do think this means that we can't settle on "the one way" to write software.  At least if we want to be able to write good software.