I had just spent most of my waking hours for an entire year learning about a single subject: Satellite Communications. I was absolutely certain I was ready to jump into the job and perform it perfectly from day one. Imagine my shock upon arriving at my first duty station (Fort Belvoir, Virginia Earth Terminal Complex) when the NCO in charge told me "Forget everything they taught you at that school. We don't really do things that way." The amazing part was that he was telling the truth.

In many ways, that is the way of software development. In college you learn the theory, the science, the techniques, and then you land your first programming job as a Software Engineer. You walk in expecting to be producing world-class applications in a few days, but if you are lucky enough to be assigned to work with a senior developer, you quickly learn that most of what you learned is good information, but not the way things are really done. The real learning starts when the classes end.

This was an interesting insight, because it speaks to an education difference between real science and computer science (or computer engineering, or whatever the better term for it is). In my undergraduate chemistry program, every class started with

"remember when you learned about $SUBJECT in $COURSELEVEL-1? that's not quite right"

and the next couple of weeks would be devoted to diving into the subject matter in greater detail with more complicated equations. At the end, we would take our new subject knowledge to re-derive the earlier coursework's equations, to show where the easier approximations came from and why they're not really wrong. This pattern continues all the way through graduate school, where eventually you get to the point where you are now at the limits of the known and the burden rests with you to demonstrate that what is know is not quite right but why the approximation has generally been good enough.

Contrast that with programming, where you learn that with a good enough specification you can implement almost anything on a compressed deadline... without learning about having to live with your code for the next two years, or about producing that specification, or about specs as targets that move faster and easier than deadlines.

Why is software development so hard? It could be argued that software development is so hard because programming is so easy. Since this seems to be contradictory, I will explain what that means.

Whether or not most people realize it, the titles "developer" and "programmer" are not interchangeable. Nearly all of us (myself included) start out as programmers. We learn the programming languages, the syntax, the data structures, the program flow controls, and how to assemble these elements to produce useful components. A good programmer knows the languages, the object models, and the techniques that are capable of producing stable, working pieces of code. But a good programmer is not a software developer. Some people remain very good programmers without ever moving to software development.

To become a software developer takes a good programmer who breaks out of the "program" mindset. A program does a specific task very well, but most professional development projects must go beyond specific tasks. When a company or government agency starts a project it is very seldom a project to create a program. Instead, it is typically to create a solution. This is a very important distinction. A program is a software solution to a single, focused problem while a system is a collection of programs that satisfy a much broader class of problems.

For example, a project to transform data from a legacy system to XML results in a program while a project to make data across legacy systems available to a web-based front end is a system. The difference is that while the transformation program does a single job very well, the legacy to web system consists of many, many components, any of which at their core are a program.

From this view of program vs. system, it makes sense that a software developer is not the same as a programmer. While a programmer must be an expert in a very narrow domain of the overall problem (depth), a developer must have a solid understanding of both the depth and breadth of the problem. Not only must the developer know how to create programs, but he must also know how to assemble the many programs that perform the various tasks required to solve the larger problem.