Note: This file has moved to notablog.

So what's a functional spec? I started a kneejerk reply to this post based on the first few paragraphs, then I realized Ben is at least as familiar with the topic as I am, though we have different points of view. So I'll leave out the bulk of my reactions and throw in a few general comments. Some folks have criticized the human tendency to avoid asking fundamental questions when they start writing requirements and functional specs. I'm not sure how much of that is related to writing and how much of it is simply part of the whole conundrum of the software development process. Most programming is done on a "craftsman" level at best; places like the SEI (Software Engineering Institute) have spent millions of dollars trying to figure out the general issues, and entire fields like CHI/HCI/Human Factors and Usability Engineering have arisen to address the core issue of figuring out how to make the software "right". Which is usually what a functional spec is supposed to be about (but not what a functional spec is really about - see below). Any writer who gets handed a simple request to "write a functional spec for this software project" has either an uphill battle in front of them, or an opportunity to really affect the software development process at their company. Most people dont know what programmers do, or how they do it. Most programmers don't know how to articulate what they do and what they need. The skills, talents, aptitudes, and the very mindset needed to both do the work, manage the work, and communicate about the work are often at opposite ends of the spectrum. This is not merely an issue of needing a multitalented person - the very states of mind to do either is often contradictory. Cultivating one destroys the other. I was going to go off on an extended exploration of this topic, but I'll skip that and simply comment that there are a lot of parallels between the writer's state of mind and the programmer's state of mind. Often with a different emphasis, but writers have a lot more in common with programmers than either typically realize. In that sense, programming is more like when you're truly trying to write creatively - not just trying to figure out how to say it (which is a lot of what technical writing is about), but trying to figure out what to say. As the recent discussion on interruptions showed, this issue is difficult to work with at the best of times. I continually get the feeling that the key fact 80-90% of the people miss - including most programmers - is how totally unlike almost every other human endeavor programming is, on an organizational scale. People keep trying to come up with metaphors to get a handle on software project management. The closest thing to software project management is any large engineering effort to design and engineer completely new systems to address completely new needs. I'm not talking about designing, building and testing a VCR or a cellular phone from scratch. While these are complex and ingenious devices and the engineers who created them are no doubt gifted, they serve a well-known and well-defined function. How much more complex might it be to design a new kind of video recording/displaying technology from scratch? But this too is a well-known concept - you take images, you record them, you replay them. Now try to tackle something for which we don't have a well-known concept. People speak language; computers speak math. Programming is about taking that language and converting it to math. Human language is the essence of flexibility. Math either adds up or doesn't. Often the language itself doesn't exist in any formal document until the software development process begins. Often the document that does exist is rife with gaps, errors, and inconsistencies (just think of any large bureaucratic set of rules and regulations). When it does, only when we attempt to convert it to math do we we realize how much nuance and ambiguity there is in language, how much human judgement is expected to fill in the gaps or deal with special cases. There is no set definition of what a functional spec is. All you can do is get a better handle on the process of developing a functional spec, not on how the text of the spec itself should be written and organized. In general, in theory the specification process is straightforward: You start by developing high-level design requirements. This is where you define the scope of the project domain and the general goals ("we're writing a database system that needs to be industry standard compliant, highly reliable, and run fast enough to be feasible for use in point of sale systems"). In reality, most people just skip right past this. Then comes the functional specification, where you nail down what you want (but not how it should be built), testing it against the requirements defined in the high-level design document. In theory this is developed as a cooperative effort between marketing, in figuring out what is needed for the product, and engineering, in figuring out what can be built and for how much. In reality, this is the substance of a haggling process between engineering and marketing, and the process is usually dominated by one side or the other. Typically in the industry, companies are either technology-driven or market-driven. This results in a corresponding dominance by engineering or marketing. In theory, the way it's *supposed* to work, upper management keeps the two in balance and helps to find the balance point that results in a successful product and a succesful company. Once a first draft functional spec is hammered out, the programmers go off and figure out how to break that up into pieces of work they can individually and cooperatively work on. As they work on them, they figure out that some things are easier, some things are harder, some things are better done one way than another. The spec shifts to fit - keeping the spec up to date is an important part of the process. In practice, this almost never happens and the spec is often left far behind by the end of the project. Most often people just don't pay attention to keeping the spec updated, or one side or the other howls bloody murder when a significant change has to be made, or the organization goes totally overboard in this direction and the spec and the software development process becomes fairly moribund and sluggish (this is what it's like at most really large IS organizations). Let's take a look at this from another angle. An important point to understand is that, software engineering being a much broader domain than civil engineering, for example, the spec is *not* like a blueprint. (By the time it's accurate enough and detailed enough to qualify as a blueprint, the program is almost done, and in fact that's a pretty good software develpoment methodology, if the project is such that you can get away with it; with seriously large projects, you usually have too many interdependencies and you need to very things by trying to actually do them.) A spec is not like a contract or a work order that defines exactly what product is to be delivered and how. A spec is not the answer, it is the question, and it is a very complex question that evolves as the asker and answerer hash out what exactly the asker is looking for. Think of it this way; the conversation between marketing and engineering is a very complex one. You start writing things down to keep track of them as you figure out what you're talking about, and building an outline of the conversation. Occasionally, as the conversation progresses and you start to have something resembling a document, you rewrite the whole thing from scratch. Finally, you start to fill in the individual points in the outline, and as you do this you come across changes that need to be made in the outline itself; things that are easier than you thought, things that are harder, things that need to be organized differently than you first thought, things that seemed important at the time but later investigation showed to be trivial, and vice versa. Typically in the industry, people impose an arbitrary and usually incorrect demarcation, a point where they say "this is what the spec is, go build it." To a certain degree this is necessary and appropriate. At some point you have to start limiting the scope of the conversation, or you'll be there forever. A good, albeit brute force, tactic to do that is to draw a line around the conversation to date and say it's done, now on to the next stage. This is necessary because typically neither side in the conversation has a really good idea of the other side's point of view. This makes controlling the conversation and keeping the conversation on track and making forward progress difficult. If you have a specification team that is lead by good marketer who has also been a good engineer, and a good engineer who has also been a good marketer, then you're in software development heaven and it's time to stop smoking that funny tobbaco. There are a few excellent books written on these topics for programmers. The ones I usually recommend are Steve McConnel's _Code Complete_ and _Rapid Development_. I also highly recommend Jakob Nielsen's _Usability Engineering_. (This essay is based on a discussion on the techwr-l mailing list, and owes much of its inspiration and some of its substance to relevant comments by Ben Kovitz).