Who is this article for?

All software development professionals will find interest in this article, but those managing, crafting or analyzing requirements may find the most interest.

Introduction

Why is so much ceremony needed around software development?  Regardless of the methodology there is a great deal of meeting, clarifying, verifying, demonstrating, and checking.   If you came from another planet and watched how we develop software you’d come to the conclusion that we don’t know how to communicate complex abstract ideas into concrete reality.  And we don’t. To be sure, we’ve tried:  detailed specifications, UML diagrams, use cases, user stories, and just plain discussion.  All of these are incremental changes to a better requirements landscape, but fleshing out our software needs still takes considerable time and effort.

Process and Methodology as Requirements Risk Mitigation

BDUF ( Big Design Up Front ) was the waterfall method to define requirements.   It was a way of saying….”Let’s get everything straight BEFORE we start the expensive process of coding and testing all this.”    It’s logical, but makes the broad assumption that people will know what they want and deviation from the initial plan will be minimal.   There are some projects where this is a very safe assumption, but not always.

Agile capitalized on this fault and said “You can’t know everything up front, so just get started and course correct along the way.”   Does this mean that agile projects always succeed? No.  It means that they allow adaptation to change.  Projects where innovation is important or where the product owner is unfamiliar with developing software naturally make good candidates with Agile.  But its pricey and if you don’t need all the involvement…..then going back to waterfall might be a good option.

With both methodologies you see an attempt to address the real problem:  understanding and materializing requirements so that we get exactly what we want.   The abstract nature of software means we’re collectively imagining what it could and should be.  But we haven’t found a silver bullet for this type of collective dreaming.

Communicating and interpreting our desires is not as easy as it seems.  Some of us are good at drawing pictures…..but most of us aren’t.  To mitigate the possibility that we’re not going in the right direction we’ve invented these elaborate software processes ( Waterfall & Agile …..and the pragmatic others ) to keep our collective minds on the same path to vision creation.

Are  There Better Ways?

Even with these processes we still end up in projects where we hear these things said:

“Oh…that’s not what I meant.  It should do this…”

“No, no,no….you didn’t understand what I was saying. “

“That wasn’t the requirement.  Read it again….I built what was asked for.”

“Yes. I know she said that….but what she really meant was…”

So, software requirements are still an error prone bottle neck in producing the product.  If we hope to make software development better;  then we must get better at this.  In paraphrasing Eliyahu Goldratt in The Goal: You can only run as fast as your slowest player.  That player for software is requirements gathering, analysis, understanding and verification.

Looking outside our field, other industries, who’ve been around longer and develop complex systems too, have similar challenges.  What do they do?

Design –  My wife is a talented contract graphic designer for a local St. Louis firm called Kuhlmann-Leavitt.  They’ve won many awards and have very satisfied customers.  All this despite the nebulous, subjective nature of graphic and three dimensional design.   How do they do it?  The design field is interesting.  They don’t estimate anything.  They simply ask that you hire them for your artistic work and set a budget.  They’ll do the best design work they can for that money.  No more, no less.   So right away the budget issue is taken out of the picture.  Lastly, they don’t ask for detailed requirements,  just some idea of what you’re looking for.  You’re not hiring Kuhlmann-Leavitt to build.  You’re hiring them to create.  My bolding hints at the nuance.  Considerable authority and autonomy is given to the designers.  This is just expected.  Software development relies much more heavily on ‘domain experts’.

Manufacturing – In the manufacturing world BDUF could be rewritten to be MDUF ( Massive Design Up Front ).  Not only is the product designed way up front and in detail, but the process and mechanisms to build the product, which aren’t always the same, are designed up front too.  Before the real product even undergoes one production run…many small demo runs are carried out.  The results are stunning.  Once the product starts rolling off the assembly line its produced to exacting tolerances.  It mostly fits the requirements of marketing and can be replicated thousands of times over.

Construction – We’re talking about unique, one of kind construction here.  Not production line assembly of a subdivision.  This one-off building is, to my mind, the most similar to software.  Some requirements come from regulations and building codes.  Others come from the purchaser of the structure.  There are multiple stakeholders.  Requirements are digested, and then artifacts are created.  A series of mock-ups are assembled and based on this a cost estimate is created.  Even with all the up-front planning there are always “gotchas” and changes that are thrown into the project.

In all three of these examples the end product is visual and intuitive which helps with requirements understanding.  A picture really is worth a thousand words.  Software, as workflow, data and creative design is more abstract.  We’re trying to craft ideas and concepts into something that is somewhat tangible.  Certainly the user interface can be mocked in a similar way and there is great value that comes from the discussions around this, but a sufficiently detailed workflow and data set always serve to vex our attempts to provide clarity to requirements.

What’s next?  What can we do?

Here’s part of our problem:

1. We discuss the first iteration of requirements with our product owners.  The person leading that discussion is gifted in facilitating this.

2. From this meeting, the requirements are crafted into user stories.

See it?  We just lost all the context, emotion, body language, nuance, and reactions by everyone else by drafting something into writing.  I don’t care how good a writer you are…you can’t replicate the requirements conversation.  So why do we try?

The next problem comes when you ask each person to interpret the written user stories.  Hopefully each team member was present for the conversation so they could somewhat remember what transpired.  But, even with attendance, the human memory is frail and human perceptions, bias and concentration vary. Each person may see and read the requirements slightly different.  In short there are too many lenses.

Rather than try to abstract requirement conversations into another form ( writing ) or create even more abstract artifacts ( UML, Use Cases ) we should seek to record them via video, photographs of whiteboard drawings, and publishing all this to some kind of requirements wiki or blog.  Doing this for each iteration review, iteration planning, or project meeting would surely capture a much more complete document of the project’s scope, changes, stakeholder motives and historical background.  Further, it would exist in perpetuity for future developers to watch……just like a movie or project documentary.

Summary

I do realize that the suggestion to move towards requirements documentaries may be fraught with organizational, legal, and other challenges.  I would never suggest that change on this scale is easy.   But I do believe there is productivity savings to be found here.  As I hinted in Decline of Written Requirements  I think we’re using 20th century tools and techniques to gather and analyze requirements in a 21st century world.  To lessen the need for process mitigation of requirements risk we may need to refocus and leverage our new tools: cell phone cameras,  high quality digital video, wikis, and blogs.