Requirements Documentaries – A Recipe

Introduction

In previous posts I’ve talked about how written requirements are on the decline and how our development methodologies and practices, at least in part, are constructed to hedge the risk of misunderstanding requirements.   As an outgrowth of those posts I’ll talk here about my vision for Requirements Documentaries as an alternative to written requirements and user stories.  I hope by the end of this post a formula will emerge for crafting video based user stories into a persistable set of requirements documentation that captures much of the context around a project.

Preparation and Coaching

How do you prepare the team for video recording?  While the discussion should be as open and natural as possible:  some coaching should be required here.  Meetings of all stripes tend to deviate from their course and sometimes into topics that are not appropriate for the organizational culture.  While these things can and should be edited out for the purpose of requirements documentaries; the team and customers should receive coaching ahead of time on how best to present their views and what topics/situations to avoid.

Other good tips should be included like: speaking clearly and loud enough for everyone to hear, avoid bad behaviors ( picking nose, tapping on the table, etc ).   We want it to be natural, but we also want it to be professional.

Tools

I believe this is a simple, cost effective set of tools that would serve as a basis for delivering requirements documentaries.

  • Whiteboard(s)
  • Wiki or requirements gathering tool ( Jira for example ).  A tool that can track versions.
  • High quality video camera(s) with sufficient battery life and backup batteries.
  • Digital camera to capture whiteboard sessions.

The Role: From Requirements Manager to Producer/Orchestrator

The role to facilitate requirements documentaries requires some new skills.  I’m calling this role “Producer” or “Orchestrator”.   But I don’t want everyone getting caught up in the title here.  It’s the skills and abilities of this role that are important.  Those are:

-Video production:  a solid understanding of digital video technologies ( hardware & software ) is important.  But additionally the person should have some training/understanding of how to make a documentary.  This would include how to edit the videos for applicability, ease of understanding, and dissemination to a heterogeneous audience.   Furthermore, this role would understand lighting, how to stage a scene, and related concepts in film/video production today.

-Facilitation & Coordination:  any good requirements manager today has to have some level of facilitation and coordination skills.  This will be instrumental when discussing the requirements, but also important in getting the right folks together to produce the documentary.  Discussions could veer off course, and it will be important for the orchestrator to guide the group back to the topic so as to make the video relevant.

-Detail Oriented:  goes without saying, but with the video medium, details may need to be clarified through, what I’ll call, adjunct clips and whiteboard captures.  So attention to detail doesn’t change, but the mediums of collection and dissemination may pose challenges for a traditional requirement’s managers.

-Organized:  just like today’s requirements the videos will need to be stitched together in some kind of wiki and organized in a manner that coincides with the software release.  Any adjunct items ( whiteboard captures, additional videos ) will need to be issued as modifiers to the original video.  In addition the orchestrator will need to keep his meetings, resources, and notes well organized.  More preparation may be needed to gather requirements in this fashion, but the benefit will be the greater context it brings to the developer’s work.

-Creative:  a little creativity to make the videos more enticing and “watchable” would be valuable.  However, we need to be careful here.  Too much creativity and the videos become more of a movie than a documentary.

-Provacateur:  a good requirements manager today thinks through the questions and challenges those in the room on their assumptions.   We still need this skill and it becomes even more important with video production.   Unless we want to issue continuous sets of adjuncts after each requirement documentary; we’ll need to flesh out as much as we can up front.   Please note, that I’m not suggesting BDUF here.  We can have 5,6, or however many requirements documentaries we need to form a release, but each of those main documentaries may have many adjuncts that clarify or modify anything missing in the main video.b

Method & Utilization

The process below could be done in an iterative or waterfall fashion.  It should be methodology agnostic.  My recommendation would be to start using this on a small, non-critical project in your enterprise.  See how it works and decide how to scale it from there.  I see this as a technique pattern for situations where your requirements may change frequently, have a complex and intricate data domain, or your team is global.  Lastly, I do realize this process is fairly basic……my intention is to start high level and as we see what works/doesn’t work we iterate back through and add more details or changes.

Potential Issues

Fear of the Camera – This is probably one of the most difficult issues.  Some folks have trouble speaking up in meetings…let alone in front of a camera.  So a sufficiently skilled orchestrator should recognize when this is occurring and attempt to bring that person into the conversation or, if completely unwilling to talk up, the orchestrator should pull that person aside, after the meeting, and have a candid discussion about whether they would like to continue participating.  It may be necessary to get a proxy to stand in for him/her.

Change – Using requirements documentaries is a big change in format.  It won’t go smoothly at first and that’s why I recommend approaching it on a small non-critical project at first.   Work out the kinks, and the issues with a group that sees the potential benefit.   Use all your change management skills and recognize the loss cycle associated with any big change.  Some may see it as opportunity, and yet others will see it as a threat.

Equipment problems – one of the reasons I stress using 2 cameras ( video and still ) for recording the requirements events:  Murphy’s Law.

Legal, Organizational Policy Issues – in some countries and companies recording by video is against the laws or corporate rules.  Check with your counsel before embarking on a project using a requirements documentary.   Legal representatives and corporate leaders who see value in this method may need to sit down and amend rules to allow this form of requirements gathering.

Politics – Recording things on video is a way of preserving the context around the requirements in addition to the actual meat, logic of the system.  It’s intended benefit it to preserve for future team members and current team members the mood, background, motivations, and reasoning behind what they are building.   This is the ideal side, but there is a less rosy edge to this.   Politics arise in almost any company, and documenting decisions on video could open the door to misuse for devious ends.   A company’s leadership embracing requirements documentaries should recognize this and put controls in place for those seeking political gain from manipulating the format.

Summary

Requirements risk exists on all projects to a greater or lesser degree.  The intention of this technique is to help mitigate that risk and provide continuity through time for a system’s definition.  I don’t suspect this will alleviate all requirements issues.  I’m too jaded by experience to think there are panaceas to every problem.  But there is some good evidence in the world of psychology and Hollywood that motion pictures are more memorable and understood more easily/rapidly.  You can see this for yourself.  When you left the theatre after seeing “The Green Lantern”  did you not understand it?   Was it totally lost on you?  Now what if I drafted that movie into a requirements document?  Logic aside…my hope is that we can catch our requirements gathering process up with the technologies that we have today for gathering that information.   If you try this technique….feel free to comment and let me know your experiences.  I’ll be anxious to hear them.


Agile or Waterfall — Aren’t We Just Struggling With Requirements Understanding?

Who is this article for?

All software development professionals will find interest in this article, but those managing, crafting or analyzing requirements may find the most interest.

Introduction

Why is so much ceremony needed around software development?  Regardless of the methodology there is a great deal of meeting, clarifying, verifying, demonstrating, and checking.   If you came from another planet and watched how we develop software you’d come to the conclusion that we don’t know how to communicate complex abstract ideas into concrete reality.  And we don’t. To be sure, we’ve tried:  detailed specifications, UML diagrams, use cases, user stories, and just plain discussion.  All of these are incremental changes to a better requirements landscape, but fleshing out our software needs still takes considerable time and effort.

Process and Methodology as Requirements Risk Mitigation

BDUF ( Big Design Up Front ) was the waterfall method to define requirements.   It was a way of saying….”Let’s get everything straight BEFORE we start the expensive process of coding and testing all this.”    It’s logical, but makes the broad assumption that people will know what they want and deviation from the initial plan will be minimal.   There are some projects where this is a very safe assumption, but not always.

Agile capitalized on this fault and said “You can’t know everything up front, so just get started and course correct along the way.”   Does this mean that agile projects always succeed? No.  It means that they allow adaptation to change.  Projects where innovation is important or where the product owner is unfamiliar with developing software naturally make good candidates with Agile.  But its pricey and if you don’t need all the involvement…..then going back to waterfall might be a good option.

With both methodologies you see an attempt to address the real problem:  understanding and materializing requirements so that we get exactly what we want.   The abstract nature of software means we’re collectively imagining what it could and should be.  But we haven’t found a silver bullet for this type of collective dreaming.

Communicating and interpreting our desires is not as easy as it seems.  Some of us are good at drawing pictures…..but most of us aren’t.  To mitigate the possibility that we’re not going in the right direction we’ve invented these elaborate software processes ( Waterfall & Agile …..and the pragmatic others ) to keep our collective minds on the same path to vision creation.

Are  There Better Ways?

Even with these processes we still end up in projects where we hear these things said:

“Oh…that’s not what I meant.  It should do this…”

“No, no,no….you didn’t understand what I was saying. “

“That wasn’t the requirement.  Read it again….I built what was asked for.”

“Yes. I know she said that….but what she really meant was…”

So, software requirements are still an error prone bottle neck in producing the product.  If we hope to make software development better;  then we must get better at this.  In paraphrasing Eliyahu Goldratt in The Goal: You can only run as fast as your slowest player.  That player for software is requirements gathering, analysis, understanding and verification.

Looking outside our field, other industries, who’ve been around longer and develop complex systems too, have similar challenges.  What do they do?

Design –  My wife is a talented contract graphic designer for a local St. Louis firm called Kuhlmann-Leavitt.  They’ve won many awards and have very satisfied customers.  All this despite the nebulous, subjective nature of graphic and three dimensional design.   How do they do it?  The design field is interesting.  They don’t estimate anything.  They simply ask that you hire them for your artistic work and set a budget.  They’ll do the best design work they can for that money.  No more, no less.   So right away the budget issue is taken out of the picture.  Lastly, they don’t ask for detailed requirements,  just some idea of what you’re looking for.  You’re not hiring Kuhlmann-Leavitt to build.  You’re hiring them to create.  My bolding hints at the nuance.  Considerable authority and autonomy is given to the designers.  This is just expected.  Software development relies much more heavily on ‘domain experts’.

Manufacturing – In the manufacturing world BDUF could be rewritten to be MDUF ( Massive Design Up Front ).  Not only is the product designed way up front and in detail, but the process and mechanisms to build the product, which aren’t always the same, are designed up front too.  Before the real product even undergoes one production run…many small demo runs are carried out.  The results are stunning.  Once the product starts rolling off the assembly line its produced to exacting tolerances.  It mostly fits the requirements of marketing and can be replicated thousands of times over.

Construction – We’re talking about unique, one of kind construction here.  Not production line assembly of a subdivision.  This one-off building is, to my mind, the most similar to software.  Some requirements come from regulations and building codes.  Others come from the purchaser of the structure.  There are multiple stakeholders.  Requirements are digested, and then artifacts are created.  A series of mock-ups are assembled and based on this a cost estimate is created.  Even with all the up-front planning there are always “gotchas” and changes that are thrown into the project.

In all three of these examples the end product is visual and intuitive which helps with requirements understanding.  A picture really is worth a thousand words.  Software, as workflow, data and creative design is more abstract.  We’re trying to craft ideas and concepts into something that is somewhat tangible.  Certainly the user interface can be mocked in a similar way and there is great value that comes from the discussions around this, but a sufficiently detailed workflow and data set always serve to vex our attempts to provide clarity to requirements.

What’s next?  What can we do?

Here’s part of our problem:

1. We discuss the first iteration of requirements with our product owners.  The person leading that discussion is gifted in facilitating this.

2. From this meeting, the requirements are crafted into user stories.

See it?  We just lost all the context, emotion, body language, nuance, and reactions by everyone else by drafting something into writing.  I don’t care how good a writer you are…you can’t replicate the requirements conversation.  So why do we try?

The next problem comes when you ask each person to interpret the written user stories.  Hopefully each team member was present for the conversation so they could somewhat remember what transpired.  But, even with attendance, the human memory is frail and human perceptions, bias and concentration vary. Each person may see and read the requirements slightly different.  In short there are too many lenses.

Rather than try to abstract requirement conversations into another form ( writing ) or create even more abstract artifacts ( UML, Use Cases ) we should seek to record them via video, photographs of whiteboard drawings, and publishing all this to some kind of requirements wiki or blog.  Doing this for each iteration review, iteration planning, or project meeting would surely capture a much more complete document of the project’s scope, changes, stakeholder motives and historical background.  Further, it would exist in perpetuity for future developers to watch……just like a movie or project documentary.

Summary

I do realize that the suggestion to move towards requirements documentaries may be fraught with organizational, legal, and other challenges.  I would never suggest that change on this scale is easy.   But I do believe there is productivity savings to be found here.  As I hinted in Decline of Written Requirements  I think we’re using 20th century tools and techniques to gather and analyze requirements in a 21st century world.  To lessen the need for process mitigation of requirements risk we may need to refocus and leverage our new tools: cell phone cameras,  high quality digital video, wikis, and blogs.