Tag Archives: pragmatic

The Root Cause of Water-Scrum-Fall

Posted on by
5

Introduction

Water-Scrum-Fall is the norm in many organizations today.  Despite the attempts of scrum coaches and consultants, the weeds of waterfall grow back into the agile garden.  What causes this?  This article looks at the root cause for the water-scrum-fall phenomenon and makes a suggestion about how to address it.

The Root Cause

Water-scrum-fall’s reality is not the result of people being unwilling to adopt scrum.  It’s not the result of a lack of passion for agile processes and practices.  Nor is it caused by a lack of executive support.  The cause?

Capital budgeting

In companies that produce software for internal use water-scrum-fall finds it’s greatest adoption.  Internally used software is strictly accounted for by the regulations in SOP98-1 and this financial machinery is what guides the need to plan up front.

Capital projects are investments.  To determine an investment’s return you need to know the estimated initial cost, and estimated revenue ( or savings ) the project will create.  These things are estimated up front so that a decision can be made on whether or not to pursue the investment.  The up-front nature of capital budgeting compliments waterfall and BDUF. It is a core financial business process guided and regulated by FASB.  Think about that for a moment…and then read on.

Scrum practitioners run up against a wall with capital budgeting.  It doesn’t fit their operational practice for developing software.  Under scrum….we shouldn’t be designing, estimating and crafting the project up front.  Instead we should approach it incrementally.  The problem with this? It ignores how enterprise software development projects ( capital investments  ) are funded.  The result is that everyone compromises and innovates.  Water-Scrum-Fall is the child of this compromise.

The Challenge

Scrum and other agile practices pose a challenge to enterprise software development efforts and the capital budgeting process.  Indirectly they say “Why are we funding this as a capital investment?  It’s not.  It’s an ongoing operational cost and should be accounted for that way.  If we don’t plan on funding this software development effort for the long haul…then why are we doing it?”  Funding a software development effort as an operational expense, as is done within software companies, does fit the scrum operational practice better.  But again…the difference between a software development effort being labeled CAPEX or OPEX is guided by FASB.  It’s not up to the company.

How Do We Fix Water-Scrum-Fall?

I don’t think there’s a silver bullet here.  But in my last article, Is There a Better Way to Estimate Capital Projects? , I threw out a suggestion for how to estimate a capital investment using tolerances.  This bypasses the need for an up-front detailed analysis of what the the LOE ( Level of Effort ) would be for the project, but still gets the business what it needs: an initial funding point and resulting NPV that augments decision making.

Summary

So water-scrum-fall is a pragmatic reaction by agilists and IT professionals to work with the business and its financial processes.  Did the originators of scrum not understand the capital budgeting process?  Were they oblivious to the financial architecture of the businesses around them?  Maybe…but to their credit; they weren’t trying to address this.  Their focus was on how to do software in an adaptive fashion so that it more organically addressed the operational realities of manifesting a complex vision.

Advertisement

Santa’s Workshop Goes Agile!

Posted on by
2

Among the aroma of fruit cake, hot cider, and sub-zero temperatures Santa Claus announced today that his North Pole Workshop would switch to using agile practices in toy production.  Santa sighted a number of reasons for the change, among them:

  • Increasing need to be first to market against growing competition.
  • Better collaboration between kids’ toy lists and toy developers.
  • Reducing the risk that kids don’t get what they want.
  • Increasing quality and accountability in toy development and delivery.

According to Santa, “We’ve analyzed this decision for some time and  it became clear that change was needed after re-reading little Timmy Todsnockerdellturtlefoof’s complaint letter from last year.  Instead of receiving a bicycle, as stipulated in his toy list; he got two fake icicles.”

The well publicized failure last year to meet the requirements of Timmy’s wish list was not the first time Santa’s North Pole Workshop hadn’t delivered, but in this day of social media and instant updates; the story took on a vicious viral characteristic….hitting #1 in the tweetosphere for a full month with the hash tag: #TimmyTodsnockerdellturtlefoofGotIcedBySanta.

When asked what type of agile practice the workshop would be adopting, Santa replied: “Scrum.  It’s well known, proven, and being used in some isolated corners of our workshop today.”  Some muted ‘boos’ could be heard from the audience.

Santa went on to say that Rudolph The Red Nosed Raindeer would head up the agile transformation of the North Pole Workshop in conjunction with an outside agile consulting firm to be named later.  That development confirmed for some that Rudolph was next in line to take over The North Pole.  Rudolph was on hand to comment to reporters after the announcement:

“We’re looking at one of the big 5 consulting firms, but we haven’t ruled out the smaller players.  Look, what’s important here is that we get to done. I have a nose for this kind of thing.  I’m not a dreamer..I’ve been a servant leader for a long time…I realize this transformation isn’t going to happen overnight, there will be pain and there’s a lot of existing process and procedure that will have to….well, frankly….go away.  But, one step at a time. I expect we’ll have a more detailed plan in May or April.  Talk to me then.”

Reactions among the North Pole elves was varied.  One older elf man pointed out that Santa had to be dragged “kicking and screaming to the decision.  He wasn’t on board at all.  Rudolph, Frosty, Mrs Claus and the Yeti really had to sell him on it.  When they broke out Timmy’s letter to remind Santa of the increasing defect rates in his shop….he went ballistic. I mean he really lost his cookies.  Then the Yeti put his foot down.  I think Santa knew what that meant.”

Others, like a younger elf woman, had a different opinion on the switch to agile:  “I’m fine with the whole agile thing, but I guess I don’t understand why they chose Scrum?  I mean….from what I read XP is way better and more applicable to our environment.  I really don’t think the elves, particularly the older ones, will like daily stand-ups. I mean most of them can barely wake up every day….let alone stand up.”

Still another view was held by Donner, “What about Kanban?  No one’s talking about Kanban, but in the reindeer house we use it all the time to limit HIP ( hooves in process). I bet Rudolph moves us there.  I think there’s going to be a power play here between Santa and Rudolph.  It’s a battle that’s been brewing for centuries.”

“Give me a break. Agile?  Really, let’s knock off the buzz and hype.   So we goof up on a few thousand toys out of the billions we make a year.  How’s agile gonna solve that?  I don’t get it.  I guess I’ll ride it out and see where this goes.  But limiting HIP in the reindeer house has done nothing but give some of us more time on our hands.  I don’t think that’s what Santa wants.”  said Blitzen.

Frosty, while at the announcement, declined to make any official comment but did say that he favored a balanced, pragmatic approach, one that would focus on the workshop’s needs rather than a dogmatic approach.

Mrs. Claus had this to say: “Rudolph is very bright.  I know he’ll make this work.  He knows where he’s going.”

Strangely, the Yeti, was not present.  But his footprint could be found in the comments that others made.

Outsiders also came to hear the announcement.  The Easter Bunny had this to say: “I understand what Santa’s doing and to be perfectly frank he’s in a different position from us in Easter Hollow.  He’s facing some real time to market issues and competition with Mom & Dad, Grandpa & Grandma, and others.  His competitors have real advantages in local sourcing, customer relationship management, and technology.  Santa just hasn’t kept up and now it’s time for a radical change.   I don’t think anything he’s doing is going to change our approach.  We pride ourselves on stability of delivery and with a 98.2% market share on Easter…we’re just not facing the same issues.”

The Easter Goose has the other 1.8% of the Easter market.

Merry Christmas agile community.  Enjoy your holidays, stay safe, and as always….take care. 😉

The Irony of Doing Agile

Posted on by
3

J.D. Hildebrand wrote this piece over at SDTimes.  He quoted Andy Hunt, one of the original Agile Manifesto signatories:

Last summer, I had the good fortune to visit a “very advanced” Agile shop. These folks really did embrace agile methods with a discipline, completeness, and a zealous fervor that would be hard to match. In many ways, they could have been a poster child for Agile methods…But these weren’t productive developers freed from mindless process dogma. They were Agile slaves. The dogma they followed was ours, and they followed it well. And as with many organizations in a similar position, they saw some promising results. Continuous integration, refactoring, unit tests, pair programming—all these techniques yielded some benefit. But they weren’t thinking, they weren’t reacting, they weren’t being agile. When problems came up, they addressed them with all the grace and elegance of a deer caught in the terrifying blaze of alien headlights. They knew how to do Agile; they didn’t know how to be agile.

Individuals and interactions over processes and tools.   ( including processes and tools labeled ‘agile’.).  Stop agilizing everything.

Agile vs Waterfall vs Lean Startup : Who Cares About the Labels?

Posted on by
3

Labels

After reading Shane Hastie’s latest InfoQ article:  Lean Startup or Agile or Lean Startup and Agile?  it became clear how much we’re all getting caught up in labels and methodologies.  My thought:  let’s drop the flag waving, become pragmatic in our approach and recognize that context plays a crucial role here.   What works for Eric Ries and his start-up may not work for the company you work for and the project you’re currently executing.

The Real Value – Techniques & Tools

The value in methodologies like Agile, Scrum,  Lean Startup,  Waterfall and others is in the techniques and tools they bring to the table.  While they are each packaged with a neat title and some intelligent proponents; we don’t need to use these in a take it or leave it fashion.  Be pragmatic and assemble your own toolkit.  I’ll reference Scott Ambler and say “context matters“.  We all need to move towards a collection, library of project and product management patterns that can be adopted within certain contexts.

Innovate 

Lastly, what’s invigorating about these debates is that we’re growing as a field.  We’re rethinking our ideas, and not settling.  This can only be good for software development and those affected, infected by it.  Yes, I meant to say “infected”.  🙂

Agile or Waterfall — Aren’t We Just Struggling With Requirements Understanding?

Posted on by
1

Who is this article for?

All software development professionals will find interest in this article, but those managing, crafting or analyzing requirements may find the most interest.

Introduction

Why is so much ceremony needed around software development?  Regardless of the methodology there is a great deal of meeting, clarifying, verifying, demonstrating, and checking.   If you came from another planet and watched how we develop software you’d come to the conclusion that we don’t know how to communicate complex abstract ideas into concrete reality.  And we don’t. To be sure, we’ve tried:  detailed specifications, UML diagrams, use cases, user stories, and just plain discussion.  All of these are incremental changes to a better requirements landscape, but fleshing out our software needs still takes considerable time and effort.

Process and Methodology as Requirements Risk Mitigation

BDUF ( Big Design Up Front ) was the waterfall method to define requirements.   It was a way of saying….”Let’s get everything straight BEFORE we start the expensive process of coding and testing all this.”    It’s logical, but makes the broad assumption that people will know what they want and deviation from the initial plan will be minimal.   There are some projects where this is a very safe assumption, but not always.

Agile capitalized on this fault and said “You can’t know everything up front, so just get started and course correct along the way.”   Does this mean that agile projects always succeed? No.  It means that they allow adaptation to change.  Projects where innovation is important or where the product owner is unfamiliar with developing software naturally make good candidates with Agile.  But its pricey and if you don’t need all the involvement…..then going back to waterfall might be a good option.

With both methodologies you see an attempt to address the real problem:  understanding and materializing requirements so that we get exactly what we want.   The abstract nature of software means we’re collectively imagining what it could and should be.  But we haven’t found a silver bullet for this type of collective dreaming.

Communicating and interpreting our desires is not as easy as it seems.  Some of us are good at drawing pictures…..but most of us aren’t.  To mitigate the possibility that we’re not going in the right direction we’ve invented these elaborate software processes ( Waterfall & Agile …..and the pragmatic others ) to keep our collective minds on the same path to vision creation.

Are  There Better Ways?

Even with these processes we still end up in projects where we hear these things said:

“Oh…that’s not what I meant.  It should do this…”

“No, no,no….you didn’t understand what I was saying. “

“That wasn’t the requirement.  Read it again….I built what was asked for.”

“Yes. I know she said that….but what she really meant was…”

So, software requirements are still an error prone bottle neck in producing the product.  If we hope to make software development better;  then we must get better at this.  In paraphrasing Eliyahu Goldratt in The Goal: You can only run as fast as your slowest player.  That player for software is requirements gathering, analysis, understanding and verification.

Looking outside our field, other industries, who’ve been around longer and develop complex systems too, have similar challenges.  What do they do?

Design –  My wife is a talented contract graphic designer for a local St. Louis firm called Kuhlmann-Leavitt.  They’ve won many awards and have very satisfied customers.  All this despite the nebulous, subjective nature of graphic and three dimensional design.   How do they do it?  The design field is interesting.  They don’t estimate anything.  They simply ask that you hire them for your artistic work and set a budget.  They’ll do the best design work they can for that money.  No more, no less.   So right away the budget issue is taken out of the picture.  Lastly, they don’t ask for detailed requirements,  just some idea of what you’re looking for.  You’re not hiring Kuhlmann-Leavitt to build.  You’re hiring them to create.  My bolding hints at the nuance.  Considerable authority and autonomy is given to the designers.  This is just expected.  Software development relies much more heavily on ‘domain experts’.

Manufacturing – In the manufacturing world BDUF could be rewritten to be MDUF ( Massive Design Up Front ).  Not only is the product designed way up front and in detail, but the process and mechanisms to build the product, which aren’t always the same, are designed up front too.  Before the real product even undergoes one production run…many small demo runs are carried out.  The results are stunning.  Once the product starts rolling off the assembly line its produced to exacting tolerances.  It mostly fits the requirements of marketing and can be replicated thousands of times over.

Construction – We’re talking about unique, one of kind construction here.  Not production line assembly of a subdivision.  This one-off building is, to my mind, the most similar to software.  Some requirements come from regulations and building codes.  Others come from the purchaser of the structure.  There are multiple stakeholders.  Requirements are digested, and then artifacts are created.  A series of mock-ups are assembled and based on this a cost estimate is created.  Even with all the up-front planning there are always “gotchas” and changes that are thrown into the project.

In all three of these examples the end product is visual and intuitive which helps with requirements understanding.  A picture really is worth a thousand words.  Software, as workflow, data and creative design is more abstract.  We’re trying to craft ideas and concepts into something that is somewhat tangible.  Certainly the user interface can be mocked in a similar way and there is great value that comes from the discussions around this, but a sufficiently detailed workflow and data set always serve to vex our attempts to provide clarity to requirements.

What’s next?  What can we do?

Here’s part of our problem:

1. We discuss the first iteration of requirements with our product owners.  The person leading that discussion is gifted in facilitating this.

2. From this meeting, the requirements are crafted into user stories.

See it?  We just lost all the context, emotion, body language, nuance, and reactions by everyone else by drafting something into writing.  I don’t care how good a writer you are…you can’t replicate the requirements conversation.  So why do we try?

The next problem comes when you ask each person to interpret the written user stories.  Hopefully each team member was present for the conversation so they could somewhat remember what transpired.  But, even with attendance, the human memory is frail and human perceptions, bias and concentration vary. Each person may see and read the requirements slightly different.  In short there are too many lenses.

Rather than try to abstract requirement conversations into another form ( writing ) or create even more abstract artifacts ( UML, Use Cases ) we should seek to record them via video, photographs of whiteboard drawings, and publishing all this to some kind of requirements wiki or blog.  Doing this for each iteration review, iteration planning, or project meeting would surely capture a much more complete document of the project’s scope, changes, stakeholder motives and historical background.  Further, it would exist in perpetuity for future developers to watch……just like a movie or project documentary.

Summary

I do realize that the suggestion to move towards requirements documentaries may be fraught with organizational, legal, and other challenges.  I would never suggest that change on this scale is easy.   But I do believe there is productivity savings to be found here.  As I hinted in Decline of Written Requirements  I think we’re using 20th century tools and techniques to gather and analyze requirements in a 21st century world.  To lessen the need for process mitigation of requirements risk we may need to refocus and leverage our new tools: cell phone cameras,  high quality digital video, wikis, and blogs.