The Coming Breakup of Scrum


Scrum, probably the most widely adopted agile software development practice, is cracking up.  The signs are evident and this article discusses the trend and what it means for agilists today.  If you practice scrum, are thinking of using it, or have used it in the past you’ll find interest in this article.

Oh yeah?  Prove it!!!

It’s a truism that rapid growth and popularity put a strain on organizations, movements, ideas…….or ,more fundamentally, the people comprising these very human vehicles.  As an idea ferments and then exponentially amplifies to encompass a wider politic of devotees and admirers there is the impending fog of dissent with the origin, the core.  From this comes splintering, debate and eventually more novelty and innovation.

Fine, but what about direct evidence for the rift in scrum?

03/05/2009 – Scrum But…Test.  Jeff Sutherland.

09/15/2009 – Ken Schwaber resigns from Scrum Alliance

06/15/2010 – Ken Schwaber establishes

04/20/2011 – The CSM certification wars

07/26/2011 – Forrester calls Water-Scrum-Fall the norm

08/02/2011 – Bloggers Suggesting Scrum Variations

10/06/2011 – Scrum Extensions announced – allowing modification to basic scrum.

As the agile movement hybridizes and evolves scrum is feeling the brunt of this pain. The forces of hybridization are ripping scrum into distinct camps whose views are aligned with their respective organizational needs.  Who are these camps and what will ultimately happen to scrum as this spider web is tugged from it’s anchors?

The Camps

The Enterprise – Most enterprises don’t build software as a core part of their operations.  Projects here are usually centered on delivering some base set of functionality for a fairly fixed price.  Stability and low cost of operation are prized.  The organization doesn’t usually see this as an investment…but as an expense to be managed and tracked.  The pressures enterprise software development groups face at the capital planning phase and the release phase have sculpted the Water-Scrum-Fall that’s prevalent.

The Software Companies –  In contrast to the enterprise, software companies build software as their core product.  It *IS* their operation.  Software companies see their products as an investment and it’s life cycle is fairly unlimited, or at least tied to the life-cycle of the company.  Pure scrum works in  these companies fairly well.  There’s usually still pressure to reduce the number of production releases but on the planning side managing features in a backlog fashion is workable, even preferable when you have a product that has a potentially unlimited lifespan.

The Startups – Startups face a different set of pressures from their venture capital investors.  The need to rapidly introduce new features and create value to customers and investors by gaining a competitive advantage quickly is paramount.  Startups are tearing scrum towards the Lean Startup methodology professed by Eric Ries.   The focus here is continuous delivery of new software: stability be damned.  Usually these startups don’t build software that’s mission critical or life threatening.

The Purist Methodologists – Among these are the various scrum trainers, agile coaches and general agile philosophers who profess a puritanical approach to scrum.  Their insistence on a dogmatic approach to scrum are rooted less in practice and more in theory.

What are they doing to scrum…where are they taking it?

These groups are doing what’s natural.  They’re catering scrum to their practice and profession of application life-cycle management ( ALM ).  But this tug of war for scrum’s future is changing it.  Instead of a simple, one-size fits all methodology, it’s becoming a framework of patterns & practices.

You can see this through the introduction of scrum extensions, which is a natural way of recognizing the various groups that have adopted scrum in some fashion.  With scrum becoming an umbrella concept, it’s importance is melting away.  It will devolve and eventually disappear. Instead the extensions, much like design patterns in software development, become the value added pieces that development teams will use and rally around.  Each camp will likely pool certain extensions, patterns as their ‘methodology’.   In time, these flavors of scrum will tear it completely apart.


Scrum’s popularity will likely be its undoing.  But is this something to be avoided or stopped?  Hardly.  Hybridization and evolution of scrum is a natural process that is both pragmatic and necessary.  Jeff and Ken’s baby is growing up and leaving the agile house.  The original agile manifesto signatories accomplished their mission; they broke the one size fits all ALM world down.  The new heterogeneous landscape of practices and patterns, while less clean than the bi-polar world of agile vs waterfall, is primed with greater opportunity for innovation.


Santa’s Workshop Goes Agile!

Among the aroma of fruit cake, hot cider, and sub-zero temperatures Santa Claus announced today that his North Pole Workshop would switch to using agile practices in toy production.  Santa sighted a number of reasons for the change, among them:

  • Increasing need to be first to market against growing competition.
  • Better collaboration between kids’ toy lists and toy developers.
  • Reducing the risk that kids don’t get what they want.
  • Increasing quality and accountability in toy development and delivery.

According to Santa, “We’ve analyzed this decision for some time and  it became clear that change was needed after re-reading little Timmy Todsnockerdellturtlefoof’s complaint letter from last year.  Instead of receiving a bicycle, as stipulated in his toy list; he got two fake icicles.”

The well publicized failure last year to meet the requirements of Timmy’s wish list was not the first time Santa’s North Pole Workshop hadn’t delivered, but in this day of social media and instant updates; the story took on a vicious viral characteristic….hitting #1 in the tweetosphere for a full month with the hash tag: #TimmyTodsnockerdellturtlefoofGotIcedBySanta.

When asked what type of agile practice the workshop would be adopting, Santa replied: “Scrum.  It’s well known, proven, and being used in some isolated corners of our workshop today.”  Some muted ‘boos’ could be heard from the audience.

Santa went on to say that Rudolph The Red Nosed Raindeer would head up the agile transformation of the North Pole Workshop in conjunction with an outside agile consulting firm to be named later.  That development confirmed for some that Rudolph was next in line to take over The North Pole.  Rudolph was on hand to comment to reporters after the announcement:

“We’re looking at one of the big 5 consulting firms, but we haven’t ruled out the smaller players.  Look, what’s important here is that we get to done. I have a nose for this kind of thing.  I’m not a dreamer..I’ve been a servant leader for a long time…I realize this transformation isn’t going to happen overnight, there will be pain and there’s a lot of existing process and procedure that will have to….well, frankly….go away.  But, one step at a time. I expect we’ll have a more detailed plan in May or April.  Talk to me then.”

Reactions among the North Pole elves was varied.  One older elf man pointed out that Santa had to be dragged “kicking and screaming to the decision.  He wasn’t on board at all.  Rudolph, Frosty, Mrs Claus and the Yeti really had to sell him on it.  When they broke out Timmy’s letter to remind Santa of the increasing defect rates in his shop….he went ballistic. I mean he really lost his cookies.  Then the Yeti put his foot down.  I think Santa knew what that meant.”

Others, like a younger elf woman, had a different opinion on the switch to agile:  “I’m fine with the whole agile thing, but I guess I don’t understand why they chose Scrum?  I mean….from what I read XP is way better and more applicable to our environment.  I really don’t think the elves, particularly the older ones, will like daily stand-ups. I mean most of them can barely wake up every day….let alone stand up.”

Still another view was held by Donner, “What about Kanban?  No one’s talking about Kanban, but in the reindeer house we use it all the time to limit HIP ( hooves in process). I bet Rudolph moves us there.  I think there’s going to be a power play here between Santa and Rudolph.  It’s a battle that’s been brewing for centuries.”

“Give me a break. Agile?  Really, let’s knock off the buzz and hype.   So we goof up on a few thousand toys out of the billions we make a year.  How’s agile gonna solve that?  I don’t get it.  I guess I’ll ride it out and see where this goes.  But limiting HIP in the reindeer house has done nothing but give some of us more time on our hands.  I don’t think that’s what Santa wants.”  said Blitzen.

Frosty, while at the announcement, declined to make any official comment but did say that he favored a balanced, pragmatic approach, one that would focus on the workshop’s needs rather than a dogmatic approach.

Mrs. Claus had this to say: “Rudolph is very bright.  I know he’ll make this work.  He knows where he’s going.”

Strangely, the Yeti, was not present.  But his footprint could be found in the comments that others made.

Outsiders also came to hear the announcement.  The Easter Bunny had this to say: “I understand what Santa’s doing and to be perfectly frank he’s in a different position from us in Easter Hollow.  He’s facing some real time to market issues and competition with Mom & Dad, Grandpa & Grandma, and others.  His competitors have real advantages in local sourcing, customer relationship management, and technology.  Santa just hasn’t kept up and now it’s time for a radical change.   I don’t think anything he’s doing is going to change our approach.  We pride ourselves on stability of delivery and with a 98.2% market share on Easter…we’re just not facing the same issues.”

The Easter Goose has the other 1.8% of the Easter market.

Merry Christmas agile community.  Enjoy your holidays, stay safe, and as always….take care. 😉

KanBan: Limiting WIP….But To What End?


The Kanban technique is popular within software development circles.  It borrows from the world of manufacturing where the original Kanban method is used to govern and regulate the production by any one operation so that it works in tandem with the overall production process and plan.  This article looks at how Kanban limits work in process inventory in software development and seeks to understand the value of doing this.

Work in Process in Manufacturing

In manufacturing Kanban is used to limit WIP inventory in manufacturing processes. But why is this done?  What’s the problem with WIP inventory in manufacturing? Isn’t having enough work and supplies for the next operation in line a good thing?

The answer centers on cost.  Excessive amounts of WIP inventory in manufacturing cost money.  Kanban isn’t about getting faster, or delivering a better quality product…it’s about not wasting materials, machine cycles and time creating something that won’t be needed. Overproduction of WIP inventory creates these costs:

1. Spoilage – too much work in process inventory can result in wasted goods ( especially food production ) because the downstream operation doesn’t need all the WIP inventory that was produced.

2. Overproduction / over purchasing of a part – If an operation in a manufacturing plant routinely produced more widgets than were needed by the next operation using them then the purchasing department is probably acquiring more raw material for that one operation than is actually needed by the overall production schedule.  This would result in the company having extra raw materials, which are often ordered in large lots for price optimization.

3. Wasted Operator Time  – This cost is a bit of a paradox.  Look at an operation that overproduces and it may appear in isolation that it is productive by a simple ratio of units produced/hours worked.  But, in truth the operation is just wasting the operator’s time because what he/she is creating isn’t needed.  The operator could be used elsewhere in the plant doing other productive work or even be sent home to help reduce labor costs ( assuming they are paid by the hour or unit produced ).  The wasted operator time may actually cause overstaffing as well.

4. Wear and Tear / Unecessary Maintenance on Machinery. – overproduction means we use our machinery unnecessarily for anything over the necessary production level.  The additional maintenance and wear and tear on the machinery can get expensive depending on the operation.

5. Increased Storage Costs – Excessive WIP inventory may need to be stored somewhere beyond the immediate operating space for a particular operation.  This usually means stocking the excessive WIP in a warehouse.  Warehouses cost money to build, operate and maintain.  By limiting WIP and adjusting cycle time and throughput through a plant you can eliminate the need for a warehouse and move toward a JIT system with your suppliers.

Does Limiting WIP inventory in Software Have a Similar Corolary?

First, what is WIP inventory in software development?  In a very thorough article on InfoQ WIP for software development is considered to be “tasks actively being worked on”.  It’s interesting to note how Kanban flips scrum upside down.  In Scrum we size the work to fit a time-boxed iteration.  In Kanban we size the time to fit a work-boxed unit.

But what are the costs associated with overproduction of WIP inventory in software development?  Are there any?  Let’s model this from the manufacturing world first and then see if we can find other costs that may be unique to the application development space:

1. Spoilage – It’s hard to say a requirement, feature, task, or idea would ever become ‘spoiled’ in the strict definition of that word.  You might be able to say that a requirement is no longer relevant or needed.  So perhaps there is some cost associated with the business analyst over-soliciting enhancement requests that will never be built.  But most new software development efforts have some baseline set of requirements that must be produced for the system to be coherent.  So letting the Business Analyst work ahead to get those nailed down in an efficient and comprehensive way that speaks to designing a ‘whole system’ is probably worth the expense.  Further, software development is not manufacturing.  It’s new product development and has an element of creativity.  This element of creativity requires some waste.  All creative efforts do.

2. Overproduction / over purchasing – If the developers over produce on tasks, resulting in a large WIP for the QA team, does this in turn cause a business analyst to generate even more requirements?  Maybe. If you’re Facebook and you’re always adding new features to your product then software development can be seen as a production line.  It’s a never ending stream of work for Facebook which probably requires some regulation of workflow to balance necessary resources.  But if you’re Pepsi and you have a simple capital project that will end when the funding runs out…then there is probably a finite set of requirements and you run no risk of overproducing on those requirements.

3. Wasted Operator time – There may be cost here if the business analyst, programmer, or QA professional are working on requirements of no real value or the requirement is not needed.  Does this happen?  Sure.  But I don’t see how Kanban prevents this from occurring.  Sometimes low value or worthless features are requested, everyone agrees they should be built, and it doesn’t become obvious until after completion that the feature should never have been added.

4. Wear and Tear / Unecessary Maintenance on Machinery –  This cost is hard to visualize for software development.  What would be the equivalent? Too much pounding on the keyboard?  Too much use of the CPUs?  Given that the computer would probably be used for something else if it were not being used by the developer/qa professional in association with their direct work; it seems there is a very minute wear and tear cost.

5. Increased Storage Costs – I can’t think of any storage costs that would be born with tasks that are not completed.  It costs virtually nothing to save these on whatever storage device houses them.  In fact the cost of storing anything in memory has been going down for decades.  Further, it’s likely the storage costs are spread across many budgets as more than one system probably shares the storage space.

So, some minor cost savings may come from limiting WIP in software development, but what should be very clear to any business professional is that Kanban in software development isn’t focused on achieving the same goals it achieves in manufacturing.

Kanban in manufacturing is seeking to limit the costs associated with overproduction of WIP.  Inventory costs are significant in manufacturing and need to be controlled, managed to limit their cost in the final product.  Kanban is a proven technique for this.

Kanban in software is seeking to limit WIP, but not for cost reasons.  The major cost in software development isn’t inventory.  It’s hours worked by people.  That cost comprises 90-95% of the cost for a finished software product.  So does Kanban in software limit hours worked?

Continuous Flow?

Software Kanban models the production flow of tasks associated with a software development effort.  Rules are established for production.  As an example:  the development team might set a limit of 3 done tasks as their kanban.  In other words they won’t produce any more tasks until at least one of the 3 done tasks is taken by the QA team for review.  This is a pull system.  Downstream processes determine the production of the upstream processes.

What makes this model successful in manufacturing is something called ‘cycle time’.  Cycle time is the normalized time required to produce a widget.  In most manufacturing processes cycle time for any one operation is well known.  This makes a pull process ideal.

In software development cycle time for any one task is variable.  This means setting a work in process limit on tasks will not necessarily guarantee continuous flow.  Further, in manufacturing a widget sent to a downstream operation is ‘complete’.  It’s not waiting on any further refining by the upstream process or any additional widgets.  In software development a task that is done isn’t necessarily testable.  It may require execution of a 2, 5, or 10 tasks for a piece of work to be testable.  So even though the QA team accepts additional work tasks from development, they may not be able to do anything with them.

Thinking through those points, it doesn’t appear that Kanban would limit hours worked by anyone.  In fact, it may actually do the opposite and increase the cost of the software development process as upstream operations wait until the downstream process completes work.  This idle time, might be seen as valuable in  allowing a team of developers, business analysts, or testers to switch to another project while waiting for completion of downstream tasks on an existing project.  But this then brings up the question of context switching and introduces even deeper complexities, entanglements into the production process.

Visibility of Workflow…the Real Value

So if Kanban doesn’t necessarily yield a more efficient development effort then what is the value?  Is there any?  The Kanban board is used to model the production flow of software development through its major person roles:   business analysis, development, QA, acceptance testing, and release management.  This board is visual and if kept up to date gives an interesting view for anyone hoping to manage the software development process.   At a quick glance the team can easily see where things are potentially bottle-necked and begin to focus on how best to free up that bottleneck.  This is the strongest value Kanban offers to software development: kaizen on the flow of tasks.


Does this article suggest Kanban is misguided in its implementation in software development?  Not at all.  Attempting to borrow techniques from other industries that have been successful is a useful, pragmatic approach.  Modeling the flow of tasks between processes has value in just opening up questions about why something isn’t done and what’s blocking completion.  However, this article does suggest that hoping to see significant, immediate cost savings in your development process using Kanban may be a false hope.  The true value will be unlocked by continually focusing on how best to increase the number of requirements produced per hour by the team.

The Irony of Doing Agile

J.D. Hildebrand wrote this piece over at SDTimes.  He quoted Andy Hunt, one of the original Agile Manifesto signatories:

Last summer, I had the good fortune to visit a “very advanced” Agile shop. These folks really did embrace agile methods with a discipline, completeness, and a zealous fervor that would be hard to match. In many ways, they could have been a poster child for Agile methods…But these weren’t productive developers freed from mindless process dogma. They were Agile slaves. The dogma they followed was ours, and they followed it well. And as with many organizations in a similar position, they saw some promising results. Continuous integration, refactoring, unit tests, pair programming—all these techniques yielded some benefit. But they weren’t thinking, they weren’t reacting, they weren’t being agile. When problems came up, they addressed them with all the grace and elegance of a deer caught in the terrifying blaze of alien headlights. They knew how to do Agile; they didn’t know how to be agile.

Individuals and interactions over processes and tools.   ( including processes and tools labeled ‘agile’.).  Stop agilizing everything.