The Irony of Doing Agile

J.D. Hildebrand wrote this piece over at SDTimes.  He quoted Andy Hunt, one of the original Agile Manifesto signatories:

Last summer, I had the good fortune to visit a “very advanced” Agile shop. These folks really did embrace agile methods with a discipline, completeness, and a zealous fervor that would be hard to match. In many ways, they could have been a poster child for Agile methods…But these weren’t productive developers freed from mindless process dogma. They were Agile slaves. The dogma they followed was ours, and they followed it well. And as with many organizations in a similar position, they saw some promising results. Continuous integration, refactoring, unit tests, pair programming—all these techniques yielded some benefit. But they weren’t thinking, they weren’t reacting, they weren’t being agile. When problems came up, they addressed them with all the grace and elegance of a deer caught in the terrifying blaze of alien headlights. They knew how to do Agile; they didn’t know how to be agile.

Individuals and interactions over processes and tools.   ( including processes and tools labeled ‘agile’.).  Stop agilizing everything.

Predictive *AND* Adaptive Planning

Introduction

If you’ve managed projects for any length of time you understand the truth.  It’s almost never the case that a project is completely predictive ( waterfall ) or completely adaptive ( agile ).  It’s a mix.  There are needs for both in any project.  Those project managers, companies, and consultancies that integrate both approaches into their project management efforts are hybridizing the agile movement and shaping the reality of the future.  Let’s look at some examples that are well known, visible, historical and outside the IT market.  Why?  I want to defuse the agile vs waterfall debate for a moment and abstract things away from technology to help us see the broader picture.

Example 1: Lewis & Clark Expedition

The expedition by Lewis and Clark to navigate the Missouri River and map the western territories was an immense, RISKY, long term project entertained by a focused project team of professionals.  This is a startup.  Their goal was to map a water born ( river ) path to the Pacific Ocean.  To accomplish this goal some level up front planning was required.  You simply couldn’t plop two guys down in St. Louis and say: “For the next two weeks go up the Missouri River, then have a retrospective on what you learned, share that learning with your sponsor in Washington D.C. and plan for the next two weeks of river navigation.”

Lewis & Clark needed supplies, people who knew the land, logistical expertise, some plan up front to start the mission. But while an up front plan was necessary, much was not known.  This is where adaptive planning comes in.  They would need to adapt as maps, events, and people turned out to be unreliable or poorly understood: winters were more severe, the Missouri didn’t go all the way to the Pacific and great Mountain chain blocked their path to the ocean, and accidentally injuries and sickness delayed progress.

While not having an up front plan could have delayed and increased the cost of their project…not being able to adapt along the way would have killed it.  It was an experiment, a gamble, and to make it work required flexibility and persistence.

Standish group would have called Lewis & Clark a failure because they exceeded their baseline schedule.

Example 2:  Apollo missions to the moon.

The missions to land men on the moon and safely return them during the late 1960s and early 1970s in the United States were massive projects of unknown risk and horrible complexity.  Failure would bring down a nation and a philosophy ( democratic freedom).  They closely mirror big ERP implementations or suitably large custom software development efforts in Fortune 500 businesses today.

Could these have succeeded without predictive planning?  It’s hard to imagine they would have.  There were so many variables and unknowns that were required to be nailed down before implementation to ensure success.  In fact, predictive planning to the Nth degree is what made this possible and successful.  Accounting for every risk, having a mitigation plan for every risk, and carefully coordinating all the sub-projects to the common goal would have been hard to accomplish via strict adaptive planning.

This isn’t to say that adaptive planning didn’t play a role in adjusting and dealing with risks/issues along the way ( think Apollo 13 ), but this project was almost completely predictive by necessity at the top-level.  There was too much at stake ( human life and a nation’s perceived status in the world ).

Even though the Apollo moon missions proved to the world that the U.S. was the preeminent technology leader vs the U.S.S.R  and would be revered for decades to come….this project, using Standish’s metrics, would have been deemed ——-> FAIL.

Example 3: Building a Table in Garage with Your Woodworking Equipment.

A more personal, and human project…building a table is something that’s been done many times in the past.  This is like building a new e-commerce site for a company.  It’s been done before, they could have just bought a vended product, but for some business reason they want to build their own.

Ok, since it’s been done before there’s a pattern, and we borrow from it.  We download the diagram, purchase our materials, and define our plan.  This is predictive.  As we begin to execute we discover flaws in the plan or the procurement process.  Wrong nails were purchased or the leg supports were cut too small.   These events require us to adapt.

After building the table we’d dust off our hands and say “Wow, that’s a little over budget and it took some extra time, but I got what I wanted.”   We’d call it a success.  Standish would call it a failure.

More?

I could go on.  What about Michelangelo’s works?  How about building the Dubai Tower?  Great human efforts ( projects ) are vision building.  Successful completion and delivery requires a range of approaches and a commitment to the end goal.

Summary

Hopefully you can visualize the adaptive *AND* predictive elements in these projects.  It wasn’t one or the other.  It was a gradient.  That’s where the present is and the future will continue to be.  Hybridization is good.  It isn’t about agile vs waterfall. It’s about achievement, belief, and success.

Stop Agilizing Everything

Introduction

Agile universities, certifications, agile consulting, traveling coaches, planning poker card sets, agile software products, agile modeling, agile arm bands, countless agile books and the crazed cycle of agile conferences.

WTF????

The buzz cycle is in overdrive and it’s electrocuted the business world with the promise of faster, better and cheaper.  This article is a plea to stop.  Stop all the hype, the opportunistic profiting, and the marketing.

Good Intentions Turned Ugly

What started out as a challenge to the software development community to think outside the box ( invent, create ), abandon a one size fits all model to approaching software development and execute your projects in a pragmatic fashion that takes account of the context you’re working in….has turned into a marketing machine of horrible dimensions.

There was a time when people talked agile and you knew they were on the vanguard; trying to solve the real problems.  They cared.  They were passionate, deliberate, and informed.  Now, when you hear a colleague professing agile…they’re most likely drinking the kool-aid poured by the snake-oil agile coach from Denver or San Fran.  The formulaic response to the core problems is all too familiar and draining:

  • Poor Requirements – You need user stories and iterations.
  • Defects in Software – Continuous integration and TDD will solve that.
  • Bad estimation – Use planning poker.  It always works.
  • Change Management – Break it up into iterations and embrace the changes given in iteration reviews.

I’m not knocking these techniques.  Many are novel inventions that do have their place in SD/AD.  But instead of being offered as potential options, patterns, techniques to solving a problem among many other potential solutions; they have become a sales pitch by the opportunist preying on desperate CIOs.  Buyer beware.  Bubbles pop and my gut says the needle to prick this balloon is getting very sharp and close.

Let’s stop agilizing everything. Good ideas, tools, and techniques don’t need the word ‘agile’ pre or post fixed to be worthwhile.

Come Back Home

So turn off the scrum-o-matic. Wipe the agile makeup from your face, and put the kanban sequin dress away. There are still problems to solve.  We haven’t unraveled this thing called software development.  It’s devilishly vexing and we need good minds focused on it.  Become neo-software-amish, come back home to the forest of software trolls and invent/create again.

NEWS ITEMS: Knowing the Solution & Risk Management in Software Development

Some of my recent news items on InfoQ……..check them out

How Will Tablets Impact Your IT Strategy?

Introduction

Gartner recently unveiled the top trends that enterprise IT should be strategically focused on.  One of those is the growing use of tablets in the work environment.   This post will take a look at the implications of increased tablet and smart phone use in the enterprise and hopefully deliver some insight into this trend beyond just a capacity replacement strategy for PCs and laptops.

Implications of Tablets and Smart Phones in the Enterprise

Let’s skip the obvious background and trend information and launch straight into the implications.

1. Printer exit strategy – Think of tablets as electronic paper.  That’s one of their utilities. Plenty of technology gurus have struggled to manifest using technology in lieu of paper only to be vexed by the utility, versatility and permanence of the 8.5 X 11 parchment of industry. So what’s different this time?  Portability, usability and eventually….sharing.  Right now it’s a little cumbersome to share notes, reports, and other virtual-papyrus artifacts with everyone in a room.  It’d be nice if my tablet recognized all those other tablets in the meeting auto-magic-ally and allowed me to share documents with them with little more than a button click.  A kind of permanent “LiveMeeting” or “WebEx” with RFID/GPS type sensory to recognize my location relative to the meeting schedule for that room.  It’s not there yet, but you can see it coming.

That alone won’t shut down your printers and get rid of the reams of stock in your office closet. Nor will it stop “Ed” at IKON solutions from frequenting your micro printing press to unclog the jam of a decade.  It will take you, the CIO, pushing, selling and implementing a bold strategy:  get rid of them.  All of them.  I’m talking about your printers.  People won’t stop using printers unless they’re gone.  Once they’re out of reach….they’ll find, and use the alternatives.

If you’re Hewlett Packard or Lexmark,  yesterday would have be a very good time to rethink your business model.  Kodak is foreshadowing you.  Tablets will get thinner, more collaborative, increasingly better at power utilization, and super cheap.

2. Embrace video/audio recording – Does anyone else see the paradox in someone with a tablet typing or writing meeting notes on his device when it’s fully capable of recording the visual/audio representation of that discussion?  Tablets and other devices can transform how your organization captures information and knowledge and shares that with others.  Written/readable documents don’t go away, but moving an organization toward a video/audio strategy should improve the quality of your work.  So much context is lost in written notes, documents, requirements, and emails.  How much does that quality cost?

Go with this strategy and here’s what changes:

  • A premium on presentation and verbal communication skills becomes essential.
  • This strategy augments your move to paperless ( number one above )
  • You need more storage and network bandwidth to capture all the videos / photos / audios:  think cloud here.
  • A tool to version, track and search this knowledge will be essential.  A wiki makes an ideal candidate.
  • Digital video and editing skills are now a key requisite on resumes.
  • Corporate policies need to be rewritten.
  • You might want to invest in pico projectors for everyone, but I think eventually tablet and laptop makers will make this a standard feature.

3. From office to work lounge –  Look at your desk/cubicle.  If you’re mobile and paperless…why do you need this space?   Work places are still relevant.  Collaboration and communication happen best in a common physical environment.  Working from home is like working in a really thick cubicle.  But to encourage the freedom and interaction that mobility and paperless bring to the office, the furniture and interior should be living, playful and open.  Many employers have already made this move:  mobile whiteboards, open touch down areas, couches, plants, and open space with lots of natural air are some of the interior elements that seem to work well for a work lounge.

The implication of work lounges and the increased interaction is that work is not work anymore.  People aren’t laborers, they’re….well….people.   Work, fun,  friends, co-workers, ideas, and profits will begin to blend.  This poses some challenges to stodgy HR policies and Tayloristic views of management.  Those who’ll succeed in this environment will be leaders, not managers and the org chart will flow around them in an organic way.   Corporate empire builders beware.

4. Office supplies / telecommunication equipment reduction –  In addition to dropping your printers and paper supplies you can now chuck your sticky notes, paper clips,  desk phones, and almost everything else in that closet.   Again, if you keep it around people will use it.  If you dump it, then they’ll get creative and use the tools they have.  Force the change.  Be the leader and save the company money.

Office Depot, Staples and others should plan their exit strategy.  Maybe they begin selling the work lounge concept and the supplies for that.  To date, I see little evidence they *get* this.

5. Killer App Coming……Plan for it -> Intersection of Identity / NFC – An earthquake is coming to the landscape of identity and access management.  Check out my earlier articles on this for background.  Your mobile device is an abstraction of you.  People will come into your employment with their credentials and data already digitized and ready to be transferred and used in your environment.  You’ll pull this data from LinkedIn, Facebook or Google+…….and using those same tools you can give them rights/permissions to systems on your cloud.    Kaboom!

In time this destroys internal LDAP systems, multiple id and password issues,  corporate HR systems and physical security access control.  These will be thrown into a social mobile nfc blender and become the domain of mega vendors.   The tech war to control the identity market will have no comparison to previous epic battles.  Those who scale this out will capture the lucrative enterprise IT market.

The implications are vast and will touch every corner of the enterprise IT market.  Plan for this NFC hurricane to shake out vendors through 2012-2013.   You’ll want to embrace those software vendors that do NFC, cloud and social identities for access.  My prediction?  Microsoft’s collapse is right around the corner.

6. Build vs Buy vs…….Download for Free.  The implication of app markets is that you now have a third generic system strategy:  download for free.   Any options analysis for system planning should consider this.   While it hasn’t happened yet, that i know of, we could well see a big vendor crash as a freely available mobile app does the approximate functionality for none of the cost.

The download for free option should also be used as a development strategy.  Maybe you find something that ‘kind of meets’ your needs.  Download, play, experiment, trial and get a feel for it.   Then, approach the developers of that app and say you have some ideas to improve it.  They might do it for free.

App markets are consumerization of IT writ large.  Our work force will be our IT department, and our IT department will turn into technology strategists, gurus, enterprise architects.  High caliber, well paid business technology talent will replace the ‘system analysts’ of today and IT departments will shrink.  Invest in your best.

7. Email’s days are numbered – Email gave us a huge productivity boost in the 90s.  Indeed it was the killer app of the first internet explosion.  But as we’ve moved through time its weaknesses are costing us.  The loss of context in email, as apposed to physical presence undermines quality of work.   Mis-understandings, multiple interpretation, cultural differences and poor writing lead to *email threads* that are a semiphore of poor quality.  Think ,just for a second, how many issues you deal with daily that revolve around clarifying what someone meant in a cetain email?   It’s astounding and it’s holding us all back.

There’s a better way, but it hasn’t been built yet.  Google’s Wave initiative is a bold attempt at remaking communications tools.   It’s close, but the email replacement will incorporate the cameras, microphones, and NFC chips that are built into tablets and smart devices.

Any CIO will want to watch this space and price out the latent, untapped potential cost savings in boosting communications quality across the enterprise.  Combine this vision with implication #5 above, and you can see the scale of change coming towards us.

8. POS industry: look out you’re about to be remade –  Tablets can be turned into POS terminals.  Enough said.  If you’re a POS vendor and you don’t realize this: what in the name of clam chowder have you been doing the last 2 years?  With NFC in 2012 an avalanche of slim, mobile terminals will usher in tap-pay while still accepting swipe pay.

Cash and checks will be digitized too and while the exact shakeout is still fuzzy to me; private digital currencies ( Ven, BitCoins ) are going to play a role here.  As I professed in my article about the externet ( internet of things ), the combination of an amorphous, unaccountable virtual world and the ability to pay with a tap lower and free the barriers to entry for those enterprising enough to believe they can challenge the global fiat currency oligarchy.  Nation states, banks, and the overlords of international finance will surely capitalize on this opportunity in some way.  Watch my blog for future posts on this….I’m still noodling on it.

9. Healthcare – Goodbye clipboard, hello iPad.  It’s all over the place, and doctors and nurses are demanding that all their software tools run on tablets.  FINALLY technology understands healthcare’s unique needs.  God bless Steve Jobs and the Apple-neers.  Steve, this was truly your greatest gift to the world…..not the iPhone.   You’ve given doctors a tool that will help them treat and solve the very problem that took you from us too early.  Rest in peace.

Taking this further, digitizing medical records and sharing that with patients is the bonfire lit by the meaningful use regulations passed in 2009.  NFC, smart devices and tablets will make the sharing part real time and collaborative.  People will really know and understand their health.

Summary

Surfing the waves of enterprise tablet integration has great possibility for the visionary C-level executive.  Will you be one of them?   My consultation to you:  tear up your current IT strategy document and vest your talent with the authority and energy to make these nine implications happen.   If you don’t…your competitors will.  You can be sure that some of the finest minds in IT are reading and following this blog.  Numbers don’t lie.  Be part of the revolution rather than a victim of it.

Software Feature Dilution

Introduction

Business analysts, requirements managers, and project managers will find the greatest interest in this article.

Software Product Planning

It occurred to me last week during one of our weekly iteration planning sessions that one of the most esoteric methods around product planning is deciding which requirements to turn into software features.  The more rigorous approaches look at the cost of the feature, the potential impact to ROI ( assuming there is one ), and the demand.

What’s wrong with this approach is that it considers the features and resulting requirements in isolation from one another.  By not considering how each new feature affects the existing product as a whole teams can and do end up with products in which the original feature set, that made the software successful, become diluted.  Those of you who’ve worked with me know my favorite example is CA’s Remedy product, but I think one could find other examples: the Microsoft Office suite of products may be in this camp.

Feature Dilution:  A Formula

So how would one go about constructing a measurement for feature dilution?  First – some assumptions:

  • You know or can retrieve the cost associated with the original marketable software release.
  • You know or can calculate the benefit ( ROI ) for the original marketable software release.
  • You have an estimated cost and benefit associated with any potential new software features.

Ok, so knowing these let’s construct a model for software feature dilution.  We’ll adapt a formula from the world of finance.

V – Value of sofware after Feature dilution =

((O x OP) +(N x (∑ IP1, IP2….IPn))) / (O + N)

where…….

O = original number of features

OP = Current NPV of product ( could use ROI too )

N = number of new features to be added

IP1, IP2, IPn = NPV of each new feature.

If you run this formula through some examples in time what you’ll find is that as a product matures new features need to continually generate greater returns to justify value to the original product and ultimately diluting the existing feature set.

This is exactly what should happen if we want to avoid the fate of an overly complex and unmanageable software product.  Just like stock market share dilution the product management team needs to justify that further feature dilution will grow the value of the product in terms of existing functionality…..not just that it will add to revenue.

Summary

Simplicity in software design has always been something great software architects knew yielded great products.  With this formula I hope I have provided at least a start to measuring simplicity in software.

I’m Skeptical on Agile – Sell Me

Introduction

This article will address a common reaction to those presented with the possibility of adopting agile in their enterprise: skepticism.  CIOs, application development managers, directors, and senior architects will glean the greatest insight from this but development professionals and project managers will find interest too.

On Being Skeptical

As a software professional your skepticism is not necessarily misplaced. There are plenty of agile coaches in the market today professing to deliver faster, better, and cheaper on a regular basis.  Their message is honey in the ears of the right executive.  It becomes even sweeter when you consider the economic climate that many businesses are facing today.  There is opportunism here and it would be well advised to vet any agile coach.

How do I know an agile coach is worth the money?

I’ve devised a simple matrix ( below ) to help guide one in validating an agile coach.

Weight Coach 1 Coach 2 Coach 3 Coach 1 Score Coach 2 Score Coach 3 Score
Number of Projects Managed 3 2 17 3 6 51 9
Years of experience in SD/AD 2 5 24 7 10 48 14
Highest Budget Management Experience ( 1 = true, 0 = false ) 1 1 1 0 1 1 0
Number of References Validated 3 3 8 1 9 24 3
CSM Certification ( 1 = true , 0 = false ) 1 1 1 1 1 1 1
PMP Certification ( 1 = true , 0 = false ) 1 0 1 1 0 1 1
FINAL SCORE



27 126 28

So let’s talk through this matrix a bit.  First, I give pretty heavy weighting to experience here.  In truth we’re not just looking for an agile coach we’re looking for someone with the battle scars of being in the AD/SD world and knowing when and where agile works vs more predictive methods.

We also want to know that they’ve actually implemented agile methods in other places hence the need for validating references.  The key word here is “implemented”.  There are plenty of folks who can regurgitate the agile manifesto and paraphrase the thinking of leading agile theorists, but agile coaches should show a track record of making it happen.

Budgetary management experience, in my opinion, is essential.  If they haven’t managed the dollars/euros/yen around a capital project ( or operating costs ) then they may have a very misguided notion of why projects succeed or fail.   The CSM or PMP who was merely accountable for a timeline with only a misty concept of how it related to money is ill-equipped to profess a transformation of your SDLC process.  Why?  Agile techniques profess delivering software in iterative cycles ( every 2 weeks ).  If some level of requirements aren’t complete by the end of each iteration you have two options on a fixed bid capital effort:

  • Don’t do that functionality.
  • Postpone it until you do have the capital available.

This sounds fine in theory, but the truth is that every system has some minimal set of requirements that must be completed for the software to be functionally usable.  If the money runs about before the agile projects succeeds in delivering this minimal functionality then your project will be seen as a failure.

Certifications show learning of theory.  I weight them low, but still think its practical to have these ( Certified Scrum Master and Project Management Professional ) if an agile coach is selling himself as a professional in software development delivery.  They should understand and know agile as well as more traditional management concepts and techniques.

Lastly, you should realize that agile adoption is not just a function of the agile coach.  The organization needs to be willing and able to accept the changes agility will introduce.

But my current process works, so why should I switch to agile?

If you have a working process and there is no immediate need to push you to agile then you should take the time to map out a strategy for your development shop.  Agile can be beneficial and Ryan Martens at RallySoft does a decent job of articulating when agile methods can benefit a development project.  His rendition of the Uncertainty vs Complexity diagram proposed in Stand Back and Deliver gives an AD manager a basic tool for plotting his/her projects along these two broad metrics.

project-types

What are the benefits if agile is applied to the right type of projects?

Better Risk Mitigation – Agile methods emphasize iterative delivery of software.  A standard cadence and check point to the project sponsors allows for defects, requirements misunderstanding, and general issues surrounding the effort to be mitigated on a timely basis.  Couple this with a daily stand up meeting where team members determine how to resolve issues and coordinate work and risks to the development effort are generally better managed.

Testing starts earlier – Agile development emphasizes vertically slicing your application and developing functionality incrementally.  This is a technical challenge, but assuming the development team can tier the system architecture this way, then your testers can usually start functional testing much earlier in the development cycle.

Increased Sponsor Satisfaction – Project sponsors are involved routinely through agile.  The developers have a direct line to the customers.  This continuous feedback loop usually leads to better communication and understanding between the team and customer.

Stronger Team Accountability – It takes time, but as the team culture shifts from command and control to a collaborative effort where developers take responsibility, collectively, for their work; the team begins to see how their efforts help/hinder the project.   An adjunct to this is an increased sense of pride in their work and kinship with each other.

What do I need to watch out for when adopting agile?

Cultural Shift – This can’t be under weighted.  Agile places greater emphasis on the team managing itself and its day to day activities. Subtly, agile preaches two things:

1. Development team and customer working together.  Meaning other managers and IT leadership have a de-emphasized role.  Your risking attrition by some of your better players if you ignore this.  Proper coaching and preparation for this change and its effect on roles and responsibilities is essential.

2. Team stepping up and coordinating activities among it’s members.  This is normally done by a PM or Dev manager or even a senior technical leader.  Some methodologies, like Scrum, emphasize a new role ( scrum master ) to take on the facilitation aspects.  For developers unaccustomed or uncomfortable with organizing and planning this may be difficult.

“Documentation is not needed”  - You may hear this from some agile coaches and theorists.  The original agile manifesto emphasizes working code over documentation, but as a development professional you’ll need to decide if this really makes sense  for your project.   Some of us have regulatory and legal reasons for documentation.

Dogmatic Views – I wrote about some of this in Bad Attitudes of Agile, but some team members will see agile as a very strict set of practices and may twist the theories and methodologies to suit their own ends.  By its very description agile is meant to be a flexible approach to software and application development not a rigid set of rules that cannot be altered.  There are the pragmatic agilists and then there are the agile zealots.  Watch out for the latter.

Summary

Benefits can accrue from agile methods.  These benefits, for the right projects, should result in better quality, reduced cost and schedule variance associated with requirements misunderstanding and defect management, and a more complimentary relationship with your customers.  As mentioned earlier skepticism is not misplaced, but by looking for an opportune experimental project to introduce agile a development manager can assess its applicability for his/her shop.