Agile Breakdown Impedance



Breakdown Impedance is the term I've just coined for a problem that I've complained about in my Agile retros. Impedance is a word most commonly known from other scenarios where, for example, the way your database implements a type isn't the same as your programming language. Thus you have a small area of casting work to do or grey area you need to investigate. It could be the fact that your doubles can't be as accurate as PostgreSQL's unbounded numeric field or it could be the way that Dates in an XML file need subtle alteration before they can be safely used.

Breakdown Impedence are the pulling factors that complicate the task of breaking an Agile story in to smaller parts. It's the blurry bit that slows the process down a little and involves everyone in the team having their own ideas. The motivators for breaking things down in Agile are:

1. Breaking it into testable chunks



The requirement to make a balanced iteration where testers get things early and aren't backed up at the end of the Iteration. Your Agile project manager will be pushing for this if she knows what she's talking about. Getting small things into testing.. even if means a feature gets a "simple UI against the model" for the testers while a second more complex one gets done straight away afterwards feels painfully like un-productiveness.. a duplication of effort but it greatly increases the chances of delivering the work in that iteration. If you try it with an open mind.. you'll be able to make up your own mind up on it. Personally I found it cringe worthy in concept but now I'm very much open to it. You know how they say 80% of a program's estimate is spent in development and another 80% is spent bug fixing? Well, I've just shown you an example of a trick that will get you from 80% to 95%. The earlier the bugs are found - the earlier they are eliminated.

2. Break it into deliverable chunks



Perhaps it's the B.A. pushing for this one or maybe again it's the Agile project manager. An example of a deliverable chunk might be a readonly version of the page on the website. It's not functional but it's half way there and that part is done. If you underestimated the size of the task by two days, it doesn't matter. That part will go live and the customer can see you're making progress and listening to them. That part can't fail. You make it editable later, in the same iteration hopefully or in the next if you overrun. We aren't saving it up and releasing three months of development for some poor tester who has to spend an entire two week iteration rushing to regression test it before it goes live. The feature is isolated and it doesn't make the site look broken. A deliverable chunk could just be a part of the model or a partial tweak to the database. Get the database in the new format ready for the next set of features. It's a chunk of real work that goes into live before the whole story is complete.

Here's a real example. Me and my team developed this cool new user interface for our website. However, we needed this new jQuery plugin that only works with the latest version of jQuery. Retrospectively someone pointed out we could have rebased the existing system to use the new jQuery first, got it out there and ensured any problems weren't convoluted with our other changes. We could have spread the bugs out across the iterations. When you break stuff down like this and everyone sees gradual improvement consistently in every iteration then you're Agile. This is the skill the customers will love you for. The benefit to you is that you can forget about it when it's done. One thing checked off the list. It's actually quite liberating, even if it does add a little extra testing and remember that testing isn't wasted. It's just more thorough. This kind of breakdown is what Agile was developed for.

3. Break it into conceptual pieces



This is what you do. This is basically how developers break things down. It generally represents how you tackle the coding in chunks. Normally it looks like this:

- "write SQL to add DB table"
- "model function"
- "call it from the relevant code sites"
- "refactor that existing bit of code to use the new shared function"
- "write the unit test".

You do this because it's what comes naturally to you. Each conceptual part is simple to remember and to see as a "chunk". Perhaps the real underlying test for this is that at each stage of development you have something which compiles? It's classic bottom-up software development and developers can't help manifesting it into post-it note tickets.

The problem though is that all the tickets need to be completed for the story to be done. How do you develop the Javascript if there is no controller to serve the request? How do you write the Ajax handler if you haven't unpicked the arguments to the Model yet? A sure fire way to recognise what you're doing is if on "day one" of working on a task, you move all three tickets into "In Progress" at the same time.

It's natural to take this approach to breaking work down but it shouldn't be your only mechanism. When you break the tasks into "this handler - that handler" you end up forcing your own hand and leave yourself no flexibility. I won't fight you against this style because as I said before it just comes naturally to you.. but you should be consciously aware of what you're doing and understand the drawbacks of it. I don't think it's what Agile is trying to bring out in you since nothing is done until it's all done.

4. Break it into "swarmable" pieces.



If you've got a Javascript Ajax query to a controller do you really want to write documentation so that two people can work on it at the same time on independent parts? If two people work on different aspects of a feature, who writes the SQL patch that you both depend on? If the task is complex how will the code line up? Will the developer writing the model end up revalidating things the developer checked in the controller?

The breakdown impedance mismatch is when these don't align with each other and they rarely do. Rarely can a tester test "just the model". Is that new jQuery plugin you've added to the code base which isn't used anywhere actually testable? Does that mean it can go live? How do you swarm if only one person is writing that framework function? You can't second guess the parameters yet. Difficult questions.

My solution to this is to break everything down as far as it will possibly go.. to the extremes.. then reassemble the pieces into deliverable and testable chunks retrospectively. For example, if two developers both wait on a SQL patch, allow yourself to swarm by creating a tiny half day task at the beginning of the story to get that written, tested and merged into master as soon as possible. Then the two devs aren't tied to together any more. "Stubs" (which are typically empty functions) declared and checked in early allow one person to write the GUI part whilst someone else writes the model part. I've tried these things and they work. I've even tried writing the model and frontend separately by writing a controller in the middle that fakes the response. The result? All aspects of the task were started earlier and at the end of the iteration we had plenty of time to remove that so called wasted "fake response" code. Writing a little contract together takes time and feels like wasted time.. but the sooner a feature is merged to master and out the door the better it is. When you work in a little team like that and you tell both people what a success it was, you reflect each others' pride and then magnify your own enjoyment of having developed it!

Two heads are better than one and a five day feature takes one person five days and two people 3 days, even with an entire day of overhead. Plus you've two days so testing starts earlier, shows up bugs earlier and you can code review each other's work, "make it blend" structurally and make yourself feel good about it. If you're going to inject at this point and say "well two devs could have been two features" I would have just countered by saying that's two half finished pieces of work which haven't been tested yet and are probably more prone to both design and real coding flaws. Give the customers what they want. That one important story now! The customer thinks you've worked harder even though you probably just sat around just being less stressed.

Summary



Hopefully these four breakdown "buckets" help surface why this part of the Agile process is particularly hairy and requires special attention. If nothing else about your Agile team structure and process changes, the breakdown approach could and possibly is still the biggest part of delivering code successfully. Perhaps if Agile isn't working for you, I could encourage you to cast an eye on how your team approaches this tricky detail.

These aren't mutually exclusive buckets. You need to carve the tasks out whilst trying to meet all the benefits of each of the buckets above when possible. The goal of this blog post is that by dividing these requirements up and naming them, hopefully I've made you more aware of the subtle differences in your decisions.

It's taken me a long time to accept that productivity wasn't a developer writing new code continuously. I used to feel every line of code generated should actually appear in the final product. Whose efficiency was based on a stream of doing solely dev work. However writing a little unnecessary code here and there, spending half a day rebuilding the RPMs early for the testers and putting things through the continuous integration server although redundant in one aspect, is greatly beneficial in reducing the risks of not getting anything done at all!