The Nine Forgettings

"If you measure the wrong thing, and you reward the wrong thing, don't be surprised if you get the wrong thing."
-- Lee Copeland

Lee's talk is about 34 minutes and challenges how the typical company does software testing.

It's a good watch, and it's free.

Post Agile Scrum

I've been following the LeanAgileScrum discussion on to Agile Project Management list with some interest. Here's the reply I sent recently:

I would like to relate a story of a personal experience from a few years ago.

I worked with a group that had a rather heavy-weight requirements process. By which I mean, big nasty template with signoffs. I once saw, literally, a five-page requirements document that equated to one line of code.

So, in comes the agile coach, and he says, "this requirements doc is junk. We're going to discuss the requirements and write things on these little index cards ..."

The requirements people first admitted the requirements docs were bad, then fought to keep them tooth and nail.

What's going ON here?

The best explanation I found was that we were taking away the one thing they knew to cling to. Oh, it wasn't very good, but it was a safety blanket. Without that, *now* how do we do our jobs?

I think moving from a heavyweight, Big-Up-Front-Everything shop to a scrum shop is much like that. Many people want prescriptive processes. They want to be told how to do it. They want to be able to follow the process instead of inventing it.

Or, at least, they _think_ they want something like that. If they *have it*, they'll feel constrained by it and hate it and complain about it, but gosh, having a template sure is a lot easier than having a blank sheet of paper. Even if it's a crappy template.

So you see these good ideas like Agile or Scrum institutionalized, procedure-ized, process-ized, turned into certifications ... and someone has to come and invent a new buzzword to say "no, stop being stupid" only more politely.

Today it's called lean, or maybe post-agile. And, if it achieves some good, I'm fine with it.

The Process Process

Special thanks to Ben Simo, who just emailed me a link to this comic strip.

The sad thing is, I'm pretty sure that I actually know the people in the strip! :-)

GASP? ... or not.

Doctors have a rule for sterilization, or "Always Wash Your Hands." Simple things that can be done to improve the outcome of every single project.

At the recent, GLSEC keynote, Bob Martin asked us "We were all doctors, and a hospital administrator called a meeting, telling us that we were spending too much time washing out hands - would we stop doing that?"

Of course not. Medical professionals know the risks involve in skipping that step, and simply won't take it.

Likewise, Accountants have a concepts of "Generally Acceptable Account Principles", or GAAP.

I started writing this blog entry intending to find some "Generally Acceptable Software Principles" (GASP). After all, if accountants and doctors can do it, why not us software guys?

And, in fact, I have a few. My colleague and friend Andy Lester does some speaking and consulting, and his immediate refrain is "Bug Tracking, Version Control, and Daily Backups." That is to say, if your software organization doesn't have these three things, trying to do anything else for "process improvement" is a waste of your time. Get Bug Tracking, Version Control, and Daily backups first.

I recall that Steve McConnell had a few talks on this subject, so I googled around, and found Software Development's Low Hanging Fruit and the Ten Most Important Ideas in Software Engineering.

Now, I like Steve McConnell. I've read his books, we have corresponded a bit, and I've quoted him quite a bit in my writing. For the most part, I like what he has to say. But his low-hanging fruit is nothing like what I would recommend, and his "ten most important ideas" reiterates the classic cost-of-bug-fix curve.

That might be true for some organizations, but it isn't true for all.

I got thinking about GASP because on one email thread this week, we discussed the possibility of having Test Driven Development go mainstream, and I wrote this:


I don't know if TDD will ever go mainstream. My prediction is that we will continue to have a large group of people who are doing a 'good enough' job of development that don't read books or go to conferences. Those people will become managers, and if they hire a new grad who knows TDD, > 80% of those new grads will just follow the prescribed, crappy process instead of looking for a unit test framework for Ada.


One of the great things about software development is that if you have an idea, you can go into your garage and build something cool and sell it. Then you grow, have to maintain legacy code, and process becomes important. All over the world we have companies on a great continuum, between life-critical software and video games for a PDA.

It would seem to me that to require software engineers to be licensed, to require, perhaps, an advanced degree (think law or medicine) and a board of examiners might increase our process, but it would create barriers to entry that would stop a lot of really smart 16-year-olds.

I was once a really smart 16-year old without so much as a high school diploma.

It seems to me that the best thing we can do as an industry, is to not outsource our discernment about what practices are "best" or "right" or "professional", but instead keep that responsibility to ourselves - and live with the consequences of carrying that responsibility. Then we can judge our practitioners by the output they produce.

What do you think?

Where have all the sapient processes gone?

Most agile test automation is, well, clerical. To borrow an analogy from James Bach, it views testing as something like an inventory clerk at a Grocery Store. "It says here we should have thirty cans of peas. How many cans of peas do we have? Thirty? Good! Check. Greenbar."

In addition to counting them, a real tester would be doing a lot of different things - looking at how the cans are stacked, checking the labels to make sure none have peeled off - "noticing" things that are just not quite right. It's relatively easy to automate the clerical part of testing, but very hard to automate the sapient part.

I find that when I talk about Sapient testing, some people get really uncomfortable. I've spent some time analyzing this, and I think it goes back to Frederick W. Taylor. In the early 19th century, Taylor proposed scientific management, suggesting that we separate the worker from the work. Henry Ford credited Taylor's work as leading to the assembly line, and making Ford a billionaire.

So, hard-coded in the business DNA of North America we have this idea that Management works ON the system, and contributors work IN the system. In that world-view, management is responsible for both eliminating general-cause variation (machines break down periodically, so invent a maintenance schedule) and handling special cause variation like customer complaints.

If management is working on every single exception to the process, then workers should be able to work by rote; contributors should be able to simply follow prescribed procedures. (This is a big part of what Aldous Huxley was talking about when we wrote Brave New World)

The problem is, what worked great for Taylor in 1907 isn't working so well a hundred years later. Today's high-tech technical contributor has more education, depth of decision making, and specialization than yesterday's foreman or line supervisor.

Instead of trying to predict the process, we need to recognize that the white collar worker *is* a manager of sorts. And that's not me talking; I'm paraphrasing the words of Peter Drucker.

What does that have to do with software testing? The best minds in the business are saying that dumbing-down software testing to prescriptive, proceduralized process is foolish. The the same time, the same discussions are happening in the development world, the requirements world, and the project management world.

Still, when I talk about sapient processes, people get edgy. Getting Frederick W. Taylor out of the IS Shop is going to be an formidable challenge.

So let's buckle up. I can't promise that it will be an easy ride, but it certainly will never be boring ...

To XP or Not?

I just posted this to the Xtreme Programming Yahoo group, and thought it was worth sharing. I start out quoting David Winslow ...

I want this project to succeed and I want to do the right thing. With that in mind I am punting for us to develop an agile xp approach to the project. However I am meeting a lot of resistance from the stakeholders as they don't understand the XP process.

My suggestion is this:

Don't call it XP

Don't call it Scrum

Instead, call it "delivering small pieces of finished work on very tight timeframes", or, if you must "lean software development" (or Toyota.)

Pair Programming, Test Driven Development, and refactoring are just plain good engineering practices. You don't need to ask permission to do them.

-----> I once worked on the "after project" where the "before project" took three years, technically worked, but didn't meet the customers needs.

At the first "requirements elicitation" meeting, I pulled out index cards and started taking notes. Some people felt uncomfortable, and I replied "After I finish these cards you will review them and sort them - and I will DELIVER the first card on the stack, COMPLETED, in a month - possibly with more."

All of a sudden, the room got very quiet. Then the customer/manager replied "ok then. Let me see those cards."

The rest, as they say, is history ...

Project Scheduling Formulas

James Bach is running an on-line class for his Rapid Software Testing Course, that I am finding quite enjoyable. More fun than the lectures (which are good) is what I'm learning from the offline forum software he is using.

Recently, someone made a post about project scheduling formulas; I enjoyed it so much that I wrote up a rather long reply, which I wanted to share here:

Hello John. I have heard that formula as well, very recently, from people in the PMI/PMP world - people I respect who consistently have good results.

However, I don't think it's magic. If we take a minute to deconstruct this ...

If a = most optimistic estimate
b = reasonable estimate
c = pessimistic estimate

Then use give an estimate of (a + 4b + c) / 6

Then A is what many projects start with. In "Waltzing With Bears" Demarco and Lister assert that technical folks are really good at coming up with the "Nano Percent Date" - the date which the project could be done if everything goes perfectly - which has a nano-percent chance of happening.

Just moving the conversation past A is helpful.

I've heard different descriptions of C, but it usually comes out to about twice B, which is usually about twice A. In other Words:

C = B*2
B = A*2

Estimate =

(A + (2*4*A) + (2*2*A) )/ 6 =

(A + 8A + 4A) /6 =

13A / 6 =

A*2.166

:-)

In other words, this isn't much more than "Take your original estimate and double it", but it's dressed up in psuedo-science. :-)

----> Now, my less sarcastic answer:

Actually, it's a little more valuable than that. You'll notice that you've got A, 4B, and C - which total six "things" - divided by six. That's a weighted average - weighted strongly towards B. However, C is the gotcha - C is where you weigh in what happens if the entire development team is killed in a hurricane and the team needs to be rebuilt from scratch. That number is usually large enough to drag the average toward the large side. This gives us our 16% buffer past just "2". (Remember, we came out to 2.166.)

Also, there are some projects where C just isn't that large, because you are doing something that is well-defined with commonplace technologies. Those types of projects often have low return-on-investment, but also low risk. In that case, using A, B, and C might be more helpful than just multiplying by a number.

When I run projects, I have two dates - a commitment date back to the business and a goal date for the team. To calculate the difference, I try to take 'most realistic estimate' in one corner and add a buffer for risk - the larger the risk, the bigger the buffer.

So my estimate math is:

Estimate = Realistic * X

Where X is something between 1 (zero risk, I'll have your essay written today, no problem) and 2. X is my risk factor; it's different on every project.

If X is greater than two, I usually suggest an architectural spike to make realistic more realistic, or some change in scope.

Now, I am not accusing you of this, but I have seen people use scheduling formulas without understanding them, and the results, though tragic, are predictable. Actually, to be honest, I find asking the "why" of the formula is far more enjoyable than plugging in numbers ...

SideBar

...(Administration) covers the surface of society with a network of small complicated rules, minute and uniform, through which the most original minds and the most energetic characters cannot penetrate, to rise above the crowd. The will of man is not shattered, but softened, bent, guided; men are seldom restrained from acting, such a power does not destroy, but it prevents existence; it does not tyrannize, but it compresses, extinguishes, and stupefies a people, till each nation is reduced to be nothing better than a flock of timid and industrious animals, of which government is the shepherd.
- Alexis de Tocqueville,Democracy In America

Substitute "Administration" with "process" and "government" with "management", and you have the sickness in "enterprise" development today.

Rethinking Process Improvement - IV

Yesterday I suggested that a lot of process improvement is trying to eliminate the overlap between roles. For example, when people talk about making job descriptions "better", that is often what they mean.

Each team draws back, clearly defines themselves and what services they offer.
Here's one example of what that might look like:



… So, who’s doing the work in the middle? (No one)

What does this mean on projects? (Things fall through the cracks)

Can we really call that 'improvement'?

Rethinking Process Improvement - III

If software development is an assembly line, then unclear roles is a real problem (see illustration.) You don't know who is supposed to tightnen the nut. It might be tightend twice, it might be tightened once, but one thing is certain: The variation in the tightening will slow the line down. So one of the ideas in traditional process improvement is to clarify roles and responsibilities, job descriptions, and so on. (Or, in other words, to "decrease the variability in the process", a line right out of the PI literature.)

I am a bit dubious of that position, but we'll get to the why tomorrow. In the mean time, what do you think?

Rethinking Process Improvement - II

This is image 2 from Winston Royce’s Paper – "Managing the Development of Large Software Systems"

Let’s look at each stage for the process – requirements, design, coding, testing … it’s an assembly line. At each step, someone produces a work product which is handed off to the next person in the line. Only the project manager owns the entire process. In fact, some of our very own job descriptions, like developer, or tester, or requirements analyst, assume that our entire role is to perform one box in the sequence.

Later in the paper, Royce went on to write:


"I believe in the concepts, but the implementation described is risky and invites failure. The problem is illustrated in Figure 4. The testing phase which occurs at the end of the development cycle is the first event for which timing, storage, I/o transfers, and so on are experienced as distinguished from analyzed. These phenomena are not precisely analyzable. … Yet if these phenomena fail to satisfy the various external constraints, then invariably a major redesign is required. A simple … patch or redo of some isolated code will not fix these kinds of difficulties. The require design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violate. Either the requirements must be modified, or a substantial change in the design is required. In effect, the development process has returned to the origin and one can expect up to a 100-percent overrun in schedule and costs."


Royce does not use the term "waterfall" in his original paper. His final recommendation involves writing the program twice; admitting that the first iteration will be problematic, accelerating it, and finding the deficiencies in the requirements and then starting over with design. The term probably came out of the arrows, which kind of look like a waterfall.

Yet too many people never got past page one of his paper. After all, this is familiar; it’s applying Fred Taylor to software. And Taylor’s a genius, right?

It turns out that Taylor’s entire body of work is based on experiments with Immigrant day laborers who had to move raw iron from the factory to box cars. The typical object of that work had a third grade education, which was typically in German.

Software development is not turning a wrench – it is intellectual work – knowledge work – that is different every time. When we do heavyweight handoffs, we lose information. We say "We really need to get the requirements right next time."

Now, one definition of insanity is doing the same thing over and over again and expecting a different result. For thirty years, Process Improvement (in capital letters) has been telling us to get the requirements right up front. And we’ve been failing, for thirty years. That is insane.

Solid presentation advice ...

Suggestions and Examples of What Not to Submit

1. Attendees are paying to take classes—they don’t want to hear a sales pitch, no matter how thickly veiled. Please do not submit classes that feature your product or describe problems that happen to be solved by your products.

Attendees and the conference organizers are equally skeptical of technical talks proposed by marketing people, Business Development Managers or CEOs—unless that person has the appropriate technical background and experience. Even the appearance of a sales pitch, such as when a talk is given by a marketer, will cause people to not attend that class.

2. Attendees don’t need to be taught why something is important (why security testing is important, why testing early in the SDLC is important, etc….). If they sign up for the class, they already know that. In your abstract, explain how the attendees will learn HOW to do something by taking your class.

3. Read the Course Catalog from the previous conference to learn how the classes are described. Try to match that, and have a colleague review your submission to make sure that your abstract makes sense. Experience has shown that an incoherent abstract is not a good sign for a coherent presentation.

- From the Software Test&Performance Call for Speakers

If you run a conference, especially a regional or non-profit one, you might want to slap something like this right onto your call for speakers. After running a regional software conference and participating in a few more, I've seen pretty much all of these problems. Heck, I've seen them in Lightning Talks.

Sometimes it can be very hard for a program chair to know who to select and who not to. The best thing we can do to help is to give them solid presentations, a strong outline, and no fluff.

Rethinking Process Improvement - I

Most of our ideas about process improvement come from a factory analogy – which was Invented by Frederick W. Taylor at the beginning of the 20th century. His idea was that you separate the work into independent roles, then each person owns one single step of the process. It is the role of management to own the process; people are simply a cog in a great big factory.

And, for 1911, it was pretty good. It made Ford a billionaire, and that’s nothing to sneeze at.

Now, think about the assembly line. More later ...

Run Corporate IT like a software company

I'm still on vacation, but this struck my eye as worth sharing ...

... corporate IT has to find ways to deliver the most important business capabilities as quickly as possible and as cheaply as possible. When the rubber hits the road, this capability is the only one that matters. Your business partners won’t ask if the back-office design is scalable. They won’t care if the requirements are thoroughly documented or the test traceability matrix is complete. When they compare your IT department to a pack of starving programmers in China, their decision is going to boil down to a simple question: who can deliver the things we need the most the fastest?

--- From the Tech Dark Side: http://www.techdarkside.com/?p=61

Interesting Links

Scott Ambler has an article in this months Dr. Dobbs Magazine on Agile Testing:

http://www.ddj.com/dept/debug/196603549

Yesterday, Sean McMillan sent me this link to Big Agile Up Front:

http://c2.com/cgi/wiki?BigAgileUpFront

I'm beginning to see a theme here, a theme that I have also seen in the real world. People want things to be "all figured out"; to have the best way of doing something.

It reminds me of that old saw that a camel is a horse designed by committee.

No thank you. Give me conscious design tradeoffs any day.

Canary in a Coal Mine

There is an interesting talk in Info.com by Ken Schwaber.

Ken is a co-author of the agile manifesto and co-inventor of Scrum.

If you don't want to listen for an hour, move the slider to 29 minutes - the section on magic thinking and lies. If you get hooked, you can come back to the first 29 minutes later.

Of course, if your organization doesn't have problems with scheduling and politics, you won't need this.

Throughput - I

I've been thinking about throughput a lot lately.

Example: It's winter in Michigan, and it's snowing. So, on my way to work this morning, I see a truck that is plowing. The truck is doing an excellent job of plowing, shifting from forward to reverse very quickly, accelerating quickly and using heavy brakes. He sure is marching and moving.

The problem was, I was trying to get past him, and I couldn't, because his sudden shifts in momentum were so uprupt that it would be dangerous to pull forward. I had to wait. And wait.

So, the truck was doing an excellent job plowing, but overall throughput on the road suffered.

We do this in software development all the time. We optimize the job for our role, focus on handoffs, signatures, and role-based contracts, instead of trying to be helpful and collaborate. Devs refuse to proceed without documented requirements instead of having a conversation and trying to figure out what the customers and analysts desire. Architects (and I use the term loosely) refuse to begin design until "all" the requirements are elaborated. Testers refuse to begin testing until all the documentation is completed.

Each of these tactics helps the individual role be efficient (or easy), but the overall project suffers. In fact, you can make a strong argument that the Waterfall model gained it's popularity not because it was good, but because it is conveinient and easy for management. (After all, for the schedule, you can just "set it and forget it.")

In my own career, in general, I've protected the project at my own expense. While I have taken some slings and arrows (and been told at least twice to "know your role, be your role" when I was trying to be helpful and get things done) it's led to more successful projects, and allow me to develop expertise in software testing, requirements, scheduling, and the overall process.

A rising tide lifts all boats, so I would like to submit this to you: Forget your role, keep the project moving, and your title and position might just take care of itself.

(Of course, there's more to it than that. More later.)

Creative Chaos - II

A friend of mine once asked me how I come up with material for one-hour talks, and I gave him this answer:

"I start with three sentences. Then I imagine all the ways that those three sentences could be misunderstood, and I build up defenses and arguments against those misunderstandings. Once I'm done, I've got a 1000-word essay or an hour of material, take your pick."

So, in a previous post I said that heavyweight methods can stifle creativity. I suggested a creative, chaotic process, much like W. W. Royce does at the end of his paper "Managing the Development of Large Software Systems" - viewable here. (Ironically enough, it is the First Page of Royce's paper that is credited for inventing the concept of the waterfall model. That's a good place to start - but for goodness sake, don't stop there!)

I left out a bunch of assumptions, like your team is staffed with great people. If you don't have a team of experienced and good people, when you eliminate the binders and templates, the team won't know what to do. Inexperienced you can do something about; break the problem down into smaller chunks and give them guidance. With a team that isn't good ... Wow. Completely different problem.

Here's my take on great people: Great people are all methodologists with a lower-case "m." They have a wide and deep set of methods, and the good judgment and discernment to choose the right method to use in the moment - something that no capital-M methodologist sitting in an office 1,000 miles away is going to be able to do.

Although I've published an article or two on methodology, how I think about methodology is slowly changing. Written today, my "Methodology" would have an emphasis on hiring great people, developing talent and teamwork that I find completely absent from your typical big thick binder. Yes, with XP, Crystal, and a few of the Agile Methods, it's implied, but it is still rarely explicit.

Still, I know of at least one organization that manages this way:

Professional Baseball

They seem to do ok at it. hmm.

Opportunity Cost - II

In a follow-up to this blog post, Dr. Cem Kaner emailed me this:

I tried to post this to your blog, but I get an error message.

MY POST
Peter Drucker provides deep wisdom for managers in a short book called "The Effective Executive."

An executive is someone who is responsible for the value of his own time. Most knowledge workers are (or should be) executives under Drucker's definition.

Drucker made the point forcefully that no executive will do everything well, and that many managers fail because they try to do too much "acceptably" rather than doing fewer things very well.

It has been decades since I had a job so simple that I could do all of it well, or so limited that I could find time to do all of it well. I have had to prioritize. And that includes not spending time getting better at things that I won't do.

I don't want to work on my weaknesses. I want to work on my ability to meet my strategic plan, whatever that is. Sometimes that requires fixing weaknesses, sometimes building on strength, often it requires developing a new strength in an area that wasn't seen previously as relevant.

I think it's interesting that my original post only offered two options: Work on your strengths, or work on your weaknesses. This is just two choices, a dilemma. Dr. Kaner is offering a third option; work on the things that map onto your goals (or strategy).

Jerry Weinberg calls this the "Rule of Three"; that when you feel limited to one or two options, you haven't thought things through enough.

Thanks for the insight, Dr. Kaner. Does anybody have a fourth?

Why Creative Chaos? - I

Five years ago I read Wicked Problems, Righteous Solutions, and it gave me words to express the way I understand software development to actually work.

Here's the deal: Software development is a creative process that you learn about as you do. That means that things change as you do them; more important than "getting it right the first time" is periodic assessment and course adjustment. That means that feedback is king.

For example, there are different ways to make soup: You can follow the directions (a prescription) or you can hire a world-class chef. If you follow the directions, what you get may be good, it may be bad, but it won't be great.

The chef isn't going to follow directions. He's going to start with something basic and flavor to taste. The college-edumacated people would call this an empirical process (one directed by feedback) instead of a controlled process.

The result of this is that if you want really great software, a predictive methodology isn't going to help much. It might be better to just put some bounds around the software (for example, a defined release process) and make it clear that like art, heavyweight methods actually stifle and hurt.

We seem to understand this idea for design; Tom Kelly has a book on it called The Art of Innovation. What bothers me is that so few people understand that, short of pressing the F5 key, all software development is a design activity. (Or, at least, if they understand it, they act as if it is not true.)

So, that's where Creative Chaos comes from. Because I specialize in software testing, I could have called it "Destructive Order", but that would just be too confusing. :-)