Testing is indespensable. Or is it?

I added a set of rules (questions, really) to my wiki, concerning product test as measurement of product quality. Please check it out and spread the word: Article on PlanetProject.

Improvement Principles

I assembled this set of principles to guide me and others through the sometimes stormy times of change. To me, 'times of change' equals 'always'. If there's one common theme among all my different job positions and private challenges, it's the wish for improvement. Use these principles whenever you embark on a journey.

What are the most valueable specifications?

During my last vacation I came across evidence of good technical value specifications. At VW's Autostadt there is a display of this wonderful old Phaeton Sedan (sorry the photo is blurred).

There also is a reproduction of a poster that was used to market the car at its times. It clearly states the most important values the car delivers. Reading through it, and thinking of the extreme focus on functions nowadays, I ask you: Did we forget to focus on the really important things, like value?

The poster reads:
It is our belief that the most important "specification" of the Cord front-frive are: Do you like its design? Does the comfort appeal to you? Does it do what you want a car to do better and easier? When a car, built by a reputable and experienced company meets these three requirements, you can safely depend on that company to adequately provide for all mechanical details. -- E.L.CORD

Hyper-Productive Teams

The Agile Engineer Ryan Shriver has stunning news from the productivity front. After he 'produced' a whopping 1.750 attendees for his webinar on the topic, he was kind enough to share the presentation slides and a companion article. I quote from the article:

Hyper-productivity describes a state of being where teams are working at much higher levels of performance such as two, three and four times more productive than their peers. 

The interesting question for agilists and other method-interested people alike is: How can I do it? Well research shows than hyper-productivity seems to come with a couple of practices, all to be found in Ryans slide set and article. Be aware, don't get trapped into the logic than you only need to follow these practices to be hyper-productive.
I'd like to highlight one theme here, and Ryan probably says it best (mark-up is mine):

[...] the difference with the hyper-productive teams is their ability to stick to the practices over the project while continually removing impediments limiting performance.  

This resonates quite a bit with something Eric Ries posted about a startup lessons learned conference, in which Kent Beck happened to develop another manifesto, apparently (mark-up is mine):

Team vision and discipline over individuals and interactions (or processes and tools)

Validated learning over working software (or comprehensive documentation)

Customer discovery over customer collaboration (or contract negotiation)

Initiating change over responding to change (or following a plan)

I wouldn't go as far as calling this the new agile manifesto. This is more a set of paradigms for startups, so one could call it a startup manifesto ;-).

History Repeating?, or A Case for Real QA

Jeff Patton of AgileProductDesign.com has an excellent article on Kanban Development. He does a great job in explaining the kanban idea, in relation to the main line of thinking in Agile. The article is about using a pull system rather than a push system like traditional (waterfall) development or agile development with estimation, specification and implementation of user stories.

Reading the first couple of sections got me thinking about another topic. Isn't this agile thing "history repeating"?

First of all, I'd like to question how common agile development really is. It's clear that the agile community talks a lot about agile (sic!), but what does the numbers tell us? I don't have any, so I'm speculating. Let's assume the percentage of agile developments is significant. (I don't want to argue against Agile, but I want to know if my time is spent right ;-)

Jeffs writes:

"Once you’ve shrunk your stories down, you end up with different problems. User story backlogs become bigger and more difficult to manage. (...) Prioritizing, managing, and planning with this sort of backlog can be a nasty experience. (...) Product owners are often asked to break down stories to a level where a single story becomes meaningless."

This reminds me so much of waterfall development, a line of thinking I spent the most of my professional career in, to be honest.

First, this sounds a lot like the initial point of any requirements management argument. You have so many requirements to manage, you need extra discipline, extra processes, and (of course ;-) an extra tool to do that. We saw (and see) these kind of arguments all over the place in waterfall projects. Second, estimation, a potentially risky view into the future, has always been THE problem in big bang developments. People try to predict minute detail, the prediction covering many months or even years. This is outside most people's capacity. Third and last, any single atomic requirement in a waterfall spec really IS meaningless. I hope we learn from the waterfall experience.

Jeff goes on:

"Shrinking stories forces earlier elaboration and decision-making."

Waterfall-like again, right?

If user stories get shrunk in order to fit in a time-box, there's another solution to the problem (besides larger development time-boxes, or using a pull system like Jeff has beautifully laid out): not make a user story the main planning item. How about "fraction of value delivered" instead, in per cent?

Jeff again:

"It’s difficult to fit thorough validation of the story into a short time-box as well. So, often testing slips into the time-box after. Which leaves the nasty problem of what to do with bugs which often get piped into a subsequent time-box."

This is nasty indeed, I know from personal experience. BTW, the same problem exists, but on the other side of the development time-box, if you need or want to thoroughly specify stories/features/requirements. Typical solution: you put it in the time-box BEFORE.

Let's again find a different solution using one of my favorite tools, the 5 Whys.

• Why do we put testing in the next time box? Because it consumes too much time.
  
• Why does it consume a lot of time? Because there is a significant number of defects to find and fix (and analyse and deploy and...), before we consider the product good enough for release. 
  
• Why is there a significant number of defects to be found with the testing stage? Because the product brings a significant number with it.
  
• Why does the test-ready product have a significant number of "inherent" defects? Because we have not reduced them significantly them further upstream.
  
• Why didn't we reduce it further upstream? Because we think testing is very effective in finding all kinds of defects, so testing alone (or along with very few other practices) is sufficient for high defect removal efficiency.
  

It is not. Period.

From an economic standpoint it is wise to do proper QA upstream, in order to arrive at all subsequent defect removal stages (including testing) with a smaller number of defects, hence with fewer testing hours needed. This works because defects are removed cheapest and fastest as close as possible to their origin.

What do I mean by proper upstream QA? Well, I've seen personally that inspections (on requirements/stories, design, code, and tests) deliver jaw-dropping results in terms of defects reduced and ROI. I'm sure there are a couple more, just ask you metrics guru of choice. The point is, see what really helps, by facts and numbers not opinions, and make a responsible decision.

Understanding the exponential function

I was pointed to a set of videos that explores the exponential function and what most people think about its consequences. Seems to be a boring topic.

It isn't. I watched the first 10 minutes, only to realize my jaw has dropped to the floor. This is stuff I knew and didn't grasp at the same time. I strongly suggest you invest the ∼75 minutes and watch it.

It hit me two days after I started to drivel about me liking sustainable solutions. I wasn't up to all my senses, obviously. ;-) 

Value Thinking: Using Scalpels not Hatchets

Ryan Shriver, managing consultant at Dominion Digital, has put out an excellent article on Value Thinking. I think his views are extremely relevant, as we see cost reduction programs all around the IT globe. I can't tell you how much Ryan is spot on, considering the organization I work for (sorry, can't tell you more in public).

The article is short, gives ample information on the WHAT and (more importantly) the WHY.

Ryan writes in due detail about 4 policies and 5 practices, his advice to struggling IT shops and IT departments.  Thanks Ryan!

It's on gantthead.com, and I recommend signing up if you aren't already, it's free.

Do you care about your customer, every day?

There's a wonderful parable on real customer satisfaction on the "i must be an acrobat" weblog by Joshua Hoover.

He compares eating out in a restaurant with inattentive waiters to developing software without delivering real results to a customer.

7 Habits of Highly Effective Risk Managers

I'm proud to announce that Raven, author of the wonderful Raven's Brain weblog published another guest post of mine.
It is on effective risk management, titled "7 Habits of Highly Effective Risk Managers".

As you can expect from my material it is a set of rules, that risk managers can easily apply to their jobs. It might also help if you are NOT the risk manager in your endeavor, but want to help someone who is.

Here's the condensed version:

R1: Be responsible.
R2: Analyse in depth.
R3: Use hard facts.
R4: Do more than one thing.
R5: It's not fire and forget.
R6: Integrate.
R7: Don't change the scales of measurement.

In the blogpost itself I explain every rule/habit in detail. Thanks Dick Karpinski for nitpicking and thanks Mr. Covey for the title.
Enjoy!

Lack of Engineering Practices Make 'Agile' Teams Fail

James Shore wrote a blog post that is among my TOP 5 in this quarter. Go read it! He argues that the industry sees so many agile projects fail because most teams that call themselves 'agile' mistake the real idea with the wrong set of practices. Maybe this quote says best what his article is about:

These teams say they're Agile, but they're just planning (and replanning) frequently. Short cycles and the ability to re-plan are the benefit that Agile gives you. It's the reward, not the method. These psuedo-Agile teams are having dessert every night and skipping their vegetables. By leaving out all the other stuff--the stuff that's really Agile--they're setting themselves up for rotten teeth, an oversized waistline, and ultimate failure. They feel good now, but it won't last.

This speaks from my heart. It's exactly what happened when I tried to turn a project 'agile' in a super waterfall organization (picking up this metaphor I'd say it's an Angel Falls organization). We will have technical debt to pay off for at least a decade. However, we found that out quite quickly ;-)
If you're looking for some method that is strong both in the management and engineering field, check out Evo by Tom Gilb. I know I recommended Mr. Gilb's work over and over again in the past 2 years, and it almost seems ridiculous. Be assured, I'm not getting paid by Tom :-)
To get a very clear perspective on iterative planning check out Niels Malotaux' Time Line.

All you want to know about Stakeholder Management

Alec Satin of the "Making Project Management Better" weblog has a brilliant list of links to Stakeholder management resources.

From his explainatory text:

In this list are articles to get you thinking about some of the less obvious issues in pleasing stakeholders. Primary selection criteria: (a) quick to read, (b) good value for your time, and (c) of interest whatever your level of experience." Markup is mine.

Enjoy!

7 Useful Viewpoints for Task Planning Oriented Towards Quick Results

Name: 7 Useful Viewpoints for Task Planning Oriented Towards Quick Results
Type: Rules
Status: final
Version: 2008-07-04

Gist: While planning tasks or activities, and at the same time focusing on quick results, it is a good idea to describe each task by means of different viewpoints. Here's my suggestion, derived from Tom Gilb's work, the BABOK and my own experience.

R1: Each task should be described in terms of
Purpose. Why are you doing this? This is the single most important category, because it may lead you to the understanding that this is not the right task.
Entry. What do you need in order to begin and complete the task? This prevents you from doing things without the proper basis, which would most likely causes unnecessary re-work.
Process. Which steps do you follow? This is a little plan for your task. It may be only one step, but most of the times there are more than one, even for small tasks. Splitting tasks into steps allows you to return to previous steps in case you learn something new.
Exit. What are the results of the task? What will be the benefit? This is what counts and what can give you the beautiful chance of realizing benefits 'along the way' some greater set of tasks.
Stakeholders. With whom will you have to speak? Whose requirements will you have to take into account? This prevents re-work caused by ignoring influential people.
Methods/Techniques. How can you do it? Note anything that may help you completing the task.
Time-Limit. How long do you plan to work on it? If you need substantially longer, you will have a clear indicator that something is wrong. Short deadlines help you focus (80/20 principle).

7 Rules for High Effectiveness

Name: 7 Rules for High Effectiveness
Type: Rules
Status: final
Version: 2008-07-03

Gist: Effective problem solvers develop a mind set of effectiveness. Here are the 7 Habits of Highly Successful People, from Stephen R. Covey, Simon & Schuster, Inc., 1989

R1: Be Proactive - take the initiative
R2: Visualize the end from the start - know where you're going ' it is better to do this quantitativly
Note: A good technique for doing R2 and R3 is the Kepner-Tregoe-Method, which is quite well described here. Better, still, is using Impact Estimation Tables (because of their much better handling ob quantitative information.)
R3: List priorities ' it is better, still, to do this quantitativly
R4: Think WIN/WIN
R5: Understand - listen, listen, listen / learn, learn, learn
R6 Synergize - make the whole more than the sum of the parts
R7: Seek Personal Renewal in various areas:
- Physical: exercise, nutrition, stress management
- Mental: reading. Thinking
- Spiritual: value clarification, meditation
- Social/Emotional: empathy, self-esteem

How to Personally Get the Most Out of Finished Projects

Name: How to Personally Get the Most Out of Finished Projects
Type: Process
Status: final
Version: 2008-05-19

Gist: Dustin Wax wrote an awesome article at the Lifehack web log. Topic: Getting Past Done: What to Do After You've Finished a Big Project
As it is significantly more stuff than fluff I will essentially post a link only, for once. However, you might like the following definition.

reflective thinking DEFINED AS active, persistent, and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends [that] includes a conscious and voluntary effort to establish belief upon a firm basis of evidence and rationality <- Dewey, J. 1933. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Lexington, MA: Heath.

S1: Read: Getting Past Done: What to Do After You've Finished a Big Project <http://www.lifehack.org/articles/productivity/getting-past-done-what-to-do-after-youve-finished-a-big-project.html> and extract the questions.

S2: Answer the questions in a quiet moment. You could also use them for a guideline in a lessons learned meeting.
Note: Reflection is method of learning with scientific proof of its effectiveness.

S3: Prepare a checklist for future use on finished projects (or link here).

8 Useful Tips to Pick an Analyst

Name: 8 Useful Tips to Pick an Analyst
Type: Rules
Status: final
Version: 2008-05-16

Gist: Every now and then I run into the task to pick a business analyst or systems analyst for a project or customer. Sometimes this starts with writing a job offer, sometimes with a colleague asking if I could take part in a job interview with some prospect. What should you look for?

R1: Clarify what tasks the candidate will have. 'Ok, this is a no-brainer

R2: Clarify what the challenge in this particular project or this particular job will be.

R3: Clarify what (analyst-specific) professional skills will be needed, now, and in the future, in this particular job.

R4: Clarify what soft skills will be needed, now, and in the future, in this particular job.

R5: Make sure the candidate knows these things (and has a sound approach to dealing with them):

• Stakeholders seem to change their mind on a regular basis. 'I think they don't, it just looks like a change
  
• Goals are seldom properly understood.
  
• Demand and Supply speak two different sociolects.
  
• Stakeholders (among other people...) have trouble communicating about future, i.e. the benefits, or the system-to-be.
  
• Other team members, including fellow analysts, architects, and QA-folk, deserve proper appreciation.
  
• A problem is a solution to a problem is a solution to... (also see the means-and-ands post)
  
• Many people (the prospect, too? Would be a bad sign for an analyst.) have a well developed answer reflex, so finding out what the problem really is can be quite a challenge.
  
• Any required product quality can be described in a meaningful, quantified way. (Refer to Tom Gilb's extensive and highly useful wokr on the subject.)
  

R6: Clarify what testing skills will be needed. 'An analyst that does not know how to design acceptance criteria?

R7: Look for someone who uses words like 'now and then', 'generally', 'frequently', 'some', 'somewhat', 'arguable', 'among other', 'on the other hand', 'also', 'able', 'allow'.

Note: The language gives you hints on the prospect's experience level. Experts tend to have many answers.

R8: Avoid people who use words like 'steady', 'always', 'every', 'all', 'absolute', 'totally', 'without doubt', 'nothing', 'only', 'entirely', 'without exception', 'must', 'have to'.

Why Managing Too Many Risks is Too Risky

I wrote a guest post over at the Raven's Brain weblog. The topic is 'Heaps of Risks - Why Managing Too Many Risks is Too Risky'.

Risk management is a source of unusual human behavior: euphoria or excessive gambling when risk is underestimated, and panic attacks or depression when we predict that things are riskier than they really are. Both are risks by themselves.
In the post I'll address a risk that is also closely related to human behavior and presents a risk: Overly extensive risk registers, i.e. a list of several dozends of risks for a given project.

Read more

If it feels too troublesome to leave a comment there, why not leave it here?

12 Helpful Rules for Refactoring a System

Name: 12 Helpful Rules for Refactoring a System
Type: Rules
Status: final
Version: 2008-04-25

Gist: to provoke 2nd thoughts if you are to take some part in a refactoring. Refactorings seem to come about more frequently these days, maybe because even the stolid of the business people start to talk about 'agile'. Like all these now-buzzwords, there's a lot of potential misunderstanding hiding in the very word.

Refactoring DEFINED AS modifying something without changing its externally observable behaviour with the purpose of making maintenance easier. Note that you can refactor many things, ranging from systems to architectures to design to code. However, relevant literature suggests that Refactoring is meant to relate to Design and Implementation. See Martin Fowler's Website. Also see Code Refactoring.

R1) Whenever you hear the word refactoring, run! 'just joking...

R2) Precisely understand what the various stakeholders think a refactoring is. You'll be surprised. Correct the wording if needed.

R3) Precisely understand what purpose the refactoring has, from each of the stakeholder's point of view. Align expectations before you start.

R4) Assume a great deal of resistance from the people who pay. Refactoring for them is another word for "Give me $$$, I'll give you what you already have! " Find out who will pay for maintenance.

R5) By all means reduce the risk of having a system with new bugs after the refactoring. This can be done by rigorous test driven or even behaviour driven development for example. Any change in behaviour should be treated as a bug.

R6) Challenge the business case (if any ...) of the refactoring. At least, do proper cost-benefit analysis. For example, the system may not live long enough to amortize refactoring.

R7) Prepare for the situation that nobody can provide exact and dependable information on how future budgets will be affected by the refactoring

R8) Back off from refactoring if significant portions of the code will be both refactored and worked over due to new requirements. Do one thing at a time and bear in mind R5.

R9) Prepare for 3 different points of view if you ask 2 people for what specific refactorings should be done.

R10) Think of short-term and long-term effects.

R11) The improvement must be both quantified beforehand and measured afterwards, at least for non-local changes to the system.

R12) Be suspicious if someone suggests only one or two specific refactoring techniques will suffice to reach a project-level goal.

5 Powerful Principles to Challenge Arguments

Name: 5 Powerful Principles to Challenge Arguments
Type: Principles
Status: final
Version: 2008-04-21

Gist: to provide a general checklist for the suspicious. The 5 principles show behavioural patters people use to make you do things in their favour, but not necessarily in your favour. Use this on (in alphabetical order) advisors, colleagues, consultants, counsels, managers, salespersons, vendors.

Sources: Scott Berkun: How to Detect Bullshit; and my all-time-favourite checklist: 12 Tough Questions by Tom Gilb.

P1) People are uncertain and tend to ignorantly stretch the facts.

• How do you know?
• What are your sources? How can I check them?
• Who told you that?
• Can you quantify the improvement?

Note: Carefully watch the answerer. If he needs a while, maybe uncomfortably shifting position, there's a good chance he's either making something up or needs time to figure out how to disguise a weak argument.

P2) People with weak arguments often have not made their homework on the topic.

• What is the counter argument?
• Who else shares this point of view?
• What are the risks of this, and what will you do about it?
• Can you quantify the improvement?
• How does your idea affect my goals and my budgets?
• What would make you change your mind?
• Have we got a complete solution?

Note: As from any facts, one can draw a set of reasonable interpretations, not just one. Everyone with intimate knowledge won't have great difficulties taking a different point of view for a while.

P3) People tend towards urgency when you are asked to make a decision with some hidden consequence.

• Can I sleep on this?
• When do we need to have a decision made? Why?
• I'd like to consult Person A first.
• Expert B, what do you think?

Note: People pressing ahead may try to throw you off your guard.

P4) People without a clear understanding of their point of view tend to inflate the language used.

• Please break this in smaller pieces, so I can understand.
• Explain this in simpler terms, please.
• I refuse to accept this until me, or someone I trust, fully understands it.
• Are you trying to say <...>? Then speak more plainly next time, please.

Note: Mark Twain once wrote in a letter to a friend: 'I'm sorry this letter is so long; I didn't have enough time to make it shorter.'

P5) People tend to have stronger arguments if they know someone is present who is hard to deceive.

• Use your network.
• Invite colleagues who have worked with these people.

Note: Simply help each other, like your family would do.

6 of the Dalai Lama's Leadership Principles

Name: 6 of the Dalai Lama's Leadership Principles
Type: Principles
Status: final
Version: 2008-04-18

Gist: to reflect upon the behavior of the 14th Dalai Lama when appearing publicly. It seems to me the following principles can also be applied in anyone's work in the position of some kind of leader. May be a project leader, a company's boss, the head of a department, or 'just" a thought leader within a group of people.

Sources: inspired by Cheri Baker at the Enlightended Manager Weblog; the 'Seeds of Compassion Webcast' from an event in Seattle, USA; some of the books the 14. Dalai Lama wrote; what people told me who had seen the Dalai Lama live.

P1: Don't direct, facilitate instead.
Note: This way, you find delight in the success and happiness of others.

P2: Make sure you want to learn something, so inquire rather that advocate. Listen, explore.
Note: In doing so, you'll show appreciation.

P3: Don't assume you know everything on the topic you're discussing. If someone brings something up that you think is wrong, show equanimity.
Note: There's the word 'equal' in equanimity.

P4: Be (and stay) humorous. Don't get angry, show compassion instead. Compassion means communication from your heart.
Note: This will facilitate creativity among the people you talk to.

P5: Encourage people to find their own answers, instead of giving all the answers you might have.
Note: also a good way of supporting P2, Learning.

P6: Be accountable.
Note: That means, you should pay attention to your actions and understand their consequences.

The Ominous Bill of Rights

Name: The Ominous Bill of Rights
Type: Principles
Status: final
Version: 2008-04-14

Gist: To present the Bill of Rights for a project development team (or contractor, originated by Tom Gilb), and to contrast it with a Bill from the customer / client point of view. I intend to balance Tom's view. PLEASE NOTICE, that these principles does not represent any official terms or principles used by my employer. This is, as all the other posts, a completely private matter.
Find a German version below.

Sources: The Contractor's Bill is from Gilb, Tom. 1988. Principles of Software Engineering Management. Wokingham and Reading, MA: Addison-Wesley. Page 23.
The Customer's Bill is out if my head.
Note: Tom did not call it 'Contractor's Bill of Rights', but only 'Bill of Rights'. It's clear what he had in mind if you read the specific principles.

Contractor's Bill of Rights:
P1: You have a right to know precisely what is expected of you.
P2: You have a right to clarify things with colleagues, anywhere in the organization.
P3: You have a right to initiate clearer definitions of objectives and strategies.
P4: You have a right to get objectives presented in measurable, quantified formats.
P5: You have a right to change your objectives and strategies, for better performance.
P6: You have a right to try out new ideas for improving communication.
P7: You have a right to fail when trying, but must kill your failures quickly.
P8: You have a right to challenge constructively higher-level objectives and strategies.
P9: You have a right to be judged objectively on your performance against measurable objectives.
P10 You have a right to offer constructive help to colleagues to improve communication.

Customer's Bill of Rights:
P11: We have a right to set the objectives for the endeavour.
P12: We have a right to change our minds about objectives at any time.
P13: We have a right to set the criteria by which we will measure your success.
P14: We have a right to be informed about exactly what you have accomplished at any time.
P15: We have a right to hear your arguments if you plan to change a decision that effects our objectives.
P16: We have a right to be informed of the consequences of a planned change, in measurable, quantified formats.
P17: We have a right to constuctively debate your planned change.
P18 We have a right to get clear documentation about any changed decision.

German Versions:
Rechte des Auftragnehmers:
Sie haben das Recht, genau zu wissen, was wir von Ihnen erwarten.
Sie haben das Recht, Dinge mit Kollegen aus der gesamten Projektorganisation zu klären.
Sie haben das Recht, klarere Definitionen von Zielen und Strategien anzustoßen.
Sie haben das Recht, Ziele in messbarer, quantifizierter Form präsentiert zu bekommen.
Sie haben das Recht, unsere Ziele und Strategien konstruktiv zu hinterfragen, um insgesamt ein besseres Ergebnis zu erzielen.
Sie haben das Recht, objektiv anhand messbarerer Ziele bewertet zu werden.
Sie haben das Recht, zu entscheiden über die Maßnahmen zur Erreichung der Ziele und ihre Reihenfolge.

Rechte des Auftraggebers:
Wir haben das Recht, die Ziele für das Vorhaben vorzugeben.
Wir haben das Recht, diese Ziele jederzeit zu ändern.
Wir haben das Recht, die Kriterien vorzugeben, anhand derer wir Erfolg messen.
Wir haben das Recht, jederzeit darüber informiert zu werden, was Sie bereits erledigt haben.
Wir haben das Recht, Ihre Argumente zu hören, falls Sie vorhaben Entscheidungen zu treffen, die sich auf die Zielerreichung auswirken.
Wir haben das Recht, über die Auswirkungen informiert zu werden, und zwar in objektiv überprüfbarer Art und Weise.
Wir haben das Recht, Ihre Entscheidung konstruktiv zu hinterfragen.
Wir haben das Recht, getroffene Entscheidungen klar dokumentiert zu bekommen.