Avoid Requirements-related Defects in IT-Systems

Name: Avoid Requirements-related Defects in IT-Systems
Type: Process
Version: 2007-04-20
Referring to: Principles for Requirements Engineering Tasks.P3
Source: Sorry, I know I used somebody else's material here but I failed to write the source down. If you feel like "hey, that's my idea" please let me know and I will be more that happy to properly credit you.

Note: The sooner a defect ist detected, the cheaper is its removal. This is a well known fact, nicely pointed out by for example by this article at Tyner Blain <…>. Why? Because in systems engineering usually things are described more than once - from different perspectives. As a rather coarse requirement, as an architecture, as a detailed requirement, as a design, as a test, as code, in manuals. The more of these artifacts have to be changed due to a defect, the more expensive. Add the cost of bringing the system to the user.
Therefore it is a good idea to avoid defects in the first place.

Concerning requirements, defects occur in 4 different steps in the process:
S1: When stakeholders incorrectly or incompletely define their requirements.
S2: When analysts document the requirements incorrectly or incompletely. Writing good requirements is not too difficult.
S3: When developers write unit tests that test the wrong implementation. Even very effective methods of continuous integration then result in very effective testing of the wrong thing.
S4: When testers make sure, that the wrong requirements were implemented or that nat all of the requirements were realised. Then we waste our time by testing for requirements that aren't really requirements

What do we do about it? We add feedback cycles accordingly:
S1: We focus on the ROI, so the stakeholders concentrate on the important requirements. Using different requirements gathering techniques on the same stakeholders help a great deal. Stakeholders oficially approve the requirements that were documented by the analysts. (BTW, that means that the requirements must be readable by stakeholders)
S2: We use a strict Specification Quality Control process to make sure we get well written requirements. One or more analysts approve the work of one other analyst. The ultimate goal in doing so is to improve the author, not the specification itself.
S3: intensive communication with analysts and architects helps developers while comprehending the requirements.
S4: Test cases and test data will be approved by the analysts.

Nonfunctional requirements and level of specification

Name: Nonfunctional requirements and level of specification
Type: Theory
Version: 2007-04-18
Source: My friend Sven

"The higher the level of a system's requirements specification, the greater the portion of non-functional requirements."

This is, because no computer system can directly realise a non-functional requirement; it has to be broken down to a set of functional requirements. This set interacts and thereby attains (or does not attain) a certain non-functional attribute.

Sven's sentence describes a necessary characteristic of good specifications. In other words if you find a high-level spec with few or no non-functional requirements, it does not mean it is not high-level. It probably means that the author forgot to include non-functional requirements.

Also, "portion" is maybe not the correct term (he used the German "Anteil"), but I think the idea is clear.

Signs and Symbols

Name: Signs and Symbols
Type: Explanation
Status: Draft
Version: 2007-04-18
Rationale: This blog makes use of Planguage, a planning language from the Gilb family.

Here's my dialect:

E Entry-Condition
S Step in a process
P Principle
R Rule
X Exit-Condition
[…] Qualifyer, Condition
<...> not sufficiently specific
or
{…} set of things
F(x) s.th. that has been derrived from x using a function F
' commentary text afterwards
italics commentary text

Principles for Requirements Engineering Tasks

Name: Principles for Requirements Engineering Tasks
Type: principles
Status: finished
Version: 2007-04-28
German: RE-Prinzipien

The RE in my projects follows these guidelines:

P1: Requirements are driven by business; technical aspects or solutions to problems never constrain requirements without a clear, documented cause.
P2: Requirements with high business value will be analysed and realised first.
P3: We avoid defects in the system instead of testing them out.
P4: (Potentially) controversal discussions take place as early as possible. Unclear, risky or hard to describe requirements are analysed early.
P5: Bandwidth will be maximised while communicating requirements and while communicating on requirements.
P6: If a rule, standard or other regulatory asset - whether it originates from the outside or inside of the project - hinders understandability of a requirement, the rule will be ignored.
P7: Solutions (Designs) to requirements can occur, but only as suggestions w/o any legal consequences.
P8: The (document) "Requirements Management Plan" will be improved continuously.
P9: ruleset.PrioritizeInEvoDevelopments applies

Evaluate the presentations

Name: Evaluate the presentations
Type: rule set
Status: Draft
Version: 2007-04-18
Referring to: Process.Call for tenders

R1: Have the presentations scheduled with little time between them.
R2: Again have different evaluation tasks assigned to the people on the team (e.g. According to their roles in the project)
R3: Have a list prepared with questions you will definately ask each supplier
R4: be strict about presentation time

Evaluate the tenders

Name: Evaluate the tenders
Type: Rules
Status: Final
Version: 2008-03-28
Referring to: Process.Call for tenders
Source [R8]: Curt Finch, Will Your Potential Software Vendor Meet Your Requirements?

Note: In one of my projects, we evaluated the written tenders first, then the presentations. Step 2 yielded a substantial change in our evaluation results. This can be taken as a hint, that written statements are not very well suited for these purposes.

R1: Has the supplier really anticipated the specific problem? Note that it can be hard to find the individual information among the heaps of tandard text in a witten tender

R2: Again have the the evaluations tasks divided and assigned to different team members

R3: you can evaluate the categories {form, content, sensed quality}
Note: For evaluation, you can something similar to Simple Solid Decision Making. We once used the following simpler, but less solid scheme:
- count: small and large problems with the tender's text, quastions not answered, good points
- give school grades
- build a ranking
Your provider management or purchasing dept maybe can supply you with a spredsheet for that.

R4 [too many suppliers on the list for presentations]: use ranking

R5: Encourange communication among the evaluation team

R6: explicitly allow "subjective" statements (the whole evaluation is kind of subjective). This works towards keeping subjectiveness out of the other criteria

R7: [lots of questions by suppliers]: give a chance to improve the tender (Do that by gathering all questions anonymously , answer them and send them back to the suppliers. Or, better, improve your spec and resubmit it)

R8: [doubts whether the statements about the past are true]: check their references. Be suspicious if references call YOU. Be suspicious if a reference uses an email address from Yahoo!, Hotmail, ... Get a company phone number and call their main number instead of the references' individual number. Do a web search on the company's name and check their website. Call ALL references.

R9: Mind that recent research shows, that decision making is not independent from your emotions

Send the call for tenders

Name: Send the call for tenders
Type: rule set
Status: final
Version: 2007-04-26
Referring to: Process.Call for tenders

R1: make use of professional help
- purchasing department can be used to reduce prices
- provider management department can provide a longlist of possible suppliers

R2: The suppliers on the longlist should be evaluated. Check references the suppliers give on their website or that has been obtained by ther key account manager. Ask tough questions! This results in a short list.
R3: Have the call for tenders sent to all suppliers at the same time to avoid legal problems. Your provider management may well have a standard letter to use, in order not to forget anything.
R4: Allow questions. Handle this rather formally, because you want to redistribute questions and answers to the suppliers.
R5 [fear of unwanted phone calls by suppliers]: Don't forget to exclude your mobile phone number from you email :-)
R6: publish the dates for the supplier's presentations
R7: direct communication to 1 or 2 persons only (on both sides)

Prepare Call for Tenders

Name: Prepare Call
Type: rule set
Status: final
Version: 2007-04-26
Referring to: Process.Call for tenders

Note: Even a call for tenders benefits from sound planning!

E1: A list of evaluation criteria has been assembled an agreed upon by the team and its management.
E2 [COTS]: the requirements have to be defined and prioritized. Note: the supplier's proposals should clearly state which part of the product fulfills which requirement, to what extent.
- also keep total cost of ownership in mind
- don't just care for functions ("what"), but also think of quantified requirements like performance attributes (see DIN 66272, "how well"), resource attributes ("how much"), and general system architecture (check references given).

R1: Experts and stakeholders should be with the evaluation team.
R2 [no expertise]: Go get external know-how, from other parts of your company or even from outside.
R3: Again, get management involved (e. g. get constraints, imform regularly, invite them for the supplier's presentations)
R4: Clearly assign tasks concernig the call itself and its evalution to individual team members. Set deadlines.
R5: Consider preparing a document template to be filled by the suppliers.

Call for tenders

Name: Call for tenders
Type: Process
Status. Draft
Version: 2007-04-18
Rationale: This process can be used as a guideline for issueing and evaluating a call for tenders. It is suitable for requesting projects, products or other external services. It describes WHAT to do (the process itself) and what should be on your mind while doing it (resp. rule sets).

Note: A project I supported recently did a call for tenders in mid 2006. They wanted an external supplyer to develop a part of a IT system. The experiences gained during the time and some older experiences in requesting proposals for hardware components and COTS products went into the following descriptions.

E1: Management has stated or validated the need for an externally supplied solution.
E2 [COTS]: The requirements for the product has been (big-up-front-requirements, BUFR) and the people concerned really understand that defined requirements will change (by a rate of 2-3% per month).
E2 [others]: the requirements for the project or other service are defined.

S1: prepare the call
S2: send the call
S3: evaluate the tenders
S4: evaluate the supplier's presentations

X1: One supplier has been chosen.
X2 [COTS]: the specification has been updated.
X3: The time invested in the call for tenders (between preparation and descision) has not exceeded 5% of total projected time for the project

Note: in the project mentioned above preparation took 2 weeks, the suppliers had 2 weeks to answer, we needed 1 week for evaluation, added an extra chance for improvement for the suppliers (1 week), presentations 2 days, final evaluation and descision 1 day. This sums up to 6+ weeks or nearly 10% of total projected time for the project.

About me

I'm Rolf, born 1971 in Germany.
My studies in computer science ended quite successfully with a thesis on linguistic methods in requirements engineering. I got employed right away by the SOPHISTs, Germany-based consultants spezialising in requirements engineering. There, I was responsible for doing requirements work, later also organising requirements work, then also educating people about requirements work. As a nice by-product, I had the chance to co-author several books there.

After a couple of years I decided that this was enough consultant's work, went to Asia for a small sabbatical, and then joined Deutsche Post World Net. A rather large group that does logistics worldwide. I currently work there in the IT department for the MAIL subdivision. Right in the middle between stakeholders and developers, again I do and organise requirements work, and the aspect of informing colleagues about my ideas and methods got so strong finally, that I decided to start this blog. Please note that everything I write here is my own private matter, and that it does not in any way represent terms or other regulations of my employer.

In general, I'm interested in requiremets engineering and systems engineering. One of my strongest beliefs is that the IT world would be an even nicer place to be, if its people stopped telling stakeholders which solutions they have, and started to listen to the problems.
It would also help a great deal if more IT people started clear conceptual thinking.

About this blog

I started this blog for two reasons:

First, I needed a way to involve my colleagues. Before I kept all what appears as single posts in the blog with my private files on my computer. When I wanted to show something to someone, I had to send it by email, it wasn't always up-to-date etc. So I use the blog to communicate ideas.

Second, I needed a way to keep track of my ideas, preferably in a structured searchable manner, while I don't have my computer with me. So I use the blog to organise my work.
Please note that everything I write here is my own private matter, and that it does not in any way represent terms or other regulations of my employer.

I like the idea of the Robertsons: they called a little competition in 2005 named "stand on my shoulders". This is my list of personal exemplars. As I am proud of standing on the shoulders of so many clear conceptual thinkers of the recent and not so recent past, with equal pride I like to see that some of my ideas will have been build upon by others. This is a way to achieve eternal life.

Along these lines, this blog is constructed by recycled material to a great extent. I add my bit wherever I find it useful. However, some of other people's ideas cannot be improved as of now.

Enjoy!

Lean Product Development

Type: RuleSet
Status: final
Version: 2007-04-20
Souce:http://www.shmula.com/340/lean-for-software-interview-with-mary-poppendieck

R1: Flow (or low inventory, or just-in-time) (inventory DEFINED AS something partially done)
R2: no workarounds, problem exposure
Note: In case of problems you don't avoid but solve them. This may mean you have to stop.
R3: Eliminate Waste (waste DEFINED AS {something without value to the user, work partially done, extra feautures you don't need right now, stuff that causes delay})
R4: Deferred Commitments (DEFINED AS idea of scheduling decisions until the last moment when they need to be made, and not making them before that, because then you have the most information. Note: Mary called it 'delayed commitments', but to my ear that's too close to 'too late'.)
R5: Deliver Fast (DEFINED AS being able to quickly release software. You need proper procedures for that, like {excellent testing procedures, automated testing, stuff that is disjoint from each other (so as you add features you don't add complexity)}

RE-Prinzipien

Typ: Regel-Satz
Status: finished
Version: 2007-04-28
English Principles for RE Tasks

Das Anforderungsmanagement von folgt folgenden Prinzipien:
R1: Die Anforderungen werden fachseitig vorgegeben und nur wo notwendig technisch oder durch Lösungen eingeschränkt.
R2: Zuerst werden die fürs Geschäft wertvollen Anforderungen analysiert und realisiert.
R3: Fehler im System werden vermieden, nicht herausgeprüft.
R4: (Potentiell) kontroverse Diskussionen werden so früh wie möglich geführt. Unklare, riskante oder aufwändig zu beschreibende Sachverhalte werden so früh wie möglich geklärt.
R5: Die Bandbreite in der Kommunikation der Anforderungen und über die Anforderungen ist so groß wie möglich.
R6: Wenn eine Regel, eine Festlegung oder ein Standard, ob vom Projekt stammend, vom Unternehmen oder von außerhalb, der Verständlichkeit der formulierten Anforderungen entgegensteht, so wird sie missachtet.
R7: Lösungen zu Anforderungen werden nur in Form von optionalen Vorschlägen behandelt.
R8: Das AM-Konzept unterliegt einem kontinuierlichen Verbesserungsprozess.
R9: Es gilt AllgRegelnPAEE ("Allgemeine Regeln zu Priorisierung von Aufgaben in evolutionären Entwicklungen")

Zu RE-Prinzipien.R3: Anforderungsbezogene Fehler in IT-Systemen
Fehler in IT-Systemen zu beheben wird umso teuerer, je „später" im Entwicklungsprozess sie entdeckt werden. Dies ist so, weil im Systems Engineering ein- und dieselbe Sache mehrfach beschrieben wird: als grobe Anforderung, als Architektur, als feine Anforderung, als Design, als Test, als Code, in Handbüchern. Je mehr Artefakte aufgrund eines Fehlers zu ändern sind, desto teuerer. Hinzu kommen u. U. noch Kosten, eine Fehlerbehebung in die Fläche zum Anwender zu bringen.
Somit ist es zweckmäßig, Fehler von vornherein zu vermeiden.

Was das Anforderungsmanagement anbelangt entstehen Fehler an 4 Stellen im Prozess:
1) Wenn die Stakeholder ihre Anforderungen falsch oder unvollständig definieren.
2) Wenn Analytiker die Anforderungen falsch/unvollständig dokumentieren. Gute Anforderungen zu schreiben ist kein Hexenwerk.
3) Wenn Entwickler Unit Tests schreiben, die die falsche Implementierung testen. Selbst mit effizienten Verfahren zur ständigen Integration von Implementierungsartefakten wird damit nur sehr effizient das Falsche getestet.
4) Wenn Tester sicherstellen, dass die falschen / nicht alle Anforderungen implementiert wurden. Man verschwendet Zeit damit, Anforderungen zu testen (unf ggf. Fehler zu beheben), die gar keine sind.

Welche Strategie verwenden wir im Projekt dagegen? Wir bauen an den entsprechenden Stellen Feedback-Zyklen ein.

Zu 1): Den ROI des Projektes in den Mittelpunkt zu stellen bewirkt, das sich die Stakeholder auf die wichtigen Anforderungen konzentrieren. Verschiedene Erhebungstechniken helfen dabei. Die Stakeholder geben die von den Analytikern geschriebenen Anforderungen inhaltlich frei.
Zu 2): SQC-Prozess zur Sicherung der Anforderungsqualität. Ein oder mehrere Analytiker geben die Erbeit eines Analytikers formal frei. Ziel ist vorrangig die Verbesserung des Analytikers, nicht der Spezifikation
Zu 3): zwischen Analytikern, Architekten und Entwicklern hilft sicherzustellen, dass die Entwickler die Anforderungen verstanden haben.
Zu 4): Die von den Testern erstellten Testfälle und Testdaten werden von den Analytikern freigegeben.

Principles of Clear Thinking

Type: Rules
Status: Draft
Version: 2007-12-20
Quelle: Principles of Clear Thinking. <- Blog on http://www.blogger.com/www.gilb.com 2007-03-27 (R1 to R10); rest: own thoughts
R1. You have to have a clear set of objectives and constraints, to evaluate proposed solutions or strategies against.
R2. You have to have a reasonable set of facts about the benefits and costs of any proposed idea, so that you can relate it to you current outstanding requirements.
R3. You have to have some notion of the risks associated with the idea, so that you can understand and take account of the worst possible case.
R4. You have to have some ideas about how to test the ideas gradually, early and on a small scale before committing to full scale implementation.
R5. If there are more than very few factors involved ( 2 to 4) then you are going to have to use a written model of the objectives, constraints, costs, benefits, and risks.
R6. If you want to check your thinking with anyone else, then you will need a written model to safely and completely share your understanding with anyone else.
R7. You will need to make a clear distinction between necessities (constraints) and desirables (targets).
R8. You will need to state all assumptions clearly, in writing, and to challenge them, or ask ‘what if they are not true?’
R9. You will want to have a backup plan, contingencies, for the worst case scenarios – failure to fund, failure for benefits to materialize, unexpected risk elements, political problems.
R10. Assume that information from other people is unreliable, slanted, incomplete, risky – and needs checking.
R11. Assume that you models are incomplete and wrong, so check the evidence to support, modify or destroy your models.

AllgRegelnPAEE

Name: General Rules for Prioritizing Tasks in Evolutionary Development
Type: RuleSet
Status: final
Version 2007-04-29
Purpose: What to do next?
Source: Niels Malotaux, www.malotaux.nl

R1: Most important requirements first
R2: Highest risks first
R3: Most educational or supporting for development first
R4: Actively Synchronise with other developments
R5: Every cycle delivers a useful, completed, result