Welcome to Food for Thought™, an e-newsletter from Software Quality Consulting. I've created free subscriptions for my valued business contacts. If you find this newsletter informative, I encourage you to continue reading. Feel free to pass this newsletter along to colleagues by clicking the Forward Email link at the bottom of this page. If you’ve received this newsletter from a colleague and would like to subscribe, please click this New Subscription link. If you don't wish to receive this newsletter, click the SafeUnSubscribe™ link at the bottom of this page, and you won’t be bothered again.

This month we discuss the issue of Best Practices and how can you determine if something purported to be a “best practice” is really going to make a difference for your organization.

Regular features to look for each month are:

• Monthly Morsel: Hints, tips, techniques and reference info related to this month’s topic

• Calendar: Conferences, workshops, and meetings of interest to software engineers, QA engineers and anyone interested in software development.

Best Practices

What’s best for your organization?

There’s been lots of talk about “best practices” over the years. What’s usually been lacking in all of this talk is:

• A definition for what constitutes something that is called a “best practice”
  
  
• A process by which a candidate practice is considered for the designation “best”
  
  
• Evidence that something called a “best practice” really works
  

What we need is the equivalent of the Good Housekeeping Seal for software development practices.

Further, while there is a generally high level of awareness regarding best practices, the adoption of so-called “best practices” tends to be very low.

The term “best practice” has clearly been overused and in many cases, misused:

“While there might be a few universally good practices (such as peer reviews), and there are certainly required practices (such as configuration and requirements management), most practices have hidden assumptions and conditions for use, and there is little empirical data to support evaluation and selection.”

Turner, R., “Seven Pitfalls To Avoid in the Hunt for Best Practices”, IEEE Software, Jan-Feb 2003, p. 67-69.

Let’s try to correct this situation by starting with some definitions of “best practice”:

The American Society for Quality defines “best practice” as a superior method or innovative practice that contributes to the improved performance of an organization, usually recognized as “best” by other peer organizations.

Capers Jones definition is

“…to be deemed a ‘best practice’, it would be reasonable to require that the practice must have been observed in a minimum of 10 companies and 50 software projects. Furthermore, the practice must be considered by those who used it to have been beneficial in all or most of the projects.”

Jones, C., Software Assessments, Benchmarks, and Best Practices, Addison-Wesley, 2000

A presentation at the 3rd OSD Conference on the Acquisition of Software Intensive Systems in January2004 offered this definition: An activity or procedure that has produced outstanding results in another situation and could be adapted to improve effectiveness, efficiency, ecology, and/or innovativeness in another situation.

The Interoperability Clearinghouse Glossary of Terms defines best practice as an activity or procedure that has produced outstanding results in another situation and could be adapted to improve effectiveness, efficiency, ecology, and/or innovativeness in another situation.

And there are dozens more definitions floating around. If we can’t seem to agree on a definition for best practices, how we say there are such things?

The truth of the matter is that the first part of the first definition given above is probably the most reasonable definition:

a superior method or innovative practice that contributes to the improved performance of an organization

It’s the second part of that definition (… usually recognized as “best” by other peer organizations ) that causes problems. For example:

• Who are these peer organizations?
  
  
• How do they recognize a practice as being “best”?
  
  
• Are there some hurdles that a candidate practice must meet before it earns the designation “best”?
  
  
• What evidence is there that something really is “best”?
  

More on these questions in a bit. First, let’s look at several groups that have identified so-called “best practices”.

Airlie Council

One of the most widely quoted sources of “best practices” is the Airlie Council.

The Airlie Council

The Airlie Council was convened in the late 1990’s by an order from Congress to identify software best practices that could significantly improve software acquisition activities on behalf of the Department of Defense. Through their efforts, they identified what has become known as the Nine Principal Best Practices. These include:

1. Formal Risk Management
2. Agreement on Interfaces
3. Formal Inspections
4. Metrics Based Scheduling and Management
5. Binary Quality Gates at the Inch–Pebble Level
6. Project–Wide Visibility of Progress to Plan
7. Defect Tracking Against Quality Targets
8. Configuration Management
9. People–Aware Management Accountability

Nine Best Practices - The Software Management Framework, prepared by the Niwot Ridge Consulting Group, June 1999
Get more info…

Software Productivity Research (SPR)

Capers Jones’ company was very active in collecting data on practices used by many companies to develop software. Through his surveys, SPR was able to identify those companies considered “best in class”. From these companies, he identified software development practices that these “best in class” companies had in common. The idea here is that these practices in some way were responsible for a company being considered “best in class”. These practices include:

1. Quality Measurements
2. Defect Prevention
3. Defect and quality estimation automation
4. Defect tracking automation
5. Complexity analysis tools
6. Test coverage analysis tools
7. Formal inspections
8. Formal testing by testing specialists
9. Formal quality assurance groups
10. Executive and managerial understanding of quality

Jones, C., Software Quality: Analysis and Guidelines for Success, Boston, MA: International Thomson Computer Press, 1997.

Software Program Manager’s Network (SPMN)

The SPMN was established in 1992 by the Assistant Secretary of the Navy to identify proven industry and government software best practices and convey these practices to managers of large-scale DoD system acquisition programs.  The SPMN 16 Critical Software Practices™ specifically address underlying cost and schedule drivers that have caused many software intensive systems to be delivered over budget, behind schedule and with significant performance shortfalls.  The SPMN 16 Critical Software Practices™ are organized in three groups: 

PROJECT INTEGRITY

1. Adopt Continuous Program Risk Management
2. Estimate Cost and Schedule Empirically
3. Use Metrics to Manage
4. Track Earned Value
5. Track Defects against Quality Targets
6. Treat People as the most important resource

CONSTRUCTION INTEGRITY

7. Adopt Life Cycle Configuration Management
8. Manage and Trace Requirements
9. Use System-based software design
10. Ensure data and database interoperability
11. Define and control interfaces
12. Design twice, code once
13. Assess reuse risks and costs

PRODUCT STABILITY AND INTEGRITY

14. Inspect requirements and design
15. Manage testing as a continuous process
16. Compile and smoke test frequently

Get more info…

Software Engineering Institute (SEI) Capability Maturity Model^SM (CMM)

Many organizations believe that the CMM^SM is useful as a resource for identifying and implementing so-called “best practices”. However, a quick visit to the SEI website reveals the following statement from the SEI:

“The Software CMM is twelve years old and no longer represents current best practices in software engineering.”

How can you determine if these so-called “best practices” will work for your organization?

Now that we have presented some definitions of what might constitute a “best practice”, we need to revisit the questions identified earlier:

• Who are the peer organizations who identify and recognize “best practices”?
  
  Currently, there are many organizations (as illustrated by the several sources above) who have claimed to identify so-called “best practices”. Currently, there is no one single organization dedicated solely to identifying and recognizing potential “best practices” for software development…
  

• How do these organizations recognize a practice as being “best”?
  
  Each of the organizations identified above use very different criteria for recognizing a practice as being “best”…
  

• Are there some hurdles that a candidate practice must meet before it earns the designation “best”?
  
  Again, since each organization has different criteria, the hurdles associated with achieving the designation “best practice” are also very different…
  

• What evidence is there that something really is “best”?
  
  Evidence (quantitative or qualitative) is sorely lacking. Capers Jones has been the primary source of what little data has been published (Get more info…). The IEEE Computer Society has published quantitative data on the effectiveness of Software Inspections. (Get more info…)

Sorting it All Out

The bottom line is that you need to apply common sense in determining if a so-called “best practice” might work well for your organization. Here are some tips that should help:

• Look at those practices that have been identified by more than one organization as a so-called “best practice”. This may help increase confidence that a practice will result in a positive improvement.
  

For example, if we look at these sources of potential “best practices” identified above, we can see that there is some overlap:

Note: The numbers in the Airlie Council, Best in Class Companies, and SPMN columns refer to the numbers in the respective lists above. The numbers in the SEI CMM column refer to the CMM level where the practice is required.

• Establish your organization’s current performance baseline. This is a critical and often overlooked step in helping to identify where software development improvement is needed.
  
  Identify a small number of key performance measures that you can use to baseline your organization’s performance. For example, one simple measure that can be indicative of an organization’s performance is the number of unplanned bug fix releases between major releases of software.
  

• Perform Root Cause Analysis on those performance measures in order identify areas where software development improvement is required. Then, identify practices that are likely to help improve performance in those areas.
  
  For example, for many organizations, decreasing time to market is a common desire. First, precisely define what is meant by this measure. What does it include, what does it not include. How is time to market measured? And, what is the current performance level?
  
  Once you’ve defined the measure, let’s say your objective is to decrease time to market by 10% over the current level. Now, you are in a position to review potential practices and select those that can have a positive impact on decreasing time to market.
  

• Determine if a change to your software development process had the desired affect.
  
  After instituting a change to your development process, say incorporating peer reviews, assess your baseline again and determine if the change achieved the desired improvement.

The Bottom Line

• You can’t expect the performance of your organization to improve if the continue to follow the same process.
  
  
• “Best practices” are only “best” if they work for you.
  
  
• Involve your staff in identifying opportunities for improvement.
  

In the coming months, we’ll discuss some of the so-called “best practices” identified in this article in more depth…

Every month in this space you’ll find additional information related to this month’s topic.

• Best Practice Information
  
  Get more info on the Nine Best Practices from the Airlie Council
  
  Get more info on the 16 Critical Software Practices™ from the SPMN
  
  Wheeler, D., et. al, editors, Software Inspection: An Industry Best Practice, IEEE Computer Society Press, 1996, Order Number BP07340
  

• Books and Articles
  
  
  1. Jones, C., Software Quality: Analysis and Guidelines for Success, Boston, MA: International Thomson Computer Press, 1997.
     
     
  2. Jones, C., Software Assessments, Benchmarks, and Best Practices, Addison-Wesley, 2000
     
     
  3. Nine Best Practices - The Software Management Framework, prepared by the Niwot Ridge Consulting Group, June 1999
     
     
  4. Harrison, W., “Best Practices: Who Says So?”, IEEE Software, Jan-Feb 2004, p. 8-9.
     
     
  5. Turner, R., “Seven Pitfalls To Avoid in the Hunt for Best Practices”, IEEE Software, Jan-Feb 2003, p. 67-69.

Every month, you’ll find news here about local and national events that are of interest to the software community …

• Software Quality Calendar
  
  There are many organizations that sponsor monthly meetings, workshops, and conferences of interest to software professionals. Find out what’s happening…
  
  
• Workshops Offered by Software Quality Consulting
  
  Software Quality Consulting offers workshops in many topics related to software process improvement. Get more info...

Software Quality Consulting provides consulting, training, and auditing services tailored to meet the specific needs of clients. We help clients fine-tune their software development processes and improve the quality of their software products. The overall goal is to help clients achieve Predictable Software Development™ – so that organizations can consistently deliver quality software with promised features in the promised timeframe.

I hope this newsletter has been informative and helpful. Your comments and feedback are most welcome. Send me your feedback…

Thanks,

Steve Rakitin

info@swqual.com