Many times, it takes more than one metric to understand, evaluate or control a software product, process, service or project. One way to show a summary view of a set of metrics is to use a Kiviat chart, also called a polar chart, radar chart or spider chart. This brief article by Linda Westfall defines Kiviat Charts and gives examples on using them.
Software Metrics, Measurement and Analytical Methods
(An excerpt from The Certified Software Quality Engineer Handbook by Linda Westfall – soon to be published by ASQ Quality Press)
A Software Quality Engineer (SQE) needs to know when and how to use different sampling techniques in order to effectively use sampling during product and project management, audits, testing, and product acceptance. Section VI.C.1 of the ASQ Certified Software Quality Engineer (CSQE) Body of Knowledge (BOK) says that an SQE should be able to, “Define and distinguish between sampling methods (e.g., random, stratified, cluster) as used in auditing, testing, product acceptance, etc.”. [CSQE BOK]
When the set of all possible items in a population is very large it may be too costly or time consuming to do a comprehensive analysis of all of the items. For example, during an audit, there is just not enough time or resources to talk to every auditee, witness every process step or look at every quality record. If the customer base is large, it may be too costly to survey all the customers to determine their satisfaction level. Evaluating or estimating attributes or characteristics of the entire system, process, product or project through a representative sample can be more efficient while still providing the required information. To legitimately be able to use a sample to extrapolate the results to the whole population requires the use of one of four statistical sampling methods.
This paper introduces the reader to a practical process for establishing and tailoring a software metrics program that focuses on goals and information needs. The process provides a practical, systematic, start-to-finish method of selecting, designing and implementing software metrics. It outlines a cookbook method that the reader can use to simplify the journey from software metrics in concept to delivered information.
Date Original Version Posted: January 3, 2002
Updated Version Posted: April 25, 2005
Utilize this template to document the design of your software metrics reports. This template can help you implement the 12 Steps to Useful Software Metrics.
Date originally Posted: Sept 11, 2002
Updated Version Posted: July 8, 2004
Example of a completed Metrics Report Definition document.
Software metrics don't solve problems – people solve problems. What software metrics can do is provide information so you can make informed decisions and better choices. According to the new ISO/IEC 15939 Software Engineering -- Software Measurement Process standard, decision criteria are the “thresholds, targets, or patterns used to determine the need for action or further investigation, or to describe the level of confidence in a given result”. In other words, you need decision criteria to obtain guidance that will help you interpret the measurement results. This paper shows you how to establish useful decision criteria for different types of metrics.
This paper won the Best Paper award at the 2003 Applications in Software Measurement (ASM) Conference.
This paper describes a Process Measurement FrameworkSM that rapidly achieves measurable results. The Process Measurement FrameworkSM is based upon the popular Goal/Question/Metric (G/Q/M) paradigm, the Juran Quality Trilogy, and the initial core measures recommended by the Software Engineering Institute (SEI). The G/Q/M Paradigm is applied to the goals of planning, control, and improvement and based on powerful metrics that have a proven track record. In order to illustrate the power of the Process Measurement FrameworkSM, real examples from industry are used. The Process Measurement FrameworkSM helps to ensure that all metrics are collected on a form, in a document, or in a database.
This paper won the best paper award at the 13th International Conference on Software Quality, October 2003.
Satisfying our customers is an essential element to staying in business in this modern world of global competition. We must satisfy and even delight our customers with the value of our software products and services to gain their loyalty and repeat business. Customer satisfaction is therefore a primary goal of process improvement programs.
So how satisfied are our customers? One of the best ways to find out is to ask them using Customer Satisfaction Surveys. These surveys can provide management with the information they need to determine their customer's level of satisfaction with their software products and with the services associated with those products. Software engineers and other members of the technical staff can use the survey information to identify opportunities for ongoing process improvements and to monitor the impact of those improvements.
This paper includes details on designing your own software customer satisfaction questionnaire, tracking survey results and example reports that turn survey data into useful information. This paper won the Best Paper award at the 2002 Applications in Software Measurement (ASM) Conference.
This brief article by Linda Westfall discusses the purpose of Cause & Effect Diagrams and gives two examples of how this quality tool can be utilized.
This brief article by Linda Westfall defines the metric of Defect Density and gives two examples of how it could be reported and utilized.
Metrics and Models in Software Quality Engineering, 2nd Edition, Stephen Kan, Addison-Wesley, Reading, MA, 2002.
Software Metrics: Best Practices for Successful IT Management, Paul Goodman; Rothstein Associates, Inc., Brookfield, Connecticut, 2004.
Practical Software Measurement: Objective Information for Decision Makers, John McGarry, David Card, Cheryl Jones, Beth Layman, Elizabeth Clark, Joseph Dean and Fred Hall, Addison-Wesley, 2002.
Software Metrics: A Rigorous and Practical Approach, Norman Fenton and James Bieman, CRC Press, 2014.
Applied Software Measurement; Global Analysis of Productivity and Quality, Capers Jones, McGraw-Hill, 2008.
Measuring Customer Satisfaction: Survey Design, Use, and Statistical Analysis Methods, 3rd Edition; Bob E. Hayes; ASQ Quality Press, Milwaukee, Wisconsin; 2008.
Measuring the Software Process, Statistical Process Control for Software Process Improvement; William Florac & Anita Carleton; Addison-Wesley, Reading, MA; 1999.
Practical Software Metrics for Project Management and Process Improvement; Robert Grady; PTR Prentice Hall, Englewood Cliffs, New Jersey; 1992.
Linda Westfall's 12 Steps to Useful Software Metrics Blog - westfallteam.com/recent-12-blogs
Capers Jones' Blog - namcookanalytics.com
International Function Point User’s Group (IFPUG) - ifpug.org
The IT Metrics and Productivity Institute - itmpi.org
Practical Software and System Measurement (PSM) - psmsc.com