12 Steps to Useful Software Metrics Course

12 Steps to Useful Software Metrics is a 3-day course designed to provide an in-depth practicum on the 12 steps to selecting, designing and implementing software metrics that will be useful to your organization. It introduces the attendees to a practical process for establishing and tailoring a software metrics program that focuses on their goals and information needs. The objective of this course is to provide a practical, start-to-finish process of “how to” select, design and implement metrics. It outlines a cookbook method that attendees can use to simplify the journey from software metric concept to delivered information. This course is intended to be team based. These teams will work together on exercises designed to not only practice learned skills but to achieve real work towards selecting and defining software metrics for use in their organizations.

This class starts with a basic overview of software metrics and measurement theory. The roles and benefits of measurements are discussed, standards and models related to software metrics are surveyed and a simple step-by-step process for selecting, defining and tailoring software metrics to meet the information needs of an organization is introduced.

Attendees will learn how to identify their software metrics customers and utilize the Goal/Question/Metric paradigm to select metrics that align with the organizational, project and process goals of those customers. Attendees practice designing metrics through standardizing entity and attribute definitions, choosing measurement functions, establishing measurement methods, defining decision criteria, designing reporting mechanisms and determining additional qualifiers.

Issues involved with data collection are discussed, including what data to collect, who should collect the data and how to collect it. Attendees learn to consider the human issues of implementing a measurement system and metric do’s and don’ts. Attendees also practice defining a human factors action plan for their metrics.

This course ends with a survey of product, process, services and project metrics used in the software industry.

Method of Instruction: This course is taught through lecture and interactive discussion. Actual examples from the software industry are utilized to make the information relevant. Throughout this course, learned skills are practiced using team exercises. The emphasis of this course is on techniques that allow the attendees to transition the skills learned in this course to their own work environments.

Target Audience: Software measurement and metrics specialists, project managers, functional managers, testers, quality engineering, developers, and other software project stakeholders involved in selecting, designing, implementing and utilizing software metrics and measures to obtain information about their software products, processes, services and projects.

Course Objectives: Upon successful completion of this course attendees will be able to:

Understand the basic metrics and measurement theory and terminology
Identify internal and external customers who need software metrics information
Select software metrics based on goals
Design and tailor the selected metrics to match your information needs
Determine what data to collect and who should collect it
Minimize the impact of human factors on the selected metrics
Interpret and communicate metric results

Course Materials: Student notebooks are designed to provide reference materials that can be utilized by the attendees long after the completion of the course. These materials include:

Copies of all presentation materials with annotated descriptive notes
A bibliography of reference materials
An index and glossary of Software Auditing Terms

Detailed Outline:

I: The Basics of Metrics & Measurement

1. Metrics & Measurement Theory

a. Demystifying Metrics

b. Software Metrics Defined

c. Measurement is common

d. Measurement Defined

e. Internal & External Attributes

f. Resource Entities & Attributes

g. Process Entities & Attributes

h. Product Entities & Attributes

i. Entities & Attributes- Exercise

j. Mapping System

k. Measurement Error

l. Validity & Reliability

m. Measurement Scales

2. Roles of Measurement

a. Roles of Measurement

b. Understand

c. Evaluate

d. Control

e. Predict

f. Effective Software Metrics

3. Standards, Models & Initiatives

a. ISO 9001:2000

b. ISO/IEC 15939

c. SEI SW-CMM^®

d. SEI CMMI^SM

e. IEEE Software Engineering Standards

f. Other Metrics Initiatives

4. 12 Steps to Useful Software Metrics

II: Selecting Metrics

1. Step 1 – Identify Metrics Customer

a. Two Historic Schools of Thought

b. Collecting Data on Everything

c. Implementing a Random Selection of Metrics

d. Step 1 – Identify Metrics Customer

e. Types of Customers

f. Identify Customer – Team Exercise

2. Step 2 – Target Goals

a. Goal/Question/Metric Paradigm

b. Step 2 – Target Goals

c. Translating Frustrations into Goals

d. Target Goals – Exercise

3. Step 3 – Ask Questions

a. Step 3 – Ask Questions

b. Drill Down to Lower-Level Goals

c. Ask Questions – Exercise

4. Step 4 – Select Metrics

a. Step 4 – Select Metrics

b. Selecting Metrics – Team Exercise

c. Selecting Metrics for Implementation

d. Selecting Metrics for New Processes

e. Selecting Metrics for Implemented Processes

f. Selecting Metrics for Improving Processes

g. Introducing Metrics Incrementally

h. Prioritizing Metrics Implementation

i. Evaluating Existing Metrics

j. Metrics Requirements Statement Template

k. Metrics Requirements Statement – Example

l. Benefits of a Metrics Requirements Statement

m. Metrics Requirements Statement – Team Exercise

III: Designing Metrics

1. Step 5 – Standardize Definitions

a. Why Standardization is Important

b. Step 5 – Standardize Definitions

c. Standardize Definitions – Example

d. Standardize Definitions – Team Exercise

2. Step 6 – Choose a Measurement Function

a. Step 6 – Choose a Measurement Function

b. Simplification

c. Selecting a Measurement Function

d. Tailoring a Function

3. Step 7 – Establish Measurement Method

a. Step 7 – Establish Measurement Method

b. Types of Measurement Methods

c. Counting Criteria – Examples

d. Measurement Functions & Methods – Example

e. Measurement Functions & Methods – Team Exercise

4. Step 8 – Define Decision Criteria

a. Step 8 – Define Decision Criteria

b. Decision Criteria for Control Type Metrics

c. Thresholds

d. Variance

e. Control Limits

f. Decision Criteria for Evaluate Type Metrics

g. Cost/Benefit Analysis

h. Analyze & Prioritize Choices

i. Entry/Exit Criteria

j. Decision Criteria for Understand & Predict Type Metrics

k. Confidence Level

l. Decision Criteria – Example

m. Decision Criteria – Team Exercise

5. Step 9 – Design Reporting Mechanism

a. Step 9 – Design Reporting Mechanism

b. Importance of Report Format

c. Graphical Techniques

d. Design Reporting Mechanism – Example

e. Report Timing

f. Report Delivery

g. Design Reporting Mechanism – Team Exercise

6. Step 10 – Determine Additional Qualifiers

a. Step 10 – Determine Additional Qualifiers

b. Additional Qualifiers – Team Exercise

IV: Implementing Metrics

1. Step 11 – Collect Data

a. Step 11 – Collect Data

b. Who Collects the Data?

c. Data Ownership – Examples

d. Data Collection Training

e. How to Collect Data

f. Data Collection Objectives

g. Inaccurate Data

h. Incomplete Data

i. Defining Data Collection - Example

j. Defining Data Collection – Team Exercise

2. Step 12 – Consider Human Factors

a. Step 12 – Consider Human Factors

b. Human Factors – What Not to Do

c. Human Factors – What to Do

d. Human Factors - Example

e. Human Factors – Team Exercise

3. Metric Report Definition Template

a. Metric Report Definition Template

b. Metrics Report Definition - Team Exercise

V: Project Metrics

1. Project Planning Metrics

a. Estimation

b. Industry Standard Size Metrics

c. Size - Lines of Code

d. Size – Function Points

e. Size – Other Size Metrics

f. Effort, Cost, Schedule Estimation Metrics

g. PERT Method

h. Estimation Models

i. Gantt Charts

j. Risk / Reward Balance

k. Risk Management Process

l. Risk Management Metrics

m. Risk Exposure

n. Risk Reduction Leverage

2. Project Tracking Metrics

a. Project Performance

b. Budget & Schedule Tracking Metrics

c. Earned Value

d. Gantt Charts – Tracking

e. Resource & Staff Tracking Metrics

f. Productivity Tracking

VI: Life Cycle Metrics

1. Requirements Metrics

a. Requirements Metrics

b. Requirements Size

c. Requirements Churn

d. Requirements Defect Density

e. Requirements Activity Status

2. Design Metrics

a. Design Metrics

b. Requirements Traceability

c. Design Defect Density

d. Design Activity Status

e. More Design Metrics

f. Design Size

g. Design Complexity

h. Cohesion

i. Coupling

3. Code Metrics

a. Code Metrics

b. Code Defect Density

c. Documentation Defect Density

d. Reuse

e. Code Activity Status

4. Test Metrics

a. Test Completeness Metrics

b. Test Coverage of Code & Calling Tree

c. Basis Path Testing

d. Design Predicate Approach

e. Operational Profile Testing

f. Test Sufficiency

g. Test Effort Variance

h. Test Activity Status

i. Problem Report Arrival Rate

j. Cumulative Problem Reports by Status

k. System Performance Metrics

l. Reliability

5. Maintenance Metrics

6. Product & Process Improvement Metrics

a. Product & Process Improvement Metrics

b. Defect Prone Components

c. Defect Escapes

d. Phase Containment Effectiveness

e. Defect Detection Efficiency

f. Defect Prevention

VII: Process Metrics For Success

1. Process Metrics for Success

a. Post Implementation Metrics

b. Post Release – Arrival Rate

c. Release-to-Release Defect Density

d. Reliability Actuals – Availability

e. Six Sigma

f. More Post Implementation Metrics

g. Problem Report Closure Metrics

h. Customer Satisfaction

Other Metrics Courses Include:

Software Metrics: is a 2-day course designed to provide an introduction to the basics of software metrics and how to establish a software metrics program combined with a survey of product, process, services and project metrics used in the software industry.

Facilitated Software Metrics Definition: A facilitated software metric definition workshop of one or more days can be added to either the Software Metrics or the 12 Steps to Useful Software Metrics classes.

Customized Courses: Our Software Metrics courses are modularized so that they can be easily customized for in-house course offerings that focus on the specific content and topics needed to meet your organization’s exact training requirements.

The Westfall Team can also customize this course or any of our other standard courses or develop unique software engineering, quality and project management courses to meet your exact in-house training needs and specifications. For example, class exercises can be tailored to include actual examples from your organization in order to make the training more relevant to your environment.

For more information about this course or other courses offered by The Westfall Team

Send an email to: lwestfall@westfallteam.com

Or call: 972-867-1172