|
The CMM establishes a set of public in available criteria describing the
characteristics of mature software organizations. These criteria can be
used by organizations to improve their processes for developing and maintaining
software, or by government or commercial organizations to evaluate the risks
of contracting a software project to a particular compa fb6 ny.
This chapter focuses on two SEI-developed methods for appraising the
maturity of an organization's execution of the software process: software
process assessment and software capability evaluation.
- Software process assessments are used to determine the state of an
organization's current software process, to determine the high-priority
software process-related issues facing an organization, and to obtain
the organizational support for software process improvement.
- Software capability evaluations are used to identify contractors who
are qualified to perform the software work or to monitor the state of
the software process used on an existing software effort.
This overview is not sufficient by itself for readers to conduct either
an assessment or evaluation. Anyone wishing to apply the CMM through these
methods should request further information on assessment and evaluation
training.
The CMM is a common foundation for both software process assessments
and software capability evaluations. The purpose of the methods are quite
different, however, and there are significant differences in the specific
methods used. Both are based on the model and the products derived from
it. The CMM, in conjunction with the CMM-based products, enables the methods
to be applied reliably and consistently.
Software process assessments focus on identifying improvement priorities
within an organization's own software process. Assessment teams use the
CMM to guide them in identifying and prioritizing findings. These findings,
along with guidance provided by the key practices in the CMM, are used (by
a software engineering process group, for example) to plan an improvement
strategy for the organization.
Software capability evaluations are focused on identifying the risks
associated with a particular project or contract for building high-quality
software on schedule and within budget. During the acquisition process,
software capability evaluations may be performed on bidders. The findings
of the evaluation, as structured by the CMM, may be used to identify the
risks in selecting a particular contractor. Evaluations may also be performed
on existing contracts to monitor their process performance, with the intent
of identifying potential improvements in the software process of the contractor.
The CMM establishes a common frame of reference for performing software
process assessments and software capability evaluations. Although the
two methods differ in purpose, the methods use the CMM as a foundation
for appraising software process maturity. Figure 4.1 provides a summary
description of the common steps in assessments and evaluations.
Figure 4.1 Common Steps in Software Process Assessments and Software
Capability Evaluations
The first step in is to select a team. This team should be trained in
the fundamental concepts of the CMM as well as the specifics of the assessment
or evaluation method. The members of the team should be professionals
knowledgeable in software engineering and management.
The second step is to have representatives from the site to be assessed
or evaluated complete the maturity questionnaire and other diagnostic
instruments. Once this activity is completed, the assessment or evaluation
team performs a response analysis (step 3), which tallies the responses
to the questions and identifies those areas where further exploration
is warranted. The areas to be investigated correspond to the CMM key process
areas.
The team is now ready to visit the site being assessed or evaluated
(step 4). Beginning with the results of the response analysis, the team
conducts interviews and reviews documentation to gain an understanding
of the software process followed by the site. The key process areas and
key practices in 190 the CMM guide the team members in questioning, listening,
reviewing, and synthesizing the information received from the interviews
and documents. The team applies professional judgment in deciding whether
the site's implementation of the key process areas satisfies the relevant
key process area goals. [ Begin Footnote ] --- These judgements may have
to take place without complete information whe fb2 n company proprietary
or security issues may be involved.--- [ End Footnote ] When there are
clear differences between the key practices in the CMM and the site's
practices, the team must document its rationale for judging that key process
area.
At the end of the on-site period, the team produces a list of findings
(step 5) that identifies the strengths and weaknesses of the organization's
software process. In a software process assessment, the findings become
the basis for recommendations for process improvement; in a software capability
evaluation, the findings become part of the risk analysis performed by
the acquisition agency.
Finally, the team prepares a key process area profile (step 6) that
shows the areas where the organization has, and has not, satisfied the
goals of the key process areas. A key process area can be satisfied and
still have associated findings, provided the findings do not identify
major problems that inhibit achieving any goals of the key process areas.
In summary, the software process assessment and software capability
evaluation methods both:
- use the maturity questionnaire as a springboard for the on-site visit,
- use the CMM as a map that guides the on-site investigation,
- develop findings that identify software process strengths and weaknesses
in terms of the key process areas in the CMM,
- derive a profile based on an analysis of the satisfaction of the goals
within the key process area, and
- present their results, to the appropriate audience, in terms of findings
and a key process area profile.
In spite of these similarities, the results of a software process assessment
or software capability evaluation may differ, even on successive applications
of the same method. One reason is that the scope of the assessment or evaluation
may vary. First, the organization being investigated must be determined.
For a large company, several different definitions for "organization" are
possible. The scope may be based on common senior management, common geographical
location, designation as a profit and loss center, common application domain,
or other considerations. Second, even in what appears to be the same organization,
the sample of projects selected may affect the scope. A division within
a company may be assessed, with the team arriving at findings based on a
division-wide scope. Later, a product line in that division may be evaluated,
with that team arriving at its findings based on a much narrower scope.
Comparison between the results without understanding the context is problematic.
Software process assessments and software capability evaluations differ
in motivation, objective, outcome, and ownership of the results. These
factors lead to substantive differences in the dynamics of interviews,
the scope of inquiry, the information gathered, and the formulation of
the outcome. The assessment and evaluation methods are quite different
when the detailed procedures employed are examined. Assessment training
does not prepare a team to perform evaluations, or vice versa.
Software process assessments are performed in an open, collaborative
environment. Their success depends on a commitment from both management
and the professional staff to improve the organization. The objective
is to surface problems and help managers and engineers improve their organization.
While the questionnaire is valuable in focusing the assessment team on
maturity level issues, the emphasis is on structured and unstructured
interviews as tools for understanding the organization's software process.
Aside from identifying the software process issues facing the organization,
the buy-in to improvement, the organization-wide focus on process, and
the motivation and enthusiasm in executing an action plan are the most
valuable outcomes of an assessment.
Software c 191 apability evaluations, on the other hand, are performed
in a more audit-oriented environment. The objective is tied to monetary
considerations, since the team's recommendations will help select contractors
or set award fees. The emphasis is on a documented audit trail that reveals
the software process actually implemented by the organization.
This does not mean, however, that the results of fdf software process
assessments and software capability evaluations should not be comparable.
Since both methods are CMM-based, the points of comparison and difference
should be evident and explainable.
For software engineering process groups or others trying to improve their
software process, the CMM has specific value in the areas of action planning,
implementing action plans, and defining processes. During action planning,
the members of the software engineering process group, equipped with knowledge
of their software process issues and business environment, can compare their
current practices against the goals of the key process areas in the CMM.
The key practices should be examined in relation to corporate goals, management
priorities, the level of performance of the practice, the value of implementing
each practice to the organization, and the ability of the organization to
implement a practice in light of its culture.
The software engineering process group must next determine which process
improvements are needed, how to effect the change, and obtain the necessary
buy-in. The CMM aids this activity by providing a starting point for discussion
about process improvement and by helping to surface disparate assumptions
about commonly accepted software engineering practices. In implementing
the action plan, the CMM and the key practices can be used by the process
groups to construct parts of the operational action plan and to define
the software process.
|
|