Software Process Reengineering Working Group Final Report
Walt Scacchi,
ATRIUM Laboratory, and
Center for Software Engineering,
University of Southern California.
213-740-4782, 213-740-8494 (fax)
(Scacchi@gilligan.usc.edu).
This report provides a summary of the working group on software process
reengineering. Broadly construed, the summary addresses two kinds of
results:
-
Recurring themes and topics, addressing items that many of
the WG participants repreatedly raised, addressed, or debated.
-
Technology roadmap: needs and efforts at USC, attempting to address
what research areas should be investigated at the CSE over the coming years
in order to best serve the software process (re)engineering
needs of the CSE industry affiliates.
Each of these will now be further described in turn.
Three themes or topic areas that garnered an emerging consensus for
their importance to software process reengineering were:
- Process metrics, referring to an overall
lack of first-hand experience or research insight into how to measure
existing or proposed software processes, so as to evaluate their
quality, flexibility, relevance, efficiency, etc.
- Institutionalizing process reengineering
and change, indicating that there is often substantial organizational
effort and resources needed to develop,
install, train, revise, or overhaul software
development processes and their associated "organizational culture."
- Reengineering process engineering paradigms,
suggesting that perhaps we must also begin to consider whether we need
new ways to think about, organize, and perform software process
(re)development, use, and evolution
in complex industrial or government settings.
Each of these can be further examined as follows.
Faced with the challenge of how to summarize the many different issues and
positions taken within the WG on the topic of software process metrics, the
WG agreed to simply state a following set of questions which highlight
their discussion:
- What metrics are needed?
- How should we measure process quality or "goodness"?
- How should we define process
metrics that the community can use to assess
effectiveness of process change or reengineering?
- Is process "baselining" (measuring or modeling the current
"as-is" process) necessary for software process reengineering?
The consensus view within the WG was
these questions represent topics appropriate for academic research, and perhaps
the research funding agencies could be tapped to support such investigations.
Changing, renovating, or innovating the current ways and means that
industrial firms and government organizations do (or do not) formulate,
perform, measure, and redesign their software development processes
in a major challenge for both practice and research. The concerns
raised in this regard posed in the following questions:
- What are the successful processes now in practice?
Can they serve as process benchmarks?
- What works, what does not work, and why do they work or not work?
- What resources, beyond process scripts or procedures, are needed
to implement process change?
- How can we rapidly "pilot" (or prototype) new software processes?
- How can we rapidly construct, analyze, and enact processes tailored
for specific organizational or business needs?
- How can we sustain the effectiveness of in-place processes, while
transitioning to new processes? Does process prototyping help?
- How do we institutionalize process flexibility?
Here, the was some debate as to whether their was an relevant research
at hand to shed light on these questions, as well as whether current
academic or industrial research projects would be able to address these
questions. Thus, in a sense, these questions point to the problematic
gap in our collective understanding of what is needed to institutionalize
a "new" culture that fosters the continuous proactice of
software process reengineering.
Given the preceding questions and dilemmas, the WG discussed and debated
the relative merits of adopting radically new ways of thinking
about the "theory and practice" software process reengineering.
- "Darwinian" (long life cycles) versus "Genetic" (short live cycles)
models of software process. One position asserted that conventional
software development projects are organized and performed over relatively
long time frames (months to years) with low frequency (meaning that it
may not be done using the as-formulated processes again with the same
people or support infrastructure). As such, the analyst (or project manager)
is tacitly encouraged
or constrained to adopt a "Darwinian" mind set when trying to understand
software process "evolution." The analogy here is that Darwin, when
studying the evolution of biological species, adopted a perspective
that saw system evolution occurring very slowly over long epochs.
Alternatively, with the rise of genetics in the 20th Century, a "genetic"
view of evolution began to emerge which could examine system evolution
experimentally, through the observation and "controlled management"
of biological species (e.g., fruit flies) whose life cycle was very short,
occurring with great frequency relative to human observation time periods.
Thus, the point here was to suggest that perhaps we need to either
focus our attention to (a) software development processes that have
relatively short cycle times and managable high observation frequencies,
or (b) to redesign software development into processes that can be
performed and managed both rapidly and frequently.
- Do we need some form of agile process (re)engineering that can be
tailored to different types of software development efforts?
Part of what was at issue here is that software development processes
are often too generic thereby offering limited support and guidance.
There was a WG consensus that software development for systems
such as
- In-house information systems or MIS
- Commericial or shrink-wrap products
- Embedded systems and applications
- Product manufacturing and test support systems
might require and benefit from different or more specialized development
processes. For example, the ongoing development of operating systems
software within a computer manufacturer is generally overwhelmed by
the large size and complexity of the (a) software source base, (b) development
team, and (c) insatiable volume of requested software alterations. In such
a setting, software process reengineering may be a luxury that can
only be undertaken infrequently and with low risk. Alternatively, a
software systems consultancy may be able to heavily invest in software
process improvements and radical redesigns if they can enable the development
of new classes of software products or services, or new sources of
revenue associated with existing software development engagements.
Thus, the concensus within the WG was that software processes need
to be (re)engineered with respect to
the type of software development activity, system, and workplace at hand.
What about workflow-driven approach to software process
(re)design?
While we are beginning to see the influence of "groupware" tools and
techniques entering into software development processes and support
environments, there is little citable crossover of approaches (or results)
employed in workflow technologies. Thus, whether or not there can or
should be some crossover of concepts and mechanisms from workflow
remains an open question. However, addressing such a question is well
within academic research capabilities.
How is business process redesign similar to or different from software
process redesign?
All members of the WG could cite cases or tales of successful
business process reengineering efforts, which in some instances produced
"silver bullet" class results (i.e., factor of 10 or greater improvements
in throughput or reduction in cost/cycle time). What is it about BPR
that can lead to such dramatic results? What can we learn from BPR
that can be applied to SPR? Again, this seemed to be an area that
should be readily addressable by academic research.
As far as the nascent practice of software process redesign has
developed so far, there was some recognition of the need to address
questions such as (a) how to avoid process suboptimization, and
(b) how do we determine, change, or shift process "granularity"
to accomodate:
- Radical vs. incremental process evolution
- Global process redesign
- Redesign of selected or "appropriate" subprocesses
Each of these questions again raised an awareness of the need for further
research and exploration, together with the overall view that such
topics might be best addressed through academic and industrial R&D
collaborations.
At this point, the WG sought to articulate some overall statement of
direction for what's needed or sought by the CSE industrial affiliates
by way of future research (3-5 years out)
in the area of software process reengineering. Two topics were then
identified:
Each is described in turn. However, to be accurate, the following
descriptions outline more of a statement of possible direction for
research investigation, rather than a detailed technology roadmap.
Time constraints did not permit such an elaboration by the WG. Nonetheless,
these directions can be presented on their own terms as follows.
The overall view of both industry and government affiliates in the WG was
that future software environments must accomodate processes that support:
- Distributed everything: people, processes, platforms,
tools, target applications, etc. with different distribution
topologies possible for each.
- Legacy systems and architectures
(which may also need to be reengineered)
- Commercial of the shelf (COTS) products
- New code and components
- Glue code (architectural middleware)
- Models and simulations of application subsystems
- New and old data sets and repositories
- New and old software engineering tools and environments
- Built-in process guidance for how to accomodate these.
Accordingly, the affiliates expressed desire to see ongoing
CSE research activity in this area.
In response, it was noted that
USC CSE and ATRIUM Lab projects are currently researching this area.
Professors Boehm and Scacchi are now working on a research project
sponsored by the USAF through Rome Laboratories to develop a roadmap
and demonstration prototype for how models and simulations of software
subsystems for domains such as C3I and MIS can be incorporated into
the "front-end" of software acquisition, requirements analysis, and
architectural specification processes. Professor Scacchi is also
leading a project sponsored by ONR that is investigating tools and techniques
for supporting and integrating distributed,
heterogeneous, and autonomous information systems, data sets, and repositories
across wide-area networks. Thus, there is effort now in place at USC
that is investigating this topic area, thereby posing the opportunity
for future contributions to CSE industrial affiliates.
A more ambitious formulation of the preceding problem area was cast
under the heading of developing a
process-driven operating system for virtual organizations.
This was seen as a long-term, high-risk, high reward type of endeavor.
It can be briefly characterized as follows.
An organizational operating system is a computing environment
that supports the development, use, and evolution of complex business
processes.
These processes include, but are not limited to, software (re)development
processes, and they are specialized to represent and address organizational
requirements while supporting high-performance process work teams.
Processes are enacted in the environment to create products or provide
services using available resources that are connected to, or
embedded within, an organization's information infrastructure.
Accordingly, business goals and organizational objectives can be formulated
as computational processes that can be iteratively executed, monitored,
and continuously improved.
Advanced organizational operating systems will operate over wide-area
networks, and support the integration of heterogenous, autonomous software
applications, tools, data sets, and repositories, whether new or legacy,
and whether located within or across organizational boundaries.
The required support will need to include process construction, modeling,
analysis, resourcing, enactment, measurement, verfication and validation,
and management of process assets and repositories.
It was then noted that Professor Scacchi is currently directing
USC ATRIUM Lab projects researching this area.
At present, this includes activity addressing
process life cycle engineering environment
(see
http://www.usc.edu/dept/ATRIUM/Process_Life_Cycle.html) and a
distributed, wide-area hypermedia infrastructure for integrating tools,
data, repositories, and processes (see
http://www.usc.edu/dept/ATRIUM/Papers/DHT-95.ps).
Thus, as above, there is effort now in place at USC
that is investigating this topic area, thereby posing the opportunity
for future contributions to CSE industrial affiliates.
Final note
In closing, we note that there is an accompanying interactive presentation
of this report posted on the WWW that can be found at
http://www.usc.edu/dept/ATRIUM/SPR.html.
This report and presentation were prepared and
maintained by Walt
Scacchi who can be reached at the e-mail address noted above.