Walt Scacchi,
ATRIUM Laboratory,
Information and Operations Management Dept.,
School of Business Administration,
and,
Center for Software Engineering,
University of Southern California.
213-740-4782, 213-740-8494 (fax)
(Scacchi@gilligan.usc.edu).
(c) Copyright, Walt Scacchi 1996
This presentation can be accessed via the World Wide Web at:
http://www.usc.edu/dept/ATRIUM/Papers/sp-tutor.html
See the companion paper on the WWW at:
http://www.usc.edu/dept/ATRIUM/Papers/Software_Productivity.ps
Return to Top
Return to Top
- What affects software productivity?
- How do we improve it?
- How much might it be worth to achieve a 20% improvement in U.S.
software productivity [10]?
- Goal: review international sample of empirical studies
of software productivity for large-scale software systems
(50K-500K SLOC), from the 1970's through the early 1990's.
Introduction (con't)
- Exisiting software productivity studies
are inadequate and misleading.
- How and what you measure determines how much productivity improvement
you see.
- Prior work shows that small-scale programming productivity has more than
an order of magnitude variation across individuals and languages
[19]
.
- Overall, we find contradictory findings and repeated shortcomings
in productivity measurement and data analysis, among the few
nuggets of improved understanding.
Return to Overview or
return to
Table of Contents.
- Measurement is a quest for certainty and control
[18].
- The relationship between measurement and instrumentation must be
clear.
- Instrumentation choices lead to trade-offs concerning:
- Who/what should perform measurement
- What types of measurements to use
- How to perform the measurements
- How to handle problematic measurement data
- How to categorize and analyze measurement data
- How to present results to minimize distortion
- Most software productivity studies assume ratio
measurement data is preferred.
- However, nomimal, ordinal, or interval measures may
be very useful.
- Thus, what types of measures are appropriate for understanding
software productivity?
Return to Overview or
return to
Table of Contents.
Return to Overview or
return to
Table of Contents.
- Walston and Felix [56] measured lines of
code per person-hour (LOC/Effort).
- Effort measured duration of complete development project, not
just time spent programming.
- No indication of the relative distribution of effort of different
development activities.
- Thus, this omission may distort the results of their analyses.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Albrecht [2][3]
introduces composite measures called "function points"
as an alternative to LOC.
- Albrecht reports 3X productivity improvement over 5 years, substantiated
with function point measurements, due to:
- Use of modern programming and database languages
- Structured coding
- Top-down implementation
- HIPO documentation guidelines
Use of these techniques may tend to naturally increase
number of function points found in software.
Albrecht's formula for determining productivity relies upon "weighting
multipliers" of unclear origin.
Albrecht, as department manager, required his programming supervisors
to collect function point data, and then reward them if their productivity
improved.
Thus, it is unclear what lead to the 3X improvement in software
productivity he reports.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Behrens [5] utlizes function point measures
to assess software productivity across 25 insurance application software
development projects.
- Results are consistent with Albretch's.
- Finding: small teams produce more function points when
compared to large teams for comparable effort.
- Finding: online development produces more function points than
batched development.
- Does not address the limitations of function points (e.g., weighting
multipliers).
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Boehm's [9][12]
software cost estimation studies which give rise to COCOMO.
- Reports that software cost drivers resemble the inverse
of productivity or benefits drivers.
- Finding: personnel/team capability has greatest effect
in driving costs and productivity:
- High staff capability and low product complexity lead to high
productivity or low cost software.
- Low staff capability and high product complexity lead to low
productivity or high cost software.
- Finding: develops quantitative suport for relative contribution of
different software development product, process, or setting characteristics.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Lawrence [39] examined the development
of 278 commercial software applications in 23 Australian organizations.
- Used multi-variate (vs. univariate) analysis to examine
software productivity variance.
- Finding: LOC, number of procedure invocations, number of functional
units, and number of transfers of control are highly correlated.
- Finding: individual programmer productivity increases with faster
turnaround time, but decreases with online source code testing and
interfacing to a database (cf. IBM DP).
- Finding: experienced programmers, structured coding, and code
walkthroughs contribute little to productivity improvement
(cf. IBM DP), (cf. TRW).
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Bailey and Basili [4] found higher
productivity over the entire system life cycle is associated with
a disciplined programing methodology.
- Finding: productivity measurements and resource utilization must be
specific to the organizational setting and local computing
environment.
- Finding: program (or product) based productvity measures and
cost estimates are less accurate and less reliable than measures
that account for organizational and computing environment
characteristics.
- Mohanty [44] and Kemerer
[30] independently confirm this last item.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
IBM (early 1980's)
- Thadhani [54]
and Lambert [37] examined the effects of
computing service support (e.g., response time, programmer skills,
and program complexity) on programmer productivity.
- Finding: A 2X productivity improvement occurs when
computer response time is less than 0.25 seconds vs.
greater than 2 seconds -- faster computers are better!
- Assertion: Unexpected delay in response time is "psychologically
disruptive" to programmers performing simple but frequent tasks.
- Conte and colleagues [17]
, in contrast,
report that its not the amount of response time that
causes the disruption, but more the unexpected variance in response
time.
Return to Sample of Software Productivity Studies,
return to Overview, or
return to Table of Contents.
- Vosburg and associates [55] produced one of
the best studies to date.
- Systematic data on programming productivity, quality, and cost was
collected from 44 projects in 17 corporate subsidiaries in 9 countries,
accounting for 2.3M LOC and 1500 person years of effort.
- Finding: product-related and process-related factors account for
approximately the same amount (~33%) of productivity variance.
- Finding: you can distinguish productivity factors that can be
controlled (process-related) from those that cannot (product-related).
ITT Advanced Technology Center (con't)
Process-related factors (more easily controlled)
- concurrent hardware-software development
- development computer size
- requirements and specification stability
- use of "modern programming practices"
- personnel experience
Program-related factors (not easily controlled)
- computing resource constraints
- program complexity
- customer participation
- size of program product
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Jeffrey [26] compared productivity
measurements among small teams (1-8 members) in 38 development
projects in three
Austrailian firms.
- Each firm used a different programming language.
- Finding: there is an optimal staff level depenging on language used,
and size of resulting system.
- Finding: adding staff beyond optimal point decreases productivity and
increases total development elapsed time.
- Concern: individual programmer productivity variations
[19] were not taken into account.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Cerveny and Joseph [15] report on software
maintenance productivity in 200 U.S. banks.
- Each bank had to implement the same software enhancements, due
to external regulatory changes.
- Finding: banks using structured design and structured coding techniques
experienced 2X effort (0.5X productivity?) change, compared to
banks which did not, and to banks which purchased packaged software
solutions.
- No measures of software LOC or function points were reported.
- Most banks did not use CASE tools.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Cusumano and Kemerer [21] compared
productivity in 24 U.S. and 16 Japanese
software development firms [20].
- Data collected from software project managers via questionnaires.
- Managers may have reported only on their best projects.
- Fortran-equivalent noncomment source lines of code was the output
measure [27],
and person-years of effort as the input measure
- Parametric and non-parametric data analyses were employed.
- Finding: apparently higher rates for software productivity in Japan
were not statistically significant.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
- Recognized measures of productivity using
lines of code, and for cost of detecting and removing code defects
are inherently paradoxical.
- Paradox emphasizes that "more (SLOC produced) is better."
- Higher-level languages may therefore lead to "lower productivity"
as fewer source statements (or Function Points)
are produced to realize a given functionality
- Similarly, cost to detect and remove defects shows its cheaper
to repair poor quality programs than high quality programs.
- Therefore, separate productivity measures into work units and cost
units.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Sought to identify what characteristics of programming effort can
be objectively measured before the task begins, and what programmer skills
are related to effort efficiency.
- Studied 36 COBOL development projects in one organization.
- Does not describe resulting program size or number of programmers
involved.
- Results are similar in kind to Albretch's IBMDP
study.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- The classic survey in applying cost-benefit analysis to system
development and use.
- "Benefits" represent commonly cited productivity improvements.
- Costs are usually underestimated and difficult to control,
while benefits are usually overestimated and difficult
to achieve.
- Omission of significant costs and hidden costs
are among the most common estimation (or measurement) errors made.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Compared 20 software cost estimation models using actual data from
one development project.
- Finding: range of costs estimated covered an order of
magnitude (10X)!
- However, each model may embed different assumptions about cost
accounting practices and weighting factors.
- Hence, different cost/productivity measures can lead to different
outcome values, for the same project.
- Kemerer's [30] study corroborates this.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Argues that most software productivity measurement studies employ
inappropriate statistical analysis techniques.
- Most productivity data is ordinal, rather than interval or
ratio.
- This implies non-parametric statistical analysis techniques should
be employed, otherwise apparently stronger relationships may be
observed.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Poorly managed or poorly organized software projects have low
productivity.
- Identifies number of strategies for improving the organization of
software development work, which can improve software productivity
and quality.
- Example: Focus on cultivating and enhancing the commitment
of software developers to project goals and career growth.
- Highly committed developers will work diligently and more
effectively together, hence more productively, then compared to
low commitment developers.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Sought to identify how to improve software productivity at TRW
by 2X(4X) in 5(10) years.
- Focused on building a software development environment to improve
productivity.
- Environment contained many tools for managing project communications
and document production.
- First developers using this initial environment "believed" their
productivity improved 25-40% (1.25X-1.4X).
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Provides an account of problems and paradoxes involved
in measuring software productivity and quality.
- Finding: a line of code (or source statement) is not an economic good!
- Identifies more than 40 project variables that affect software
productivity.
- However, "empirical" data used to support his argument is over-smoothed and
from unnamed sources.
- Similarly, he does not explain how his model works, or what equations
its solves, in contrast to Boehm's COCOMO model
[9].
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Boehm reports that software productivity measurements often ignore
many important input and output measures.
- Inputs include:
- different life cycle phases requiring different levels of effort and
skill.
- production of documentation, facilities management, staff training,
etc.
- support personnel: contract administrators and project managers.
- computing platforms, communication facilities, and other organizational
resources.
- Outputs include:
- both simple and complex source code statements are counted equally.
- whether/how to count non-executable code, reused code, or
carriage returns as "source statements."
- whether to count source statements before or after post-processing
or optimization transformations.
- Result: metrics other than source statements are (still)
correlated with number of source statements, and do not offer
better understanding.
- Source statements (or other metrics of program syntax) are best
treated as no more than ordinal indicators when trying to
understand what affects software productivity.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Empirically examined the effect of teamwork in developing both
formal and informal software specifications.
- Finding: observed variation in productivity and specification
quality could be best explained in terms of recurring teamwork
structures.
- Six teamwork structures (patterns of interaction) were observed
across five teams, and team frequently shifted from one structure
to another.
- High productivity and high product quality results could be traced
back to observable patterns of teamwork.
- Lakhanpal's [38] study corroborates this.
- Teamwork structures, cohesiveness, and shifting patterns of teamwork
are also salient productivity variables.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Surveyed the "beliefs" of software engineers as to the effectiveness
of CASE tools on improving productivity.
- Finding: SE'rs believe the greatest improvement in productivity
will come from the use of CASE tools that enhance their ability to
produce various reports and analyses, interactive screen displays,
and structured diagrams.
- however, there is no systematic data or study that validates these
beliefs.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Studied organizational changes in worker productivity and quality
of work life resulting from introduction of large automated system.
- Some workers experienced productivity increases, while others
experienced productivity decreases.
- Recurring tasks were made easier, while uncommon tasks became more
difficult to complete.
- Distribution of organizational know-how shifted among users.
- Finding: attempts to improve productivity through introduction of
new environments may not be pareto optimal, but instead may
differential and unequally distributed.
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Report that programmers are 2X-4X more productive when coding Ada
versus Fortran or Pascal-like languages.
- However, they do not report whether the language constructs
that programmers' used are comparable.
- They also do not indicate whether program measures were applied
before or after post-processing, since Ada code formatters
have been found to as much as triple source statement
counts (cf. [10]).
Return to
Other studies of Software Productivity and
Cost
return to
Overview, or
return to
Table of Contents.
- Brynjolfsson [14] provides a
wide-ranging review of
empirical studies examining the relationship between information
technology and productivity.
- IT is defined to include software systems for transaction processing,
strategic information systems, and other applications.
- Examines studies drawn from multiple economic sectors in the
US economy.
- Finding: apparent "productivity paradox" in the development and use
of ITs is due to:
- Mismeasurement of inputs and outputs.
- Lags due to adaptation and learning curve effects.
- Redistribution of gains or profits.
- Mismanagement of IT within industrial organizations.
- Thus, one significant cause for our inability to understand software
productivity is found in the mismeasurement of
productivity data.
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
Return to
Summary of Software Productivity Drivers
return to
Overview, or
return to
Table of Contents.
- Provide substantial (and fast!) computing resource infrastructure
- Use contemporary SE tools and techniques
- Employ development aids that help project coordination
- Use "appropriate" programming languages
- Employ process-center development environments
Return to
Summary of Software Productivity Drivers
return to
Overview, or
return to
Table of Contents.
- Develop small-to-medium complexity systems
- Reuse of software that already addresses the problem
- No real-time or distributed software to develop
- Minimal constraints for validation of accuracy, security, and ease of
modification
- Stable requirements and specifications
- Short schedules to avoid slippages
Return to
Summary of Software Productivity Drivers
return to
Overview, or
return to
Table of Contents.
- Small, well-organized project teams
- Experienced development staff
- People who collect their own productivity data
- Shifting patterns of teamwork structures
Return to
Sample of Software Productivity
Studies,
return to
Overview, or
return to
Table of Contents.
Return to Overview or
return to
Table of Contents.
- Increase software production productivity or quality
- Develop more valuable products for lower costs
- Rationalize higher capital-to-staff investments
- Streamline or downsize software production operations
- Identify production bottlenecks or underutilized resources
- But trade-offs exist among these!
Return to Challenges for software
productivity measurement,
return to Overview or
return to
Table of Contents.
- Programmer self report
- Project or team manager
- Outside analysts or observers
- Automated performance monitors
- Trade-offs exist among these
Return to Challenges for software
productivity measurement,
return to Overview or
return to
Table of Contents.
Return to Challenges for software
productivity measurement,
return to Overview or
return to
Table of Contents.
- Delivered source statements, function points, and reused/external
components
- Software development analyses
- Documents and artifacts
- Application-domain knowledge
- Acquired software development skills
Return to What should be
measured,
return to Overview or
return to
Table of Contents.
- Requirements analysis: frequency and distribution of requirements
changes, and other volatility measures.
- Specification: number and interconnection of computational
objects, attributes, relations, and operations in target system.
- Architectural design: design complexity; the volatility
of the architecture's configuration, version space, and design team
composition; ratio of new to reused architectural components.
- Unit design: design effort; number of potential design defects
detected and removed before coding.
- Coding: effort to code designed modules; ratio of inconsistencies found
between module design and implementation by coders.
- Testing: ratio of effort allocated to spent on module, subsystem, or
system testing; density of known error types; extent of automated
mechanisms employed to generate test case data and evaluate test case
results.
Return to What should be
measured,
return to Overview or
return to
Table of Contents.
- Ad hoc problem solving and articulation work
- Project-oriented job shop
- Batched job shop for product families
- Pipeline or concurrent development
- Flexible software factory
- System integration assembly line
Return to What should be
measured,
return to Overview or
return to
Table of Contents.
- Programming language(s)
- Application type
- Computing platforms
- Disparity between host and target platforms
- Software development environment
- Personnel skill base
- Dependence on outside organizations
- Extent of client or end-user participation
- Frequency and history of mid-project platform upgrades
- Frequency and history of troublesome anomalies and mistakes
in prior projects
Return to What should be
measured,
return to Overview or
return to
Table of Contents.
Return to Challenges for software
productivity measurement,
return to Overview or
return to
Table of Contents.
- Quantitative surveys using sample populations and statistical
analysis
- Qualitative "field" observations and experiments
- Triangulation studies: combining the best of qualitative and quantitative
research designs, sampling, and analysis
- Simulation studies
Return to How to measure software
productivity,
return to Overview or
return to
Table of Contents.
- The life history of a software project in terms of
its products, processes, and production setting
Return to How to measure software
productivity,
return to Overview or
return to
Table of Contents.
Levels:
- Individual, Group/team, Department or organizational unit,
Division or company, or Industrial sector.
Terms (Perspectives):
- Technology-centered; Organization-centered; Customer satisfaction
and staff relations centered; Who controls, benefits most, or wins
(politically centered); Social interaction, discourse, and negotiation
of reality.
- Multiple concurrent analytical perspectives
Return to How to measure software
productivity,
return to Overview or
return to
Table of Contents.
- Get the best from well-organized people.
- Make development steps more efficient and more effective.
- Simplify, collapse, or eliminate development steps.
- Eliminate rework.
- Build simpler products or product families.
- Reuse proven products, processes, and production settings.
Return to Challenges for software
productivity measurement,
return to Overview or
return to
Table of Contents.
- Understanding software productivity requires a plethora of qualitative
and quantative data from multiple sources.
- The number and diversity of variables indicate that software
productivity cannot be understood simply as a ratio source code/function
points produced per unit of time/budget.
- A more systematic understanding of interrelationships, casuality,
and systemic consequences is required.
- We need a more robust theoretical framework, analytical methods, and
support tools to address current shortcomings.
Return to Challenges for software
productivity measurement,
return to Overview or
return to
Table of Contents.
Return to Overview or
return to
Table of Contents.
- Conventional measures of source statements and function points do
little in helping to understand or improve software productivity.
- We lack an articulated theory of software production.
- We need to construct models, hypotheses, or measures that account
for software production in different settings.
- These models and measures should be tuned to account for
the mutual influence of software products, processes, and setting
characteristics specific to a development project.
Return to Alternative directions
for software productivity measurement and improvement,
return to Overview or
return to
Table of Contents.
- Why are software developers so productive in the presence of technical
and organizational constraints?
- The potential for productivity improvement is not an inherent property
of any new software technology.
- Software developers must realize the potential for productivity
improvement.
- Technological impediments and organizational constraints can nullify
this potential.
- Thus, a basic concern must be to identify and cultivate software
productivity drivers.
- Examples include workplace and career incentives, teamwork structures,
new technology investments, high commitment staff, etc.
Return to Alternative directions
for software productivity measurement and improvement,
return to Overview or
return to
Table of Contents.
- Nominal, ordinal, and interval measures can be employed when
constructing qualitative process models of software production.
- These measures can incorporate subjective and impressionistic
data from local software development experts.
- These models and data can then be iteratively refined to determine
what further qualitative or quantitative data to collect.
- Quantitative data can then be used to substantiate or refute the
frequency or distribution of the findings described in qualitative
terms.
Return to Alternative directions
for software productivity measurement and improvement,
return to Overview or
return to
Table of Contents.
Return to Alternative directions
for software productivity measurement and improvement,
return to Overview or
return to
Table of Contents.
- Knowledge can be acquire via interviews, observational field studies,
and other enthographic strategies.
- Seek to collect data amenable to extremal and comparative analysis.
- The challenges of what should be measured
provide a starting point.
- Goal: to develop a descriptive model of software production
such that any conclusion can be traced back to the orginating data.
- Goal: to develop the descriptive model so that it can be
represented and processed within a knowledge-based system.
Return to Develop Knowledge-Based
Systems that Model Software Production,
return to Overview or
return to
Table of Contents.
- We need to employ a knowledge representation scheme that can accomodate
the kinds of data and measures already noted.
- Suggestive examples include the Schema Representation Language
[49][47] and
the Software Process Specification Language
[40][41][42].
- Goal: to facilitate computational analysis, simulation,
querying, and explanation of software production situations, events,
and activities.
Return to Develop Knowledge-Based
Systems that Model Software Production,
return to Overview or
return to
Table of Contents.
- Requires a knowledge base for storing facts, hueristics, and
reasoning strategies about software production.
- Requires a question-answering subsystem for retrieving facts
stored or deduced from known relationships among facts.
- Requires a simulator for exploring alternative trajectories for
software development given different production data.
- Can facilitate the automated construction of knowledge-based
software production support environments
[40][42][22]
.
Return to Develop Knowledge-Based
Systems that Model Software Production
return to Overview or
return to
Table of Contents.
- Requires an underlying computational model of development states, actions,
plans, schedules, expectations, histories, etc. in order to answer
dynamic "what-if" questions.
- Goal: the simulation should embody a deep model of software
production that can be substantiated and refined with quantified
data on the frequency and distribution of states, actions, etc.
arising during software production.
Return to Alternative directions
for software productivity measurement and improvement,
return to Overview or
return to
Table of Contents.
- Initiate comparative case studies or surveys of current software
practices.
- Clean and analyze collected data, via knowlege acquisition.
- Codify structured or patterned data in a knowledge specification
language, via knowledge representation.
- Simulate and iteratively refine, acquire, represent, and analyze
expanding knowledge of software production variables, and the
interrelationships.
- Embed knowledge-based software productivity support within
advanced software development or project management environment.
Return to Alternative directions
for software productivity measurement and improvement,
return to Overview or
return to
Table of Contents.
What affects software productivity and how do we improve it?
- State of the art in software productivity measurement falls short
of what's needed.
- Software productivity is affected by the interplay of software
products, production processes, and production settings.
- Software productivity must be measured using nominal, ordinal,
interval, and ratio measures.
- Multiple measures are needed to help articulate causal
patterns of software production.
- We can accept the naive results from current studies, or
choose to persue a new initiative for software productivity
knowledge engineering.
- Initial efforts at knowledge-based software process engineering
have produced encouraging results, and the growth of new understandings
about what affects software productivity.
Return to Top, or
return to Overview or
return to
Table of Contents.
1. Abdel-Hamid, T. and S. Madnick, Impact of Schedule Estimation on
Software Project Behavior. IEEE Software 3(4), (1986), 70-75.
2. Albrecht, A., "Measuring Application Development Productivity",
Proc. Joint SHARE/GUIDE/IBM Application Development Symposium
(October, 1979), 83-92.
3. Albrecht, A. and J. Gaffney, "Software Function, Source Lines of
Code, and Development Effort Prediction: A Software Science
Validation", IEEE Trans. Soft. Engr. SE-9(6), (1983), 639-648.
4. Bailey, J. and V. Basili, "A Meta-Model for Software Development
Resource Expenditures", Proc. 5th. Intern. Conf. Soft. Engr., IEEE
Computer Society, (1981), 107-116.
5. Behrens, C.A., "Measuring the Productivity of Computer Systems
Development Activities with Function Points", IEEE Trans. Soft.
Engr. SE-9(6), (1983), 648-652.
6. Bendifallah, S. and W. Scacchi, "Understanding Software Maintenance
Work", IEEE Trans. Soft. Engr. SE-13(3), (1987), 311-323.
7. Bendifallah, S. and W. Scacchi, "Work Structures and Shifts: An
Empirical Analysis of Software Specification Teamwork", Proc. 11th.
Intern. Conf. Soft. Engr., IEEE Computer Society, (1989), 345-357.
8. Bhansali, P.V., B.K. Pflug, J.A. Taylor, and J.D. Wooley, "Ada
Technology: Current Status and Cost Impact", Proceedings IEEE,
79(1), (1991), 22-29.
9. Boehm, B., Software Engineering Economics Prentice-Hall, Englewood
Cliffs, NJ (1981)
10. Boehm, B.W., "Improving Software Productivity", Computer, 20(8),
(1987), 43-58.
11. Boehm, B., M.. Penedo, E.D. Stuckle, R.D. Williams, and A.B. Pyster,
"A Software Development Environment for Improving Productivity",
Computer 17(6), (1984), 30-44.
12. Boehm, B. and R.W. Wolverton, "Software Cost Modelling: Some Lessons
Learned", J. Systems and Software 1 (1980), 195-201.
13. van den Bosch, F., J. Ellis, P. Freeman, L. Johnson, C. McClure,
D. Robinson, W. Scacchi, B. Scheft, A. van Staa, and L. Tripp,
"Evaluating the Implementation of Software Development Life Cycle
Methodology", ACM Software Engineering Notes, 7(1), (1982), 45-61.
14. Brynjolfsson, E., "The Productivity Paradox of Information
Technology," Communications ACM, 36(12), (1993), 67-77.
15. Cerveny, R.P., and D.A. Joseph, "A Study of the Effects of Three
Commonly Used Software Engineering Strategies on Software
Enhancement Productivity", Information & Management, 14, (1988),
243-251.
16. Chrysler, E., "Some Basic Determinants of Computer Programming
Productivity", Communications ACM 21(6), (1978), 472-483.
17. Conte, S., D. Dunsmore, and V. Shen, Software Engineering: Models
and Measures Benjamin-Cummings, Palo Alto, CA (1986).
18. Curtis, B., "Measurement and Experimentation in Software
Engineering", Proc. IEEE 68(9), (1980), 1103-1119.
19. Curtis, B., "Substantiating Programmer Variability", Proc. IEEE,
69(7), (1981).
20. Cusumano, M., Japan's Software Factories, Oxford Univ. Press, New
York, (1991).
21. Cusumano, M. and C.F. Kemerer, "A Quantitative Analysis of U.S. and
Japanese Practice and Performance in Software Development",
Management Science, 36(11), (1990), 1384-1406.
22. Garg, P.K., P. Mi, T. Pham, W. Scacchi, and G. Thunquest, "The SMART
Approach to Software Process Engineering," Proc. 16th. Intern. Conf.
Software Engineering, Sorrento, Italy, IEEE Computer Society,
(1994), 341-350.
23. Garg. P.K. and W. Scacchi, "On Designing Intelligent Software
Hypertext Systems", IEEE Expert, 4, (1989), 52-63.
24. Hanson, S.J. and R.R. Kosinski, "Programmer Perceptions of
Productivity and Programming Tools", Communications ACM, 28(2),
(1985), 180-189.
25. Irving, R., C. Higgins, and F. Safayeni, Computerized Performance
Monitoring Systems: Use and Abuse, Communications ACM, 29(8),
(1986), 794-801.
26. Jeffrey, D.R., "A Software Development Productivity Model for MIS
Environments", J. Systems and Soft., 7, (1987), 115-125.
27. Jones, T.C., "Measuring Programming Quality and Productivity", IBM
System J. 17(1), (1978), 39-63.
28. Jones, C., Programming Productivity, McGraw-Hill, New York, (1986).
29. Keen, P.G.W., "Information Systems and Organizational Change",
Communications ACM, 24(1), (1981), 24-33.
30. Kemerer, C.F., "An Empirical Validation of Software Cost Estimation
Models", Communications ACM, 30(5), (1987), 416-429.
31. Kemerer, C.F., "Improving the Reliability of Function Point
Measurement -- An Empirical Study," IEEE Trans. Software
Engineering, 18(11), (1992), 1011-1024.
32. Kemerer, C.F., "Reliability of Function Point Measurement -- A Field
Experiment", Communications ACM, 36(2), (1993), 85-97.
33. Kidder, T. The Soul of a New Machine, Atlantic Monthly Press,
(1981).
34. King, J.L. and E. Schrems, "Cost-Benefit Analysis in Information
Systems Development and Operation", ACM Computing Surveys 10(1),
(1978), 19-34.
35. Kling, R. and W. Scacchi, "The Web of Computing: Computing
Technology as Social Organization", Advances in Computers 21 (1982),
3-87.
36. Kraut, R., S. Dumais, and S. Koch, "Computerization, Productivity,
and Quality of Work-Life", Communications ACM, 32(2), (1989),
220-238.
37. Lambert, G.N., "A Comparative Study of System Response Time on
Programmer Development Productivity", IBM Systems J. 23(1), (1984),
36-43.
38. Lakhanpal, B., "Understanding the Factors Influencing the
Performance of Software Development Groups: An Exploratory
Group-Level Analysis," Information and Software Technology, 35(8),
(1993), 468-471.
39. Lawrence, M.J., "Programming Methodology, Organizational
Environment, and Programming Productivity", J. Systems and Software
2 (1981), 257-269.
40. Mi, P. and W. Scacchi, "A Knowledge-Based Environment for Modeling
and Simulating Software Engineering Processes", IEEE Trans.
Knowledge and Data Engr., 2(3), (1990), 283-294. Reprinted in Nikkei
Artificial Intelligence, 20(1) (1991), 176-191 (in Japanese).
41. Mi. P. and W. Scacchi, "Modeling Articulation Work in Software
Engineering Processes," Proc. 1st. Intern. Conf. Software Process,
IEEE Computer Society, Redondo Beach, CA, (1991)
42. Mi, P. and W. Scacchi, "Process Integration for CASE Environments,"
IEEE Software, 9(2), (May 1992), 45-53. Reprinted in Computer-Aided
Software Engineering, 2nd. Edition. Chikofsky (ed.), IEEE Computer
Society, (1993).
43. Mittal, R., M. Kim, and R. Berg, "A Case Study of Workstation Usage
During Early Phases of the Software Development Life Cycle", Proc.
ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical
Software Development Environments, (1986), 70-76.
44. Mohanty, S.N., "Software Cost Estimation: Present and Future",
Software-Practice and Experience 11 (1981), 103-121.
45. Norman, R.J. and J.F. Nunamaker, "CASE Productivity Perceptions of
Software Engineering Professionals", Communications ACM, 32(9),
(1989), 1102-1108.
46. Pengelly, A, M. Norris, R. Higham, "Software Process Modelling and
Measurement -- A QMS Case Study," Information and Software
Technology, 35(6-7), (1993), 375-380.
47. Reddy, Y.V., M.S. Fox, N. Husain, and M. McRoberts, "The
Knowledge-Based Simulation System", IEEE Software 3(2), (1986),
26-37.
48. Romeu, J.L. and S.A. Gloss-Soler, "Some Measurement Problems
Detected in the Analysis of Software Productivity Data and their
Statistical Significance", Proc. COMPSAC 83, IEEE Computer Society,
(1983), 17-24.
49. Sathi, A., M.S. Fox, M. Greenberg, "Representation of Activity
Knowledge for Project Management", IEEE Trans. Pattern Analysis and
Machine Intelligence, 7(5), (1985), 531-552.
50. Scacchi, W., "Managing Software Engineering Projects: A Social
Analysis", IEEE Trans. Soft. Engr., SE-10(1), (1984), 49-59.
51. Scacchi, W., "On the Power of Domain-Specific Hypertext
Environments", J. Amer. Soc. Info. Sci., 40(5), (1989a).
52. Scacchi, W., "Designing Software Systems to Facilitate Social
Organization", in M.J. Smith and G. Salvendy (eds.), Work with
Computers, Vol. 12A, Advances in Humans Factors and Ergonomics,
Elsevier, New York, (1989b), 64-72.
53. Scacchi, W., "The Software Infrastructure for a Distributed System
Factory", Soft. Engr. J., 6(5), (September 1991), 355-369.
54. Thadhani, A.J., "Factors Affecting Programmer Productivity During
Application Development", IBM Systems J. 23(1), (1984), 19-35.
55. Vosburg, J., B. Curtis, R. Wolverton, B. Albert, H. Malec, S. Hoben
and Y. Liu "Productivity Factors and Programming Environments",
Proc. 7th. Intern. Conf. Soft. Engr., IEEE Computer Society, (1984),
143-152.
56. Walston, C.E. and C.P. Felix, "A Method of Programming Measurement
and Estimation", IBM Systems J. 16(1), (1977), 54-65.
Return to Top, or
return to Overview or
return to
Table of Contents.
This interactive presentation
page is maintained by
Walt Scacchi
who can be reached at the e-mail address noted above. This page was last
updated on
28 March 1996.
Return to Top