(Re)Engineering Research Grants Management:
From Acquisition Reform to Knowledge Brokering at ONR
Walt Scacchi1, John Noll1,
Cedric Knight2, and Capt. Felton "Jay" Miller3
University of Southern California, Los Angeles, CA.
Technologies Inc., Ridgecrest, CA.
Office of Naval Research, Arlington, VA
Copyright © 1997 Walt Scacchi, The University of
In this paper, we briefly describe our approach and experience in a research
effort focused on (re)engineering the activity of research grants
management at the Office of Naval Research. We
found that we could contribute to a substantial reduction in process cycle
time and operational costs associated with the funding of thousands
of research grant procurement actions. Accordingly,
we focus our discussion on topics that underlie these results.
We also observe that knowledge brokering is an area
where a new R&D initiative could lead to more effective and efficient
research funding and research program management, as well as serve
the mutual self-interests of the Federal research funding agency
and researcher communities.
Keywords: Intelligent information integration, electronic transactions
and electronic commerce technologies, knowledge-based process engineering,
process-driven intranets for research management.
We have been involved in a multi-year research project investigating
ways to reinvent and reengineer corporate financial operations in
military procurement and acquisition organizations. Most recently, this
effort has been directed at the management of research grants by
the Office of Naval Research using electronic commerce and knowledge-based
process engineering technologies.
Through our research effort, we have found that
dramatic reduction in the cycle time and operational costs of research
grants management processes can be achieved. Internal ONR performance
measures now reveal that a factor of 10 in reduction of
process cycle time has been
realized in procurement action lead time (PALT). PALTs, used
for cross-industry benchmarking purposes, indicate the elapsed time at ONR
from a research grant funding authorization to the time when the
grant recipient can begin to expend their funding award. As more than 5000
research grant funding actions are performed per year at ONR (1995-1997),
of average PALT from 70 days (in 1994) to 7 days (in 1997, with further
reductions possible) represents a significant demonstration of acquisition
reform at ONR. In addition, some of the information processing and
workflow redesign that we collectively developed with ONR personnel
has led to the identification of annual operational savings estimated in the
range of $10M-$15M, and elimination of these expenses henceforth.
How did we achieve these results? What research methods
and prototype tools did we employ to achieve these results? Can similar
order of magnitude improvements be made in managing and "brokering"
scientific research programs by ONR program managers and scientific
officiers (i.e., the people who solicit and review research proposals,
and collectively recommend funding actions)?
This paper seeks to briefly explain what we did in our research
to achieve our results, and our thoughts for the extension
of this line of research to other federal research grants agencies.
Additional details beyond the scope of this paper can be found in
materials that have been posted on the WWW, or as listed in this paper's
Furthermore, we also identify an emerging opportunity for
further research, namely,
how this effort can be expanded to address
knowledge brokering processes at ONR and elsewhere.
But first, we want to specifically address topics that are relevant
to the Workshop on R&D Opportunities in Federal Information Services.
Knowledge-Based Process Architecture Research projects at the USC
have primarily focused on the knowledge-based engineering of complex
organizational processes. Since 1990, these efforts have investigated
the development of tools, techniques, and concepts for engineering
organizational process architectures in domains such as
software engineering, new product development, supply
chain logistics, corporate financial operations,
software acquisition, and
military procurement. These process architectures
are knowledge representations that model the processes, products,
organizational roles and team composition, information infrastructure,
and development tools central to an organization in its "routine"
work operations. A descriptive characterization of the knowledge ontology
that we employ that allows us to rapidly transition our efforts across
different organizational domains can be found elsewhere
[MS96]. Similarly, the knowledge-based tools and
techniques we employ in engineering these process architectures
across the process life cycle is also described elsewhere
[SM96]. Accordingly, in this study, our focus was directed
at four major processes of ONR's research grants management activity:
Grants Pre-Award (proposal solicitation), Award (funding decision and
obligation), Administration (funds disbursement and field office operations),
and Close-out (completion and reporting compliance).
Application Domain for Multiple Federal Agencies
In 1994, ONR awarded a research grant to the ATRIUM Laboratory to
investigate the development of alternative process architectures
military procurement at the Naval Air Warfare Center, Weapons Division,
in China Lake, CA. This effort was later extended to investigate
ONR research grants management processes, principally those involved
in ONR's Acquisition Directorate. In both situations, the ATRIUM
Laboratory was configured to operate as a kind of process
reinvention collaboratory [cf. KMW96]
where personnel from NAWC/ONR could
meet off-/on-site with USC researchers, supported with process elicitation,
visualization, integration, and execution support tools that could
be accessed over the Internet
[NS91,SM96,NS97b].While our efforts
were of modest scale, we note that NAWC-WD is among the largest of the
US Navy's 1000+ procurement centers, and ONR is the largest of DoD's
research grant agencies (e.g., Darpa, AFOSR, ARO) in terms of
grant actions, as well as one the Federal government's largest research
grants agencies (NSF, NASA, DOE, etc.). Thus, results we might acheive
through our research efforts may be applicable to a large number of government
information service centers.
Effort Needed to Obtain Useful Research Results
The application domain of procurement and acquisition is not
"rocket science." Instead, it is usually considered a back office
business activity concerned with corporate financial operations,
expenditure management, and status reporting. It is an activity
that is governed by a large number of changing acquisition regulations
and policies at Federal, DoD, and Navy levels. Federal Acquisition
Regulations (FARs) apply to all government agencies, including those
involved in funding research grants.
In many ways, the processes and artifacts used to
manage procurement contracts are quite similar to those used to manage
research grants. Similarly, the personnel at ONR who administer grant awards
from ONR's five field offices in the US, are also the same people who manage
contract fulfillment obligations as well as FAR compliance and
reporting requirements from research institutions.
Thus, the domain of procurement and acquisition of research
grants and service contracts for Federal agencies is likely to be highly
tractable and sufficiently structured to enable successful
domain knowledge engineering. Furthermore, what we learn about ONR's
research grants management process architecture may be refined and tuned
for application in other DoD or Federal research grants agencies with
Addressing Barriers to Resistance
Procurement and Acquisition Divisions are populated with personnel that
are usually not specialists with advanced information technology.
These people are not computer scientists, nor can they be expected
to be familiar with knowledge-based tools, techniques, or concepts
know being employed within the research community. Furthermore,
they may often be expected to "resist" the intervention by "outsiders"
whose purpose may be perceived as eliminating their jobs or administrative
authority. Nonetheless, ONR like other government agencies, is under
substantial pressure to accept increasing workloads with shrinking budgets.
Our approach to understanding the process architecture of research grants
management activities at ONR was based on involvement, participation,
and (intellectual) engagement of personnel from the top to the bottom
of the organization chart. We needed to educate ONR personnel
in our motives and methods, and they needed to educate us on the
generic and circumstantial variants of their work processes and information
flow. Two or three iterations were typically performed, particularly
with key domain experts. Follow-up validations by other personnel
not necessarily involved in these iterations were also performed.
Furthermore, agreements were established early
on between the research team and ONR personnel covering the following items:
As a result of these efforts and agreements, we found little or no
resistance, since our efforts were defined and structured as inherently
collaborative in purpose, method, and outcome.
- the research
team would identify multiple opportunities for
(re)design of work processes
information flow, and information integration (See
Appendix for examples);
- effort would be directed at improving personnel effectiveness and workflow
without increasing anyone's workload--personnel had to be more satisfied
with the new work arrangements;
- the processes
examined would be developed in three forms: AS-IS (present
form), TO-BE (alternative process architecture), and transitional forms
(steps taken in 30 day increments to evolve from the AS-IS to TO-BE forms);
- no new personnel positions would be created;
- ONR personnel would make final decisions on the selection of improvement
alternatives that would be implemented;
- any improvements to be implemented had to be "self-motivating" or
enable local organizational incentives to increase the likelihood of their
successful implementation and routinization.
Key Enablers and Support Technologies
Our method and agreement for research engagement as noted was
a key enabler for achieving the results we did. Similarly, the
engineering tools, techniques, and concepts that we have been developing and
experimenting with at USC for the past seven of so years, were a key enabler.
This of course should be no surprise--we built them, we use them, we evolve
them to meet our emerging research needs. We have an investment in making
them a key enabler, as well as serving to differentiate our effort
when competing for external research funding.
Nonetheless, we cannot be satisfied on this basis alone. Instead, we
must find ways to make our research technologies accessible as prototypes to
external customers and users, such as personnel at ONR Headquarters
and at its five field offices across the US. As such, part of our research
effort has been directed at prototyping an Internet-based
information infrastructure that could be used to capture, analyze, convey,
prototype, demonstrate, and refine alternative architectures for
organizational processes, such as ONR's research grants management activity
Such an infrastructure must eventually be able to support activities
In turn, such a prototype can serve as testbed or process-driven
intranet for ONR research management. Accordingly, we can employ this testbed
to demonstrate delivery of a research project's inputs and outputs,
in a form that can be accessed, engaged, and served across ONR's
multiple sites. Otherwise, those familiar with the Electronic Research
in which five Federal research agencies (ONR, NIH, DOE, AFOSR, DOT)
currently participate may observe that such an infrastructure
addresses a spectrum of NewERA concerns. However, in our research
project, we have the constraint (or "luxury") of limiting our
investigation to a single research agency (ONR), together with
(a) the ability to identify and prototype alternative process architectures
for research grants management, and (b) access to an (in-progress) integrated
information infrastructure that can directly support activities such
as those just noted.
- identification of Fleet and basic scientific needs, leading to the
establishment of new research programs
- preparation, review, revision, and distribution of electronic research
- receipt and integration of external research grant awards or programs from
other Federal agencies (Darpa, Nasa, DOT, etc.)
- preparation, submission, review and revision of electronic research
proposals and budgets
- integration of multiple heterogeneous information systems
and data repositories (INRIS, CAMIS, STARS, etc.)
that at present asynchronously record research
- preparation, distribution, and administation of electronic research
grant award packages, containing records of all grant actions pertaining
to a research grant award (funding increase or decrease, incremental
funding, grant renewals, no-cost funds extension, address changes, etc.)
- electronic data interchange for electronic invoicing and
electronic funds transfer transactions between ONR and research institutions
- on-demand tracking and reporting on the status of in-progress procurement
actions, and research program funding obligations, encumberances, and actual
expenditures (expenditure management)
- field office monitoring of regulatory compliance, record keeping
practices, and resource control systems
(e.g., for tracking equipment or property purchased with research grant funds)
at grant receiving institutions
- receiving, tracking, archiving, querying, retrieving, and browsing
findings, reports, or online prototypes resulting from research grant
- conveying research progress and orchestrating advanced technology
demonstrations for customers within the Fleet
in order to substantiate, expand, or decrease further research
Are the Research Results Rapidly Deployable and Demonstrable?
The relative ease with which our research results can be deployed outside
of ONR is a matter of opinion. Nonetheless, we can demonstrate and
provide WWW-based presentations on what we have done, how it
was accomplished, and how it might be applied, reproduced, or reused
in other Federal research grants agencies. To this end, we have developed
an exploratory scenario for how external government agencies that fund research
grants through ONR (e.g., Darpa, NASA) might operate.
Specifically, a recurring problem of internal and external research
programs is tracking the status of
expenditure management: how much unallocated
research funding is available at present for funding obligations,
and how much is being or has been actually spent (encumbered or expended).
For example, if PALTs require process completion times measured in months
or weeks, then uncertainty, misunderstanding, and
organizational inefficiency can occur relative to the status of
funds availability. Reducing PALTs
to days (or even to hours!) can reduce some of these dilemmas.
being able to address practical problems such as expenditure management,
which turn out to be of great importance in
research funding and reallocation decision-making (i.e., "strategic"
decision-making situations by research program officiers and division
managers), may likely determine the eventual success in being able to
apply research efforts such as ours in other settings. Such a scenario
can be described in more detail at the Workshop, if the opportunity arises.
Related Areas for Research Attention: Knowledge Brokering
Most of what has been described so far focuses on acquisition and
research grants management activities, and how a research effort
such as ours can lead to significant reductions in process cycle time
and operational cost. However, there remains perhaps an intimately
related area for further investigation that we are seek
to address. This concerns the activities involved in the establishment,
management, and fulfillment of research programs by Federal agencies.
At ONR, the source of problems to be addressed by a
research program is often Commands within the USN Fleet. In turn,
Commands within the Fleet are also the source of the budget authority
providing the funds to be expended in acquiring research results
through independent investigations. ONR program officiers (also called
science officiers), program managers, and division directors
must increasingly organize and manage knowledge brokering processes.
Knowledge brokering refers to the activities of an organizational
agency whose brokers (program officiers) find "servers" (researchers) who can
"marshall, integrate, and deliver" services and results
(create new knowledge, conduct experiments, prototype technology, produce
research reports, etc.)
for "clients" (the Fleet). The scope of activity that knowledge brokers
at ONR must articulate is growing. Many program officiers must now
organize R&D programs that span from basic research studies, through
applied research and prototyping studies, to advanced technology
demonstrations that address customer needs. These processes presently
take years, a decade, or more to complete.
Whether focused only on ONR,
or more broadly at any/all Federal reseearch agencies, a number of
basic questions can be asked about the processes, architectures,
artifacts, support systems, etc, that support Federal knowledge
brokering activities: What are these processes, architectures, etc.?
How do they work, how do they work best, and how do they go wrong?
Can they be computationally modeled, analyzed, simulated, and so forth
across their life cycle? How might Federal knowledge
brokering process architectures be redesigned for optimal and adaptive
performance? Can their
cycle time and cost be substantially reduced? Can the quality
and other customer satisfaction criteria for knowledge services and results
be systematically improved?
Can large research programs be made more affordable, timely, and
of higher yield through improved
understanding of "the science of science research program management"?
We believe questions such as these merit further investigation.
Such investigation is likely to be within the self-interests of:
through such a field of inquiry or program initiative.
- the Federal and institutional customers who want scientific
research to be done,
the community of Federal research agencies who administer
the Nation's annual multi-billion investment in scientific research
programs and projects,
the community of researchers, particularly those most fluent in the
relevant disciplines enabling intelligent integration of information
and supporting systems,
We therefore welcome the opportunity to discuss matters such as these,
together with our research experiences outline in this paper,
at the Workshop on R&D Opportunities for Federal Information Services.
is Director of the ATRIUM Laboratory at USC and research
professor in the IOM department in the Marshall School of Business.
He has been on the faculty at USC since 1981,
after completing his Ph.D. in computer science at UC Irvine.
John Noll is a research associate at the ATRIUM Laboratory at USC.
He completed his Ph.D. in computer science at USC in 1996. He has worked
with Dr. Scacchi since 1988, and in the ATRIUM Laboratory since its inception
is President and CEO of New Directions Technologies Inc.,
a company he founded in 1995. Prior to this, Mr. Knight was a Commander in the
Supply Corp, US Navy (retired 1994) and Director of Procurement at the
Naval Air Warfare Center--Weapons Division, at China Lake, CA,
a position he held from 1988-1994.
In addition, he has also directed or commanded other Navy
procurement, logistics, transportation, and supply centers during his
military career. Mr. Knight serves as a sub-contractor on this project.
Captain Jay Miller, SC, USN,
is Director of Acquisition for the Office of Naval
Research, a position held since 1994. Prior to this position, Capt. Miller
has directed or commanded various Navy procurement, logistics, and
supply centers. Captain Miller retires from the USN on 1 May 1997.
Preparation of this report was supported by a grant from the
Office of Naval Research (contract number N00014-94-1-0889). No endorsement
implied. None of the material in this report should be construed as
a statement of policy, procedure, or preference
by ONR, the US Navy, or any other US government agency.
Contributors in the ONR Acquisition Directorate participating in this
study include Jim Carbonara, Frank O'Day, Diana Nichols, John Starcher,
and many others.
In addition, former ATRIUM Laboratory researchers, Dr. Peiwei Mi,
now at Andersen Consulting, and Dr. Mark Nissen,
now at Naval Postgraduate School, contributed to the concepts and tools
employed in this investigation.
B.W. Boehm and W. Scacchi,
Simulation and Modeling for Software Acquisition (SAMSA).
Final Report, Center for Software Engineering, USC, Los Angeles, CA, March
R.T. Kouzes, J.D. Meyers, and W.A. Wulf.
Collaboratories -- Doing Science on the Internet, Computer,
29(8):40-48, August, 1996.
P. Mi and W. Scacchi.
A Meta-Model for Formulating Knowledge-Based Models of Software
Decision Support Systems, 17(3):313-330. 1996.
Valuing IT through Virtual Process Measurement.
Proc. 15th. Intern. Conf. Information Systems,
Vancouver, Canada, 309-323. December 1994.
Knowledge-Based Organizational Process Redesign:
Using Process Flow Measures to Transform Procurement,
unpublished Ph.D. Dissertation, IOM Dept., USC School of Business
Administration, Los Angeles, CA, March 1996.
J. Noll and W. Scacchi.
Integrating Heterogeneous Information Repositories: A Distributed Hypertext
Approach, Computer, 24(12):38-45, December 1991.
J. Noll and W. Scacchi.
Supporting Distributed Configuration Management in Virtual Enterprises.
Proc. 7th. Software Configuration Management Workshop,
R. Conradi (ed.), Boston, MA, Lecture Notes in Computer Science, Volume TBD,
Springer-Verlag, New York.(to appear, 1997).
J. Noll and W. Scacchi.
Supporting Software Development Projects in Virtual Enterprises.
(submitted for publication, January 1997)
W. Scacchi and P. Mi.
Process Life Cycle Engineering: A Knowledge-Based Approach and Environment.
Intern. J. Intelligent Systems in Accounting, Finance, and
Management, (to appear, 1997).
Examples of ONR Grants Management process redesign alternatives
|Diagnosis ||Applicable Hueristics ||Expected ROI|
|Manual step sequence||Consolidate and automate||Med-High|
|Linear step sequences||Identify parallelization opportunities||High|
|Many reviews steps||Joint collaborative reviews||High|
|Many data validation steps||Rule-based review system||Med-High|
|Many data validation steps||Push validation responsiblities upstream||Med-High|
|Manual assembly of compound documents||Rule-based document builder||Low-Med|
|Duplicating and circulating documents||Automate distribution and archiving||Med-Very High|
|Replace paper documents||Employ electronic proposals and grant documents||High-Very High|
|Islands of automation||Intranet with process support, data integration, and product navigation||Low-High|
|Wide-area workflow||Internet-based process enactment||Med-High|
Process diagnosis results from a set of analysis routines and procedures
that we have developed for "measuring" and classifying process flowgraphs
or sub-graph patterns [Ni94,Ni96].
Applicable hueristics are selected from a growing
base of experience
and published studies of successful process redesign tactics.
Expected ROI (return on investment) represents the anticipated payoff
determined from qualitative or intuitive assessments by ONR personnel and
the research team, under the assumption that in-house ONR staff or
external contractors could be engaged and funded to implement
the necessary redesigns, albeit in a cost-effective and timely manner.
Thus, these are simply subjective judgements of the participants.
All alternatives, except the rule-based document builder, are presently
being investigated or implemented. The outcome of some of these
redesigns, such a process step consolidation, automation, and
parallelization, have led to, for example, a collapse
of 31 process steps in ONR Grant Administration into 1 step,
and the compression of 24 steps for Grant Award into 5 steps
(or 3 steps, when invoking Federal Reinvention Laboratory waivers
on FAR-induced paperwork requirements
applicable to ONR). Similarly, the employment of
electronic grant proposals elminates a multi-million operational cost
associated with the processes that handle conventional paper-based
research proposals, but in the context of the other process alternatives.
This document was last updated on Wednesday 16 April 1997 at 12:00pm.