(Re)Engineering Research Grants Management:
From Acquisition Reform to Knowledge Brokering at ONR

Walt Scacchi1, John Noll1, Cedric Knight2, and Capt. Felton "Jay" Miller3
1ATRIUM Laboratory, University of Southern California, Los Angeles, CA.
2New Directions Technologies Inc., Ridgecrest, CA.
3Acquisition Directorate, Office of Naval Research, Arlington, VA


Email:
Scacchi@gilligan.usc.edu 






Copyright © 1997 Walt Scacchi, The University of 
Southern California. 









Abstract

In this paper, we briefly describe our approach and experience in a research
effort focused on (re)engineering the activity of research grants
management at the Office of Naval Research. We
found that we could contribute to a substantial reduction in process cycle
time and operational costs associated with the funding of thousands
of research grant procurement actions. Accordingly,
we focus our discussion on topics that underlie these results.
We also observe that knowledge brokering is an area 
where a new R&D initiative could lead to more effective and efficient 
research funding and research program management, as well as serve
the mutual self-interests of the Federal research funding agency 
and researcher communities.





Keywords:
 Intelligent information integration, electronic transactions
and electronic commerce technologies, knowledge-based process engineering,
process-driven intranets for research management.


Introduction



We have been involved in a multi-year research project  investigating
ways to reinvent and reengineer corporate financial operations in
military procurement and acquisition organizations. Most recently, this
effort has been directed at the management of research grants by
the Office of Naval Research using electronic commerce and knowledge-based
process engineering technologies. 


Through our research effort, we have found that
dramatic reduction in the cycle time and operational costs of research
grants management processes can be achieved. Internal ONR performance
measures now reveal that a factor of 10 in reduction of
process cycle time has been
realized in procurement action lead time (PALT). PALTs, used
for cross-industry benchmarking purposes, indicate the elapsed time at ONR
from a research grant funding authorization to the time when the
grant recipient can begin to expend their funding award. As more than 5000 
research grant funding actions are performed per year at ONR (1995-1997), 
a reduction
of average PALT from 70 days (in 1994) to 7 days (in 1997, with further
reductions possible) represents a significant demonstration of acquisition
reform at ONR. In addition, some of the information processing and
workflow redesign that we collectively developed with ONR personnel
has led to the identification of annual operational savings estimated in the
range of $10M-$15M, and elimination of these expenses henceforth. 


How did we achieve these results? What research methods
and prototype tools did we employ to achieve these results? Can similar
order of magnitude improvements be made in managing and "brokering" 
scientific research programs by ONR program managers and scientific 
officiers (i.e., the people who solicit and review research proposals, 
and collectively recommend funding actions)? 


This paper seeks to briefly explain what we did in our research
to achieve our results,  and our thoughts for the extension
of this line of research to other federal research grants agencies.
Additional details beyond the scope of this paper can be found in 
presentation
materials that have been posted on the WWW, or as listed in this paper's
References.
Furthermore, we also identify an emerging opportunity for
further research, namely,
how this effort can be expanded to address
knowledge brokering processes at ONR and elsewhere.
But first, we want to specifically address topics that are relevant
to the Workshop on R&D Opportunities in Federal Information Services.



Knowledge-Based Process Architecture
 Research projects at the USC
ATRIUM Laboratory
have  primarily focused on the knowledge-based engineering of complex
organizational processes. Since 1990, these efforts have investigated
the development of tools, techniques, and concepts for engineering
organizational process architectures in domains such as
large-scale 
software engineering, new product development, supply
chain logistics, corporate financial operations, 
software acquisition, and

military procurement. These process architectures
are knowledge representations that model the processes, products, 
organizational roles and team composition, information infrastructure,
and development tools central to an organization in its "routine"
work operations. A descriptive characterization of the knowledge ontology
that we employ that allows us to rapidly transition our efforts across
different organizational domains can be found elsewhere 
[MS96]. Similarly, the knowledge-based tools and
techniques we employ in engineering these process architectures
across the process life cycle is also described elsewhere
[SM96]. Accordingly, in this study, our focus was directed
at four major processes of ONR's research grants management activity:
Grants Pre-Award (proposal solicitation), Award (funding decision and
obligation), Administration (funds disbursement and field office operations), 
and Close-out (completion and reporting compliance).



Application Domain for Multiple Federal Agencies

In 1994, ONR awarded a research grant to the ATRIUM Laboratory to
investigate the development of alternative process architectures
for 
military procurement at the Naval Air Warfare Center, Weapons Division,
in China Lake, CA. This effort was later extended to investigate
ONR research grants management processes, principally those involved
in ONR's Acquisition Directorate. In both situations, the ATRIUM
Laboratory was configured to operate as a kind of process
reinvention collaboratory [cf. KMW96] 
where personnel from NAWC/ONR could
meet off-/on-site with USC researchers, supported with process elicitation,
visualization, integration, and execution support tools that could
be accessed over the Internet 
[NS91,SM96,NS97b].While our efforts
were of modest scale, we note that NAWC-WD is among the largest of the 
US Navy's 1000+ procurement centers, and ONR is the largest of DoD's
research grant agencies (e.g., Darpa, AFOSR, ARO) in terms of
grant actions, as well as one the Federal government's largest research
grants agencies (NSF, NASA, DOE, etc.). Thus, results we might acheive
through our research efforts may be applicable to a large number of government
information service centers.



Effort Needed to Obtain Useful Research Results
 
The application domain of procurement and acquisition is not 
"rocket science." Instead, it is usually considered a back office
business activity concerned with corporate financial operations,
expenditure management, and status reporting. It is an  activity
that is governed by a large number of changing acquisition regulations
and policies at  Federal, DoD, and Navy levels. Federal Acquisition
Regulations (FARs) apply to all government agencies, including those
involved in funding research grants. 
In many ways, the processes and artifacts used to
manage procurement contracts are quite similar to those used to manage
research grants. Similarly, the personnel at ONR who administer grant awards
from ONR's five field offices in the US, are also the same people who manage
contract fulfillment obligations as well as FAR compliance and 
reporting requirements from research institutions. 
Thus, the domain of procurement and acquisition of research
grants and service contracts for Federal agencies is likely to be highly
tractable and sufficiently structured to enable successful
domain knowledge engineering. Furthermore, what we learn about ONR's
research grants management process architecture may be refined and tuned
for application in other DoD or Federal research grants agencies with
affordable effort.



Addressing Barriers to Resistance

Procurement and Acquisition Divisions are populated with personnel that
are usually not specialists with advanced information technology.
These people are not computer scientists, nor can they be expected
to be familiar with knowledge-based tools, techniques, or concepts
know being employed within the research community. Furthermore,
they may often be expected to "resist" the intervention by "outsiders"
whose purpose may be perceived as eliminating their jobs or administrative
authority. Nonetheless, ONR like other government agencies, is under
substantial pressure to accept increasing workloads with shrinking budgets.



Our approach to understanding the process architecture of research grants
management activities at ONR was based on involvement, participation,
and (intellectual) engagement of personnel from the top to the bottom
of the organization chart. We needed to educate ONR personnel
in our motives and methods, and they needed to educate us on the
generic and circumstantial variants of their work processes and information
flow. Two or three iterations were typically performed, particularly
with key domain experts. Follow-up validations by other personnel 
not necessarily involved in these iterations were also performed. 
Furthermore, agreements were established early
on between the research team and ONR personnel covering the following items:



As a result of these efforts and agreements, we found little or no
resistance, since our efforts were defined and structured as inherently
collaborative in purpose, method, and outcome.



Key Enablers and Support Technologies

Our method and agreement for research engagement as  noted was
a key enabler for achieving the results we did. Similarly, the 
knowledge-based process
engineering tools, techniques, and concepts that we have been developing and 
experimenting with at USC for the past seven of so years, were a key enabler.
This of course should be no surprise--we built them, we use them, we evolve
them to meet our emerging research needs. We have an investment in making
them a key enabler, as well as serving to differentiate our effort 
when competing for external research funding.
Nonetheless, we cannot be satisfied on this basis alone. Instead, we
must find ways to make our research technologies accessible as prototypes to
external customers and users, such as personnel at ONR Headquarters
and at its five field offices across the US. As such, part of our research
effort has been directed at prototyping an Internet-based
information infrastructure that could be used to capture, analyze, convey,
prototype, demonstrate, and refine alternative architectures for
organizational processes, such as ONR's research grants management activity
[NS97b,NS97a].
Such an infrastructure must eventually be able to support activities
associated with:





In turn, such a prototype can serve as testbed or process-driven
intranet for ONR research management. Accordingly, we can employ this testbed 
to demonstrate delivery of a research project's inputs and outputs,
in a form that can be accessed, engaged, and served across ONR's 
multiple sites. Otherwise, those familiar with the Electronic Research
Administration initiative
(NewERA)
in which five Federal research agencies (ONR, NIH, DOE, AFOSR, DOT)
currently participate may observe that such an infrastructure
addresses a spectrum of NewERA concerns. However, in our research
project, we have the constraint (or "luxury") of limiting our
investigation to a single research agency (ONR), together with
(a) the ability to identify and prototype alternative process architectures
for research grants management, and (b) access to an (in-progress) integrated
information infrastructure that can directly support activities such
as those just noted.




Are the Research Results Rapidly Deployable and Demonstrable?

The relative ease with which our research results can be deployed outside
of ONR is a matter of opinion. Nonetheless, we can demonstrate and
provide WWW-based presentations on what we have done, how it
was accomplished, and how it might be applied, reproduced, or reused
in other Federal research grants agencies. To this end, we have developed
an exploratory scenario for how external government agencies that fund research
grants through ONR (e.g., Darpa, NASA) might operate. 
Specifically, a recurring problem of internal and external research
programs is tracking the status of
expenditure management: how much unallocated
research funding is available at present for funding obligations, 
and how much is being or has been actually spent (encumbered or expended).
For example, if PALTs require process completion times measured in months
or weeks, then uncertainty, misunderstanding, and
organizational inefficiency can occur relative to the status of
funds availability. Reducing PALTs
to days (or even to hours!) can reduce some of these dilemmas.
Subsequently, 
being able to address practical problems such as expenditure management,
which turn out to be of great importance in 
research funding and reallocation decision-making (i.e., "strategic"
decision-making situations by research program officiers and division
managers), may likely determine the eventual success in being able to
apply research efforts such as ours in other settings. Such a scenario
can be described in more detail at the Workshop, if the opportunity arises.




Related Areas for Research Attention: Knowledge Brokering

Most of what has been described so far focuses on acquisition and
research grants management activities, and how a research effort
such as ours can lead to significant reductions in process cycle time
and operational cost. However, there remains perhaps an intimately
related area for further investigation that we are seek
to address. This concerns the activities involved in the establishment,
management, and fulfillment of research programs by Federal agencies.


At ONR, the source of problems to be addressed by a
research program is often Commands within the USN Fleet. In turn,
Commands within the Fleet are also the source of the budget authority
providing the funds to be expended in acquiring research results
through independent investigations. ONR program officiers (also called
science officiers), program managers, and division directors 
must increasingly organize and manage knowledge brokering processes.


Knowledge brokering refers to the activities of an organizational
agency whose brokers (program officiers) find "servers" (researchers) who can 
"marshall, integrate, and deliver" services and results
(create new knowledge, conduct experiments, prototype technology, produce
research reports,  etc.)
for "clients" (the Fleet). The scope of activity that knowledge brokers
at ONR must articulate is growing. Many program officiers must now
organize R&D programs that span from basic research studies, through
applied research and prototyping studies, to advanced technology
demonstrations that address customer needs. These processes presently
take years, a decade, or more to complete. 


Whether focused only on ONR,
or more broadly at any/all Federal reseearch agencies, a number of
basic questions can be asked about the processes, architectures,
artifacts, support systems, etc, that support Federal knowledge
brokering activities: What are these processes, architectures, etc.?
How do they work, how do they work best, and how do they go wrong? 
Can they be computationally modeled, analyzed, simulated, and so forth
across their life cycle? How might Federal knowledge
brokering process architectures  be redesigned for optimal and adaptive
performance? Can their
cycle time and cost be substantially reduced? Can the quality
and other customer satisfaction criteria for knowledge services and results
be systematically improved?
Can large research programs be made more affordable, timely, and
of higher yield through improved
understanding of "the science of science research program management"?


We believe questions such as these merit further investigation.
Such investigation is likely to be within the self-interests of:


through such a field of inquiry or program initiative.


We therefore welcome the opportunity to discuss matters such as these,
together with our research experiences outline in this paper,
at the Workshop on R&D Opportunities for Federal Information Services.





Biographies




  • Walt Scacchi 
is Director of the ATRIUM Laboratory at USC and research 
professor in the IOM department in the Marshall School of Business.
He has been on the faculty at USC since 1981,
 after completing his Ph.D. in computer science at UC Irvine. 

  • John Noll is a research associate at the ATRIUM Laboratory at USC.
He completed his Ph.D. in computer science at USC in 1996. He has worked
with Dr. Scacchi since 1988, and in the ATRIUM Laboratory since its inception
in 1993.

  • Cedric Knight 
is President and CEO of New Directions Technologies Inc.,
a company he founded in 1995. Prior to this, Mr. Knight was a Commander in the
Supply Corp, US Navy (retired 1994) and Director of Procurement at the 
Naval Air Warfare Center--Weapons Division, at China Lake, CA, 
a position he held from 1988-1994. 
In addition, he has also directed or commanded other Navy
procurement, logistics, transportation, and supply centers during his 
military career. Mr. Knight serves as a sub-contractor on this project.

  • Captain Jay Miller, SC, USN, 
is Director of Acquisition for the Office of Naval
Research, a position held since 1994. Prior to this position, Capt. Miller
has directed or commanded various Navy procurement, logistics, and 
supply centers. Captain Miller retires from the USN on 1 May 1997.

    

    Acknowledgements
    
Preparation of this report was supported by a grant from the
Office of Naval Research (contract number N00014-94-1-0889). No endorsement
implied. None of the material in this report should be construed as
a statement of policy, procedure, or preference
by ONR, the US Navy, or any other US government agency.
Contributors in the ONR Acquisition Directorate participating in this
study include Jim Carbonara, Frank O'Day, Diana Nichols, John Starcher,
and many others.
In addition, former ATRIUM Laboratory researchers, Dr. Peiwei Mi, 
now at Andersen Consulting, and Dr. Mark Nissen,
now at Naval Postgraduate School, contributed to the concepts and tools
employed in this investigation.
 
    

    References
    

    

    BS96
    
B.W. Boehm and W. Scacchi,
Simulation and Modeling for Software Acquisition (SAMSA).
Final Report, Center for Software Engineering, USC, Los Angeles, CA, March 
1996. 

http://sunset.usc.edu/SAMSA/samcover.html


    KMW96
    
R.T. Kouzes, J.D. Meyers, and W.A. Wulf.
Collaboratories -- Doing Science on the Internet, Computer,
29(8):40-48, August, 1996.


    MS96
    
P. Mi and W. Scacchi.
 A Meta-Model for Formulating Knowledge-Based Models of Software
  Development.
  Decision Support Systems, 17(3):313-330. 1996.

http://www.usc.edu/dept/ATRIUM/Papers/Process_Meta_Model.ps


    Ni94
    
M. Nissen.
Valuing IT through Virtual Process Measurement.
Proc. 15th. Intern. Conf. Information Systems, 
Vancouver, Canada, 309-323. December 1994.

http://www.usc.edu/dept/ATRIUM/Papers/Process_Measurement.ps


    Ni96
    
M. Nissen.
Knowledge-Based Organizational Process Redesign:
Using Process Flow Measures to Transform Procurement,
unpublished Ph.D. Dissertation, IOM Dept., USC School of Business
Administration, Los Angeles, CA, March 1996.


    NS91
    
J. Noll and W. Scacchi.
Integrating Heterogeneous Information Repositories: A Distributed Hypertext
Approach, Computer, 24(12):38-45, December 1991.

http://www.usc.edu/dept/ATRIUM/Papers/Distributed_Hypertext.ps



    NS97a
    
J. Noll and W. Scacchi.
Supporting Distributed Configuration Management in Virtual Enterprises.
Proc. 7th. Software Configuration Management Workshop,
R. Conradi (ed.), Boston, MA, Lecture Notes in Computer Science, Volume TBD, 
Springer-Verlag, New York.(to appear, 1997). 

http://www.usc.edu/dept/ATRIUM/Papers/DHT-SCM7.ps


    NS97b
    
J. Noll and W. Scacchi.
Supporting Software Development Projects in Virtual Enterprises.
(submitted for publication, January 1997)

http://www.usc.edu/dept/ATRIUM/Papers/DHT-VE97.html


    SM96
    
W. Scacchi and P. Mi.
Process Life Cycle Engineering: A Knowledge-Based Approach and Environment.
Intern. J. Intelligent Systems in Accounting, Finance, and 
Management, (to appear, 1997). 

http://www.usc.edu/dept/ATRIUM/Papers/Process_Life_Cycle.html

    


    

    Appendix: 
Examples of ONR Grants Management process redesign alternatives
    

    


    

    Diagnosis Applicable Hueristics Expected ROI
    

    Manual step sequenceConsolidate and automateMed-High
    

    Linear step sequencesIdentify parallelization opportunitiesHigh
    

    Many reviews stepsJoint collaborative reviewsHigh
    

    Many data validation stepsRule-based review systemMed-High
    

    Many data validation stepsPush validation responsiblities upstreamMed-High
    

    Manual assembly of compound documentsRule-based document builderLow-Med
    

    Duplicating and circulating documentsAutomate distribution and archivingMed-Very High
    

    Replace paper documentsEmploy electronic proposals and grant documentsHigh-Very High
    

    Islands of automationIntranet with process support, data integration, and product navigationLow-High
    

    Wide-area workflowInternet-based process enactmentMed-High
    

    

    

    
Process diagnosis results from a set of analysis routines and procedures
that we have developed for "measuring" and classifying process flowgraphs
or sub-graph patterns [Ni94,Ni96].

    Applicable hueristics  are selected from a growing 
base of experience
and published studies of successful process redesign tactics.

    
Expected ROI (return on investment) represents the anticipated payoff 
determined from qualitative or intuitive assessments by ONR personnel and
the research team, under the assumption that in-house ONR staff or
external contractors could be engaged and funded to implement
the necessary redesigns, albeit in a cost-effective and timely manner.
Thus, these are simply subjective judgements of the participants.
All alternatives, except the rule-based document builder, are presently
being investigated or implemented. The outcome of some of these
redesigns, such a process step consolidation, automation, and
parallelization, have led to, for example, a collapse
of 31 process steps in ONR Grant Administration into 1 step,
and the compression of 24 steps for Grant Award into 5 steps
(or 3 steps, when invoking Federal Reinvention Laboratory waivers
on FAR-induced paperwork requirements
applicable to ONR). Similarly, the employment of
electronic grant proposals elminates a multi-million operational cost
associated with the processes that handle conventional paper-based
research proposals, but in the context of the other process alternatives.



    

    
This document was last updated on Wednesday 16 April 1997 at 12:00pm.