(Re)Engineering Research Grants Management:
From Acquisition Reform to Knowledge Brokering at ONR

Walt Scacchi1, John Noll1, Cedric Knight2, and Capt. Felton "Jay" Miller3
1ATRIUM Laboratory, University of Southern California, Los Angeles, CA.
2New Directions Technologies Inc., Ridgecrest, CA.
3Acquisition Directorate, Office of Naval Research, Arlington, VA


Email:
Scacchi@gilligan.usc.edu 

Copyright © 1997 Walt Scacchi, The University of Southern California.


Abstract

In this paper, we briefly describe our approach and experience in a research effort focused on (re)engineering the activity of research grants management at the Office of Naval Research. We found that we could contribute to a substantial reduction in process cycle time and operational costs associated with the funding of thousands of research grant procurement actions. Accordingly, we focus our discussion on topics that underlie these results. We also observe that knowledge brokering is an area where a new R&D initiative could lead to more effective and efficient research funding and research program management, as well as serve the mutual self-interests of the Federal research funding agency and researcher communities.

Keywords:
Intelligent information integration, electronic transactions and electronic commerce technologies, knowledge-based process engineering, process-driven intranets for research management.

Introduction

We have been involved in a multi-year research project investigating ways to reinvent and reengineer corporate financial operations in military procurement and acquisition organizations. Most recently, this effort has been directed at the management of research grants by the Office of Naval Research using electronic commerce and knowledge-based process engineering technologies.

Through our research effort, we have found that dramatic reduction in the cycle time and operational costs of research grants management processes can be achieved. Internal ONR performance measures now reveal that a factor of 10 in reduction of process cycle time has been realized in procurement action lead time (PALT). PALTs, used for cross-industry benchmarking purposes, indicate the elapsed time at ONR from a research grant funding authorization to the time when the grant recipient can begin to expend their funding award. As more than 5000 research grant funding actions are performed per year at ONR (1995-1997), a reduction of average PALT from 70 days (in 1994) to 7 days (in 1997, with further reductions possible) represents a significant demonstration of acquisition reform at ONR. In addition, some of the information processing and workflow redesign that we collectively developed with ONR personnel has led to the identification of annual operational savings estimated in the range of $10M-$15M, and elimination of these expenses henceforth.

How did we achieve these results? What research methods and prototype tools did we employ to achieve these results? Can similar order of magnitude improvements be made in managing and "brokering" scientific research programs by ONR program managers and scientific officiers (i.e., the people who solicit and review research proposals, and collectively recommend funding actions)?

This paper seeks to briefly explain what we did in our research to achieve our results, and our thoughts for the extension of this line of research to other federal research grants agencies. Additional details beyond the scope of this paper can be found in presentation materials that have been posted on the WWW, or as listed in this paper's References. Furthermore, we also identify an emerging opportunity for further research, namely, how this effort can be expanded to address knowledge brokering processes at ONR and elsewhere. But first, we want to specifically address topics that are relevant to the Workshop on R&D Opportunities in Federal Information Services.

Knowledge-Based Process Architecture

Research projects at the USC ATRIUM Laboratory have primarily focused on the knowledge-based engineering of complex organizational processes. Since 1990, these efforts have investigated the development of tools, techniques, and concepts for engineering organizational process architectures in domains such as large-scale software engineering, new product development, supply chain logistics, corporate financial operations, software acquisition, and military procurement. These process architectures are knowledge representations that model the processes, products, organizational roles and team composition, information infrastructure, and development tools central to an organization in its "routine" work operations. A descriptive characterization of the knowledge ontology that we employ that allows us to rapidly transition our efforts across different organizational domains can be found elsewhere [MS96]. Similarly, the knowledge-based tools and techniques we employ in engineering these process architectures across the process life cycle is also described elsewhere [SM96]. Accordingly, in this study, our focus was directed at four major processes of ONR's research grants management activity: Grants Pre-Award (proposal solicitation), Award (funding decision and obligation), Administration (funds disbursement and field office operations), and Close-out (completion and reporting compliance).

Application Domain for Multiple Federal Agencies

In 1994, ONR awarded a research grant to the ATRIUM Laboratory to investigate the development of alternative process architectures for military procurement at the Naval Air Warfare Center, Weapons Division, in China Lake, CA. This effort was later extended to investigate ONR research grants management processes, principally those involved in ONR's Acquisition Directorate. In both situations, the ATRIUM Laboratory was configured to operate as a kind of process reinvention collaboratory [cf. KMW96] where personnel from NAWC/ONR could meet off-/on-site with USC researchers, supported with process elicitation, visualization, integration, and execution support tools that could be accessed over the Internet [NS91,SM96,NS97b].While our efforts were of modest scale, we note that NAWC-WD is among the largest of the US Navy's 1000+ procurement centers, and ONR is the largest of DoD's research grant agencies (e.g., Darpa, AFOSR, ARO) in terms of grant actions, as well as one the Federal government's largest research grants agencies (NSF, NASA, DOE, etc.). Thus, results we might acheive through our research efforts may be applicable to a large number of government information service centers.

Effort Needed to Obtain Useful Research Results

The application domain of procurement and acquisition is not "rocket science." Instead, it is usually considered a back office business activity concerned with corporate financial operations, expenditure management, and status reporting. It is an activity that is governed by a large number of changing acquisition regulations and policies at Federal, DoD, and Navy levels. Federal Acquisition Regulations (FARs) apply to all government agencies, including those involved in funding research grants. In many ways, the processes and artifacts used to manage procurement contracts are quite similar to those used to manage research grants. Similarly, the personnel at ONR who administer grant awards from ONR's five field offices in the US, are also the same people who manage contract fulfillment obligations as well as FAR compliance and reporting requirements from research institutions. Thus, the domain of procurement and acquisition of research grants and service contracts for Federal agencies is likely to be highly tractable and sufficiently structured to enable successful domain knowledge engineering. Furthermore, what we learn about ONR's research grants management process architecture may be refined and tuned for application in other DoD or Federal research grants agencies with affordable effort.

Addressing Barriers to Resistance

Procurement and Acquisition Divisions are populated with personnel that are usually not specialists with advanced information technology. These people are not computer scientists, nor can they be expected to be familiar with knowledge-based tools, techniques, or concepts know being employed within the research community. Furthermore, they may often be expected to "resist" the intervention by "outsiders" whose purpose may be perceived as eliminating their jobs or administrative authority. Nonetheless, ONR like other government agencies, is under substantial pressure to accept increasing workloads with shrinking budgets.

Our approach to understanding the process architecture of research grants management activities at ONR was based on involvement, participation, and (intellectual) engagement of personnel from the top to the bottom of the organization chart. We needed to educate ONR personnel in our motives and methods, and they needed to educate us on the generic and circumstantial variants of their work processes and information flow. Two or three iterations were typically performed, particularly with key domain experts. Follow-up validations by other personnel not necessarily involved in these iterations were also performed. Furthermore, agreements were established early on between the research team and ONR personnel covering the following items:

As a result of these efforts and agreements, we found little or no resistance, since our efforts were defined and structured as inherently collaborative in purpose, method, and outcome.

Key Enablers and Support Technologies

Our method and agreement for research engagement as noted was a key enabler for achieving the results we did. Similarly, the knowledge-based process engineering tools, techniques, and concepts that we have been developing and experimenting with at USC for the past seven of so years, were a key enabler. This of course should be no surprise--we built them, we use them, we evolve them to meet our emerging research needs. We have an investment in making them a key enabler, as well as serving to differentiate our effort when competing for external research funding. Nonetheless, we cannot be satisfied on this basis alone. Instead, we must find ways to make our research technologies accessible as prototypes to external customers and users, such as personnel at ONR Headquarters and at its five field offices across the US. As such, part of our research effort has been directed at prototyping an Internet-based information infrastructure that could be used to capture, analyze, convey, prototype, demonstrate, and refine alternative architectures for organizational processes, such as ONR's research grants management activity [NS97b,NS97a]. Such an infrastructure must eventually be able to support activities associated with: In turn, such a prototype can serve as testbed or process-driven intranet for ONR research management. Accordingly, we can employ this testbed to demonstrate delivery of a research project's inputs and outputs, in a form that can be accessed, engaged, and served across ONR's multiple sites. Otherwise, those familiar with the Electronic Research Administration initiative (NewERA) in which five Federal research agencies (ONR, NIH, DOE, AFOSR, DOT) currently participate may observe that such an infrastructure addresses a spectrum of NewERA concerns. However, in our research project, we have the constraint (or "luxury") of limiting our investigation to a single research agency (ONR), together with (a) the ability to identify and prototype alternative process architectures for research grants management, and (b) access to an (in-progress) integrated information infrastructure that can directly support activities such as those just noted.

Are the Research Results Rapidly Deployable and Demonstrable?

The relative ease with which our research results can be deployed outside of ONR is a matter of opinion. Nonetheless, we can demonstrate and provide WWW-based presentations on what we have done, how it was accomplished, and how it might be applied, reproduced, or reused in other Federal research grants agencies. To this end, we have developed an exploratory scenario for how external government agencies that fund research grants through ONR (e.g., Darpa, NASA) might operate. Specifically, a recurring problem of internal and external research programs is tracking the status of expenditure management: how much unallocated research funding is available at present for funding obligations, and how much is being or has been actually spent (encumbered or expended). For example, if PALTs require process completion times measured in months or weeks, then uncertainty, misunderstanding, and organizational inefficiency can occur relative to the status of funds availability. Reducing PALTs to days (or even to hours!) can reduce some of these dilemmas. Subsequently, being able to address practical problems such as expenditure management, which turn out to be of great importance in research funding and reallocation decision-making (i.e., "strategic" decision-making situations by research program officiers and division managers), may likely determine the eventual success in being able to apply research efforts such as ours in other settings. Such a scenario can be described in more detail at the Workshop, if the opportunity arises.

Related Areas for Research Attention: Knowledge Brokering

Most of what has been described so far focuses on acquisition and research grants management activities, and how a research effort such as ours can lead to significant reductions in process cycle time and operational cost. However, there remains perhaps an intimately related area for further investigation that we are seek to address. This concerns the activities involved in the establishment, management, and fulfillment of research programs by Federal agencies.

At ONR, the source of problems to be addressed by a research program is often Commands within the USN Fleet. In turn, Commands within the Fleet are also the source of the budget authority providing the funds to be expended in acquiring research results through independent investigations. ONR program officiers (also called science officiers), program managers, and division directors must increasingly organize and manage knowledge brokering processes.

Knowledge brokering refers to the activities of an organizational agency whose brokers (program officiers) find "servers" (researchers) who can "marshall, integrate, and deliver" services and results (create new knowledge, conduct experiments, prototype technology, produce research reports, etc.) for "clients" (the Fleet). The scope of activity that knowledge brokers at ONR must articulate is growing. Many program officiers must now organize R&D programs that span from basic research studies, through applied research and prototyping studies, to advanced technology demonstrations that address customer needs. These processes presently take years, a decade, or more to complete.

Whether focused only on ONR, or more broadly at any/all Federal reseearch agencies, a number of basic questions can be asked about the processes, architectures, artifacts, support systems, etc, that support Federal knowledge brokering activities: What are these processes, architectures, etc.? How do they work, how do they work best, and how do they go wrong? Can they be computationally modeled, analyzed, simulated, and so forth across their life cycle? How might Federal knowledge brokering process architectures be redesigned for optimal and adaptive performance? Can their cycle time and cost be substantially reduced? Can the quality and other customer satisfaction criteria for knowledge services and results be systematically improved? Can large research programs be made more affordable, timely, and of higher yield through improved understanding of "the science of science research program management"?

We believe questions such as these merit further investigation. Such investigation is likely to be within the self-interests of:

through such a field of inquiry or program initiative.

We therefore welcome the opportunity to discuss matters such as these, together with our research experiences outline in this paper, at the Workshop on R&D Opportunities for Federal Information Services.


Biographies

  • Walt Scacchi is Director of the ATRIUM Laboratory at USC and research professor in the IOM department in the Marshall School of Business. He has been on the faculty at USC since 1981, after completing his Ph.D. in computer science at UC Irvine.
  • John Noll is a research associate at the ATRIUM Laboratory at USC. He completed his Ph.D. in computer science at USC in 1996. He has worked with Dr. Scacchi since 1988, and in the ATRIUM Laboratory since its inception in 1993.
  • Cedric Knight is President and CEO of New Directions Technologies Inc., a company he founded in 1995. Prior to this, Mr. Knight was a Commander in the Supply Corp, US Navy (retired 1994) and Director of Procurement at the Naval Air Warfare Center--Weapons Division, at China Lake, CA, a position he held from 1988-1994. In addition, he has also directed or commanded other Navy procurement, logistics, transportation, and supply centers during his military career. Mr. Knight serves as a sub-contractor on this project.
  • Captain Jay Miller, SC, USN, is Director of Acquisition for the Office of Naval Research, a position held since 1994. Prior to this position, Capt. Miller has directed or commanded various Navy procurement, logistics, and supply centers. Captain Miller retires from the USN on 1 May 1997.

    Acknowledgements

    Preparation of this report was supported by a grant from the Office of Naval Research (contract number N00014-94-1-0889). No endorsement implied. None of the material in this report should be construed as a statement of policy, procedure, or preference by ONR, the US Navy, or any other US government agency. Contributors in the ONR Acquisition Directorate participating in this study include Jim Carbonara, Frank O'Day, Diana Nichols, John Starcher, and many others. In addition, former ATRIUM Laboratory researchers, Dr. Peiwei Mi, now at Andersen Consulting, and Dr. Mark Nissen, now at Naval Postgraduate School, contributed to the concepts and tools employed in this investigation.

    References

    BS96
    B.W. Boehm and W. Scacchi, Simulation and Modeling for Software Acquisition (SAMSA). Final Report, Center for Software Engineering, USC, Los Angeles, CA, March 1996. http://sunset.usc.edu/SAMSA/samcover.html
    KMW96
    R.T. Kouzes, J.D. Meyers, and W.A. Wulf. Collaboratories -- Doing Science on the Internet, Computer, 29(8):40-48, August, 1996.
    MS96
    P. Mi and W. Scacchi. A Meta-Model for Formulating Knowledge-Based Models of Software Development. Decision Support Systems, 17(3):313-330. 1996. http://www.usc.edu/dept/ATRIUM/Papers/Process_Meta_Model.ps
    Ni94
    M. Nissen. Valuing IT through Virtual Process Measurement. Proc. 15th. Intern. Conf. Information Systems, Vancouver, Canada, 309-323. December 1994. http://www.usc.edu/dept/ATRIUM/Papers/Process_Measurement.ps
    Ni96
    M. Nissen. Knowledge-Based Organizational Process Redesign: Using Process Flow Measures to Transform Procurement, unpublished Ph.D. Dissertation, IOM Dept., USC School of Business Administration, Los Angeles, CA, March 1996.
    NS91
    J. Noll and W. Scacchi. Integrating Heterogeneous Information Repositories: A Distributed Hypertext Approach, Computer, 24(12):38-45, December 1991. http://www.usc.edu/dept/ATRIUM/Papers/Distributed_Hypertext.ps
    NS97a
    J. Noll and W. Scacchi. Supporting Distributed Configuration Management in Virtual Enterprises. Proc. 7th. Software Configuration Management Workshop, R. Conradi (ed.), Boston, MA, Lecture Notes in Computer Science, Volume TBD, Springer-Verlag, New York.(to appear, 1997). http://www.usc.edu/dept/ATRIUM/Papers/DHT-SCM7.ps
    NS97b
    J. Noll and W. Scacchi. Supporting Software Development Projects in Virtual Enterprises. (submitted for publication, January 1997) http://www.usc.edu/dept/ATRIUM/Papers/DHT-VE97.html
    SM96
    W. Scacchi and P. Mi. Process Life Cycle Engineering: A Knowledge-Based Approach and Environment. Intern. J. Intelligent Systems in Accounting, Finance, and Management, (to appear, 1997). http://www.usc.edu/dept/ATRIUM/Papers/Process_Life_Cycle.html

    Appendix: Examples of ONR Grants Management process redesign alternatives

    Diagnosis Applicable Hueristics Expected ROI
    Manual step sequenceConsolidate and automateMed-High
    Linear step sequencesIdentify parallelization opportunitiesHigh
    Many reviews stepsJoint collaborative reviewsHigh
    Many data validation stepsRule-based review systemMed-High
    Many data validation stepsPush validation responsiblities upstreamMed-High
    Manual assembly of compound documentsRule-based document builderLow-Med
    Duplicating and circulating documentsAutomate distribution and archivingMed-Very High
    Replace paper documentsEmploy electronic proposals and grant documentsHigh-Very High
    Islands of automationIntranet with process support, data integration, and product navigationLow-High
    Wide-area workflowInternet-based process enactmentMed-High

    Process diagnosis results from a set of analysis routines and procedures that we have developed for "measuring" and classifying process flowgraphs or sub-graph patterns [Ni94,Ni96].

    Applicable hueristics are selected from a growing base of experience and published studies of successful process redesign tactics.

    Expected ROI (return on investment) represents the anticipated payoff determined from qualitative or intuitive assessments by ONR personnel and the research team, under the assumption that in-house ONR staff or external contractors could be engaged and funded to implement the necessary redesigns, albeit in a cost-effective and timely manner. Thus, these are simply subjective judgements of the participants. All alternatives, except the rule-based document builder, are presently being investigated or implemented. The outcome of some of these redesigns, such a process step consolidation, automation, and parallelization, have led to, for example, a collapse of 31 process steps in ONR Grant Administration into 1 step, and the compression of 24 steps for Grant Award into 5 steps (or 3 steps, when invoking Federal Reinvention Laboratory waivers on FAR-induced paperwork requirements applicable to ONR). Similarly, the employment of electronic grant proposals elminates a multi-million operational cost associated with the processes that handle conventional paper-based research proposals, but in the context of the other process alternatives.


    This document was last updated on Wednesday 16 April 1997 at 12:00pm.