U.S.-Australia Project on AI, Human-Machine Teaming, and the Future of Intelligence Analysis
Hello, I’m Ylli Bajraktari, CEO of the Special Competitive Studies Project. Our weekly newsletter aims to further SCSP’s mission to make recommendations to strengthen America’s long-term competitiveness as AI and other emerging technologies are reshaping our national security, economy, and society with thought-provoking publications. In this edition of our newsletter, SCSP’s Nandita Balakrishnan, Katherine Kurata, and William Usher discuss how integrating Human-Machine Teaming would be a critical advancement for the Intelligence Community (IC) because it would significantly speed up its ability to deliver vital insights to policymakers.
U.S.-Australia Project on AI, Human-Machine Teaming, and the Future of Intelligence Analysis
Today, we published The Future of Intelligence Analysis: AI and Human-Machine Teaming, a report that reflects the work that the Special Competitive Studies Project (SCSP) Intelligence Panel has conducted over the past year and a half in collaboration with the Australian Strategic Policy Institute (ASPI).
Rapid advances in the development of artificial intelligence (AI) technologies over the last few years, particularly the deployment of Generative AI (GenAI) chatbots powered by large language models (LLMs), have demonstrated the potential for AI to revolutionize how the U.S. and Australian intelligence communities (ICs) conduct all-source analysis. Intelligence analysts undertake the critical work of synthesizing the vast array of publicly-available and classified information to derive national security-relevant insights about adversary capabilities, plans, and intentions which they do to support policymakers and warfighters. Canberra and Washington depend on IC analysts to quickly and accurately “connect the dots” to provide timely strategic warning and to spot hidden opportunities for decision makers to influence events in a positive direction.
As capable as our IC analysts are at this mission, the explosion of digitally-available information is outstripping analysts’ ability to sort and make sense of it all. At the same time, the IC has lagged the private sector in adopting AI and other advanced software tools at the scale needed to handle ever-growing volumes of information. To meet this challenge, we argue that the ICs should aim to build high-performance Human-Machine Teams (HMTs) that enable analysts to leverage the computational and growing reasoning capabilities of AI while retaining overall human oversight over analytic assessments and outputs. While we do not argue that AI systems should fully replace the subject matter expertise of the human analyst, we assess that more rapid adoption of LLMs would significantly enhance ICs’ ability to deliver critical insights at speed, and across a wider array of topics than traditional intelligence sources are designed to penetrate. At the moment, the IC is closely tracking AI developments and even deploying LLMs through pilot programs for limited analytic functions, but their use is not nearly widespread nor comprehensive enough to meet future needs.
The IC needs to act quickly because HMTs are not simply a “nice to have” for intelligence work. We can be sure that our adversaries are seeking to use any technology they can against us. The IC must not only keep pace but think about how to flexibly adapt to the evolution of these GenAI tools. Building such teams will require not only investments in AI tools and infrastructure, but also thoughtful planning about how the IC will manage analytic HMTs and hold them accountable for their analytic work. To make this argument, we focus on three key aspects of the integration of HMTs into intelligence analysis in this report:
First, unpacking the potential of GenAI to improve efficiency and effectiveness throughout the analytic workflow. To do so, it is important to first establish the current state-of-play for LLMs, including understanding their limitations, and how they can be leveraged at different stages of intelligence analysis.
Second, assessing the expected trajectory of GenAI advancements over the short-, medium-, and long-term such that analytic leadership in the ICs can make reasoned bets about where they should make critical investments to keep a persistent competitive advantage.
Third, in order to prepare for this future, we make five key recommendations to analytic leadership on where specifically they should target resources and effort in order to build a comprehensive strategy for the deployment of AI tools.
1. Generative AI’s Potential for Bolstering the Analytic Mission
The analytic workflow is a cyclical process where new information is synthesized and integrated into analytic products for customers, who in turn provide feedback that guides what new information and insights are required. LLMs can impact this workflow by creating novel responses or generating scenarios to user questions, proposing new products, consolidating copious amounts of data, and even helping analysts write, coordinate, or review a product.
We recognize that integration of these tools are not without challenges. First, because the IC must maintain a very high bar for the quality and accuracy of the assessments it produces, it traditionally has had a low tolerance for new tools, especially those for which legal and ethical guidelines are nascent. Second, these tools are only useful for the IC if they truly provide added value beyond the capabilities of a human analyst with no loss of subject matter expertise. Third, these tools must meet high standards for transparency, explainability, and accountability. And fourth, deployment of these tools must be done with some level of coordination between friendly services to ensure that future collaboration is robust.
Therefore, existing LLMs will need to be adjusted in order to fulfill these objectives and to be useful to the IC. For example, LLMs even just trained on open-source intelligence would provide these benefits, but they could be enhanced if trained on classified data as well. Similarly, the IC will need to fine-tune the weights of the model and the prompts to stay consistent with analytic standards. Most crucially, the IC will need to invest in tools that will help LLMs meet explainability requirements because in analytic products, analysts justify their reasoning through the use of evidence, and LLM-generated insights will have to meet the same standards.
2. The Coming Wave of AI Advancements
Even over the last few years, we have witnessed the exponential growth of GenAI. In less than two years, OpenAI has released five versions of ChatGPT, each building on the previous iteration’s sophistication. The IC, therefore, cannot only look only to the current technological state-of-play but must also anticipate GenAI’s future trajectory over the course of the 5-, 10-, or 20-years. The swift but uncertain evolution of AI technologies means that ICs need to remain progressive but adaptive. It must balance quickly and safely deploying these tools while also clearly ensuring the proper integration of the expertise and skills of human analysts.
By keeping abreast of likely developments, the IC can assess where and how to target investment of time and resources, including to research and partnerships. For example, over time, LLMs and token window size will become larger, which will allow analysts to make more sophisticated queries and reference a wider selection of information, including classified information. Analysts can also leverage improvements in grounded search and feedback mechanisms to more efficiently dispose of irrelevant information and avoid mischaracterization. Finally, developments in more sophisticated systems like compound systems and agentic systems (defined by the ability to achieve objectives in complex environments without explicit instruction through the execution of actions) will help analysts conduct more complex analyses with limited instruction.
3. Looking to the Future
The future of GenAI holds great opportunities, but the real question is how do the ICs actually ready themselves to leverage these opportunities? Above all, it is important that the IC eschew decision paralysis given the rapidly changing technological terrain and instead accept a baseline level of risk. For analytic managers, pressing their home agencies to invest in AI-related infrastructure offers a great starting point but they must be forward-leaning and act with a sense of urgency or else they will perpetually remain behind the curve. To push integration and deployment forward we recommend five actions:
Design for Continuous AI Model Improvements. The ICs must adapt their technology acquisition processes – and budgets – to enable for multiple annual updates. This will involve experimentation on all the fronts of AI development including larger LLMs, expansions in context lengths, and maturation of more sophisticated systems. However, over a longer time horizon, investments in strategic, systemic data and technology management to support interoperable AI technologies within and across ICs will be essential.
Start Automating Portions of the Analytic Workflow Now. Analytic leadership must move quickly to deploy AI tools that are currently accessible. The overall focus should be to shrink the amount of time required to deliver insight to policymakers while maintaining stringent standards for quality, accuracy, and analytic tradecraft. We assess that the quickest benefit will come from focusing initial attention on areas that have the heaviest amount of human redundancy, such as the analytic review process.
Build Human-Machine Analytic Teams. IC leaders should stand up analytic teams that purposefully blend the relative strengths of humans and machines. In order to build trust, the ICs should conduct careful study into analysts’ needs and mission priorities, design legal, policy, and governance frameworks to ensure robust human oversight, and as necessary, create new tradecraft standards.
Create AI-Ready Training and Incentive Structures for the Analytic Workforce. To effectively integrate these systems will require a workforce that is prepared, motivated, and adept at exploiting these tools to their fullest potential. The ICs will need to invest in digital acumen, both through the recruitment of highly-trained talent and the upskilling of the existing workforce as well as into incentives and rewards for successful usage. These investments are necessary not only for an effective, rapid deployment of AI tools but to make sure that the workforce is prepared to adapt as the tools evolve.
Collaborate to Develop a Shared U.S.-Australian Analytic AI Roadmap. IC-IC collaboration will also be imperative for the deployment of these tools and will be paramount to an integration roadmap that will bolster intra- and inter-agency operability. Potential areas for cooperation include articulating ethical and analytic standards for the use of AI systems, exchanging findings from AI testing and evaluation programs, sharing best practices in the management of human-machine teams, and piloting the use of AI to tackle discrete intelligence analysis problems on a shared high-side data cloud.
Are you a member of the U.S. government who consumes intelligence but is not a member of the Intelligence Community? SCSP is currently seeking responses for its 2nd annual IC Customer Survey. This exciting research builds on SCSP’s previous IC survey work and will provide valuable insights into how intelligence practices and products need to adapt to a rapidly changing global threat landscape. If you are interested in contributing to this research, click here to access the survey.