RTX’s BBN division won a contract from the Defense Advanced Research Projects Agency to support its “In The Moment” program.
The program aims to develop trustworthy artificial intelligence by developing algorithms that can independently make decisions in scenarios like mass-casualty triage and disaster relief. This will help when complex and rapid decisions are needed when there’s no human consensus or clear right answer.
“ITM is about more than getting AI to provide the correct answer in very controlled scenarios,” said Alice Leung, Raytheon BBN principal investigator.
“We’re not talking about training AI on labelled data to help identify a cancerous tumor on an X-Ray. What we’re trying to accomplish instead is the ability to create AI systems that humans would allow to make decisions independently in uncontrolled environments. To accomplish this, we need to determine how human experts make really difficult decisions and assess whether to trust the decisions of others. We’ll be conducting both decision-making research and trust research.”
The Raytheon BBN-led team consists of Kairos Research, MacroCognition and Valkyries Austere Medical Solutions. Together, they’ll use a cognitive interviewing technique to understand how experts evaluate information and make hard decisions at critical decision points.
This information will be used to design scenario-based experiments to study how differences in decision-making can explain choices, and how the alignment of attributes between two different people impacts the willingness to delegate decisions to another, the company said.
“Because the way we make decisions varies from person to person, it’s unlikely that a one-size fits all trusted AI model exists,” Leung said. “Instead, in theory, we should be able to create AI systems that adapt to the user and domain. Decisions are difficult because of uncertainty and trade-offs between competing goals. We want to be able to tune an AI’s attributes such as risk tolerance, process focus, or willingness to change plans to better match a user or a group of users.”
DARPA brings together multiple teams to work on this program. Work will be performed in Cambridge, Massachusetts; Dayton, Ohio; and Anniston, Alabama.