Beavercreek, Ohio – Parallax Advanced Research is conducting several future-focused projects through its Independent Research and Development (IRAD) department, two of which are "Evolving Group Goal Networks from Observations on the Ground" (EGGNOG) and "Fast Recovery via Intrinsic Goals Applied to Exploration" (FRIGATE).
IRAD is an allowable cost under an organization's overhead funding, is initiated and conducted without a direct Department of Defense (DoD) budget, and is independent of DoD oversight. IRAD includes basic research, applied research, development, and systems and other concept formulation studies that aren't grants or contracts from the U.S. Government or third parties.
Parallax's IRAD program is a manifestation of the organization's vision for advancing the Science of Intelligent Teaming™, defined as basic and applied research of highly diverse machine and human teams and how they interact and perform with one another. Criteria for a project to be accepted by the IRAD program include how project outcomes benefit federal research centers; whether the research project supports U.S. Government research requirements; whether the project aligns with the emerging mission and research focus areas of U.S. Government funding and collaborating organization(s); and whether the project provides a clear description of the overall team and assets, its follow-on funding potential, and its overall budget and schedule.
One of IRAD's key advantages is that the research organization owns the IP generated through the process.
“All large research organizations have robust IRAD programs. What’s rarer is when smaller organizations like Parallax have an IRAD program because, although it is an allowable cost, it still affects our rates and competitiveness,” said Viktoria Greanya, chief scientist at Parallax. “As a result, small research organizations usually don’t allocate funding for IRAD projects. However, IRAD is vital to an organization’s growth because that’s where the research staff gets to explore early-stage topics and areas with significant potential to receive U.S. Government and/or DoD funding.”
EGGNOG
At the heart of the EGGNOG project, led by Parallax scientist Dr. Van Parunak, is a social stimulator called Social Causality with Agents using Multiple Perspectives (SCAMP). SCAMP is a psychologically and socially realistic simulator that can model how people make decisions. Parallax scientists constructed SCAMP under a previous Defense Advanced Research Projects Agency (DARPA) program called "Ground Truth." The EGGNOG project aims to demonstrate that SCAMP's Hierarchical Goal Networks (HGN) can evolve to match observed behavior.
To achieve the project's central objective, researchers try to apply genetic search to grow HGNs that generate observed behavior in a simulation. The expected main result is the ability to fit SCAMP models to observed behavior.
“Hierarchical goal network says how important an event is to achieve the goals of each of the groups of agents,” said Dr. Parunak. “Having different hierarchical goal networks affects agent behavior. The question we posed in EGGNOG was: what would the goal network look like that would generate a given path, since that plays into the agent’s behavior?”
Products of the EGGNOG project have a diverse set of potential customers, including the Defense Advanced Research Projects Agency (DARPA), the National Weather Service for finding a method to modulate response to weather models, and advertisers seeking to predict behavior as the result of decisions and actions.
“I'm very excited that Parallax sees the potential of this technology for solving real world problems. Currently, we're focused on the U.S. Defense, but as we go after other domains, I'd be excited to apply it there. In fact, we have written proposals to apply it to problems in health care, for example, infectious diseases,” said Dr. Parunak.
The research results were presented at the annual meeting of the Computational Social Science Society of the Americas in October 2022.
FRIGATE
The FRIGATE project is led by Parallax scientists Dr. Dustin Dannenhauer, Dr. Matthew Molineaux, and Dr. Mary Frame. The primary objective of FRIGATE is to develop new artificial intelligence exploration learning capabilities, which allow agents to guide their learning.
“We are looking at scenarios such as one where the agent might be on a vehicle or in another environment and it's deciding in real time what it should try to learn,” said Dr. Dannenhauer. “We've come up with a system that generates these learning goals that the agent will try to pursue. In case it fails to achieve these goals, the agent will still be learning valuable negative examples that will help it perform better and then achieve its mission.”
Applications of this technology include the recovery of vehicles if an unexpected event happens due to weather patterns, crossing enemy lines, and other complex situations in the real world.
“Right now, a lot of assets are lost due to unexpected events. With FRIGATE, if the system detects that something strange is going on in an unknown situation, it could turn on this learning capability and may be able to recover it when it otherwise couldn't,” said Dr. Dannenhauer. “There are also risks involved because it is uncertain where the system might land while learning in an adverse situation. However, it is better than having no means of recovery in case an asset is lost.”
Research for FRIGATE is like intrinsic reinforcement learning, which involves mechanisms that induce curious agents to learn unspecified learning goals by themselves spontaneously. A general problem with reinforcement learning is how to conduct better, more efficient exploration. The FRIGATE system explores faster and visits more states in the environment than other systems. Over the same period, it will learn better models too.
“As of now, we've only experimented with baseline systems. When we conduct longer experiments against intrinsic reinforcement learning, the FRIGATE system will learn more accurate models about its actions and will be able to know when it can take an action and what the effect of an action will be. We are trying to find out whether we can beat some of the intrinsic reinforced learning approaches that are state-of-the-art right now,” said Dr. Dannenhauer.
These are early-stage results, and the FRIGATE research team is planning more extended experiments to understand the technology's capability better.
Read more about the technology and research utilized in FRIGATE in this published research paper, titled "Self-directed Learning of Action Models using Exploratory Planning."
Parallax's IRAD program is constantly evolving. The organization's leadership is planning activities and events to support ideation generation, such as voice-of-the-customer presentations in different technology spaces related to current research at Parallax or high-priority areas such as space applications.
###
About Parallax Advanced Research
Parallax is a 501(c)(3) nonprofit that tackles global challenges by accelerating innovation and developing technology and solutions through strategic partnerships with government, industry, and academia across Ohio and the Nation. Together with academia, Parallax accelerates innovation that leads to breakthroughs. Together with the government, Parallax tackles critical global challenges and delivers new solutions. Together with the industry, Parallax develops groundbreaking ideas and speeds them to market.