Earlier in November, the innovation team spoke with Dustin Dannenhauer about his work on modeling computer games for AI, and how we can apply what we’ve learned from complex games to the real world. Dustin is a research scientist in the autonomy and artificial intelligence (AI) unit at Parallax Advanced Research.
Dustin, who has been at Parallax since 2020, began developing an AI-friendly interface to the video game “Dungeon Crawl Stone Soup,” in 2017. The niche game is one of a class of games called “roguelikes,” after the original Rogue. In them, a player uses a character to traverse a dungeon, collect objects, and navigate their way out of dynamic environments while battling monsters and avoiding what Dustin calls "permanent death."
Dustin explains, “these are simple games in some respects. For instance, their graphics are two-dimensional, and the software is comparatively lightweight. But they’re sophisticated from a decision-making perspective. People will play them for years before they win their first game."
The sophistication of a game like Dungeon Crawl Stone Soup is exactly why Dustin is interested in them from an artificial intelligence perspective. Currently, there is no AI agent that can play the game as well as expert humans. For one, the state-action space is vast. A single character may learn from 100+ different spells and encounter as many as 600+ unique monsters. Further, the environment is always changing, requiring spontaneous decisions and, most importantly, the rewards for an AI agent to learn the appropriate responses to a given situation are sparse.
“Reinforcement learning systems perform best when they receive consistent rewards to learn. If they receive a reward after 100 actions, they don’t know which sub-sequence of actions caused the reward. It would take a long time for the agent to learn the right choices," said Dustin.
Dustin’s work is novel not just because he’s trying to solve a sophisticated problem with AI but also because he’s approaching it from an important perspective. Folks familiar with AI probably know about deep learning models, which are used all over the place. These are often referred to as “black box” models that are both complex and prone to learning undesired biases from data, making them difficult for humans to understand how or why an algorithm has come to a particular conclusion.
In contrast, symbolic relational models – one of the models Dustin is developing for Dungeon Crawl Stone Soup – are more interpretable. Since human language is littered with relative and symbolic notions these kinds of models are easier for a human to understand.
“Artificial intelligence is always operating within a human context. Having AI agents that are understandable to humans is hugely important,” said Dustin.
Since beginning this as a side project in 2017, his work has gained traction within the scientific community. In 2019, he published a workshop paper on the promise of Dungeon Crawl Stone Soup as an evaluation domain for artificial intelligence. A year later, researchers from Facebook cited it.
Now, he’s proposing ways to integrate his work with deep learning models to solve real-world problems. As it happens, the Dungeon Crawl Stone Soup problem shares many similarities with military-relevant contexts, making it especially compelling. He’s proposed this project in two Internal Research & Development Program (IRAD) cycles at Parallax to use the game as an evaluation domain for new ideas in AI, and the work is pertinent to a third IRAD submitted in 2021.
In addition to integrating the project into his professional work, Dustin livestreams his AI coding on YouTube, while simultaneously showing the AI agent running the game side-by-side. He has posted code for the agent online, along with accompanying documentation for anyone interested in joining the fun.
Read the latest publication on this project here.
Dustin Dannenhauer works remotely out of Washington D.C. and can be reached by contacting us.
For continued reading on Dustin’s work, visit his website at http://www.dustindannenhauer.com/.
Dannenhauer, D., Dannenhauer, Z. A., Decker, J., Amos-Binks, A., Floyd, M. W., & Aha, D. W. dcss-ai-wrapper: An API for Dungeon Crawl Stone Soup providing both Vector and Symbolic State Representations. Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning. International Conference on Automated Planning and Scheduling (ICAPS). 2021.