AI Case Study
Electronic Arts is developing an AI agent which can test for quality assurance during video game production by learning to play against humans
EA is developing AI to test video games for quality assurance during game development. The current proof-of-concept research has resulted in an AI that can play a first-person shooter video game against humans with a measure of success.
Consumer Goods And Services
Entertainment And Sports
From Ground AI: "Video game AI algorithms typically rely on having access to hidden world-state information that is very different to the rendered image that a human player observes. As a result, agents controlled using such algorithms often act in a manner that breaks player immersion, due to their inhuman like behavior. Further, due to the rigid nature of such scripted algorithms the range of behaviours that such AIs can display is also often quite limited."
According to EA: "The agent is pretty proficient at the basic Battlefield gameplay, and has taught itself to alter its behavior depending on certain triggers, like being low on ammo or health. At the moment, the agents aren’t very good at planning ahead. So to help the self-learning agent get a head start with basic action combinations, we let it observe 30 minutes of human play—a process called imitation learning—before letting it train on its own. The agents that we show in our demo have subsequently practiced for six days against versions of itself and some simple old-fashioned bots, playing on several machines in parallel. In total that equates to roughly 300 days of total gameplay experience. Our short-term objective with this project has been to help the DICE team scale up its quality assurance and testing, which would help the studio to collect more crash reports and find more bugs."
EA states that they "have conducted playtests, pitting AI agents against human players in a simplified game mode, restricted to handguns. While the human players outperformed the agents, it wasn’t a complete blowout by any stretch."
As per the arXiv paper, EA calls their model Multi-Action per time step Imitation Learning: "an A3C derivative algorithm that combines supervised imitation learning (learning via guidance from an expert teacher) with temporal difference RL (learning via trial and error), throughout training; using only a small amount of expert data. Imitation learning is used as an auxiliary task in order to help the agent achieve it’s primary goal, playing the game. We describe a neural network architecture that outputs multiple discrete actions per time step without having to model combinations of actions as separate actions, and describe a loss function that allows the policy to be trained. Combining multiaction per time step RL with imitation learning in this manner allows higher quality expert data to be used, as it circumvents the difficulties associated with recording expert data when the expert is limited to single action per time step interactions with the environment."
As reported by The Verge, "EA announced that it’s been training AI agents in 2016’s WWI shooter Battlefield 1... it says the methods it’s developing will help improve future games: providing tougher, more realistic enemies for human players and giving developers new ways to debug their software."
From the arXiv paper, training sets contain "approximately 50% on-policy live agent data and 50% expert data. Expert data was generated prior to training by recording episodes of human play. At each time-step, the following information was stored in a memory buffer: input observation, expert action vector, reward, terminal state and game features vector. The game features vector contained the agents health and ammo to simulate the on-screen text that a human player can read. At each time step, the agent observes a 128x128 pixel RGB image. An in-house developed 3D FPS video game was used as the training environment."