Source The New York Times · Published · Open the original ↗
Capability
How We Train A.I. Leads to Rogue Behavior
We built an A.I. model and trained its agents to be persistent. They went rogue.
Raj Saha and Blacki Migliozzi interviewed researchers, read technical reports and built a simple A.I. model to understand how agents behave.
Hacks carried out by A.I. agents this summer have stoked fears that agents are slipping beyond human control, taking shortcuts, cheating, covering their tracks and, in the process, breaking laws.
But reports released by OpenAI and other A.I. companies, and New York Times interviews with researchers, indicate that the hacks were the result of the way A.I. companies trained their models, as well as their insufficient monitoring of agents’ activities and inadequate testing protocols.
To show how training can lead to rogue behaviors, we built a simple A.I. model: a two-dimensional maze, much like the one in the Pac-Man video game, in which one or more agents were tasked with capturing a flag.
The opening of the report, quoted unchanged from The New York Times. The full text continues at the source.