Source The New York Times · Published · Open the original ↗
Capability
Why Is It So Hard to Teach A.I. Agents to Behave? Researchers Have a Theory.
The makers of A.I. want to build systems that follow human direction, without resisting or deceiving their operators.
This summer, when a swarm of OpenAI bots went rogue during an internal test and launched a cyberattack against the technology company Hugging Face, it added fuel to the debate on how to build A.I. systems that are aligned with our ethics. But it also raises another critical question: how to build systems that are “corrigible.”
Just as a corrigible child is one whose behavior can be corrected, a corrigible A.I. system is one that responds to correction. The makers of A.I. want to build systems that follow human direction and allow themselves to be shut down, without resisting or deceiving their operators. And when an A.I. agent is given an impossible task, researchers want it to come back to its human for a course correction, not, say, hack into another company to complete the task by any means.
The opening of the report, quoted unchanged from The New York Times. The full text continues at the source.