Source Alignment Forum · Published · Open the original ↗
Lab notes
Desiderata for the Theory of Agents
TLDR: I give informal conceptual desiderata for the theory of agents we need to solve ASI alignment, and show a few examples of how existing theories fall short of these desiderata. A Theory of What? The word "agent" is often used in the AI safety discourse and in other research communities, but the intended meaning and scope of this concept varies substantially. For the purpose of this sequence, I informally define an agent to be a system with inputs ("observations") and outputs ("actions") that is effective at steering its environment…
Alignment Forum publishes this post on its own site. Supence shows only what the publisher allows to be quoted.