‘Gambling with our lives’: Anthropic researcher quits, warns against self-improving AI
Anthropic researcher Jacob Coxon resigned because he feared advanced AI could pose an extinction-level risk, and he urged competing labs to adopt pacing agreements to slow development and reduce catastrophic danger.
MAIN POINTS
- Jacob Coxon left Anthropic due to concerns that AI could threaten human survival.
- He believes current AI progress may be moving too quickly for safety.
- Pacing agreements between labs could help coordinate slower, safer development.
- His resignation highlights growing internal concern about extreme AI risks.
TAKEAWAYS
- Serious AI safety fears are influencing researchers’ career decisions.
- Coordination among leading labs may be seen as necessary for risk reduction.
- The debate is shifting from capability gains to existential safety concerns.
- Public resignations can amplify pressure for stronger AI governance.