The laboratory must accept correction too.
These are the governing requirements we propose to implement. Their legal form, financing, appointments, and enforcement must be settled before consequential operation.
Human agency is the end.
Assistance should strengthen people’s ability to understand, choose, act, correct, dissent, and leave. No single customer, founder, or aggregate score can silently erase the interests of other affected people. Protected rights and procedures for contested authority must be explicit.
Independence must have actual powers.
Review sits outside the line responsible for capability delivery and release targets. It needs protected appointments, its own evidence access and resources, and effective authority to suspend an activity. The body that appoints reviewers must itself have a disclosed, challengeable succession process.
A stop survives a change of leadership.
Weakening safety powers requires approval from both the governing body and an independently appointed oversight body. Removing a protected reviewer requires stated grounds and an independent process. Existing stops remain binding through changes of personnel or charter until new evidence supports independent authorization to resume.
Financing must survive a justified pause.
Before work expands, publish a costed pause plan and funding terms that do not compel an unapproved release. Specify the committed operating runway, review budget, contingency responsibilities, and restrictions on investor influence. Capped returns may be considered; they are not an alignment guarantee.
Capture requires secure custody and succession.
Prepare mechanisms to halt dangerous operation, revoke controlled credentials, preserve evidence, and transfer custody through an independently supervised process. A legal license cannot recall unknown copies. Automatic release of dangerous weights is not a shutdown mechanism.
Challenge must reach the institution.
Provide protected staff reporting, recorded dissent, accessible routes for affected people to challenge a decision, and a process for correcting public claims. Publish safe methods and negative results; give qualified independent reviewers access to sensitive evidence under appropriate controls.
Who decides.
Who can stop the decision.
| Decision | Accountable authority | Required check |
|---|---|---|
| Run an experiment within an approved scope | Research lead | Security approval of environment and evaluation approval of protocol |
| Grant tools, credentials, or external reach | Security lead | Documented scope and independent review of consequential additions |
| Define final evaluation and failure criteria | Evaluation lead | Criteria recorded before the final assessment; subsequent changes remain visible |
| Authorize a bounded external pilot | Independent review function | Written case for that exact model, configuration, population, and duration |
| Pause an activity with a credible severe risk | Designated safety reviewer or incident authority | Immediate suspension followed by documented review; no retaliation for good-faith reporting |
| Resume after a blocking failure | Independent review function | New evidence addressing the failure; leadership cannot grant itself an exception |
| Weaken safety powers or remove protected reviewers | Governing body plus independently appointed oversight body | Independent consent under disclosed procedures; recorded reasons and conflicts; existing stops remain binding through the change |