c5.5 Require human approval for high-consequence actions

Bound autonomy at the point of consequence

An agent that can act on its own is convenient — until it deletes production data, moves money, or emails customers because a prompt injection told it to. The fix is not to remove autonomy everywhere, but to insert a human-in-the-loop checkpoint in front of the specific operations that are sensitive, mutative, or destructive. Everything else stays autonomous; the high-consequence actions pause for an explicit human approval.

Model the approval as a pausing workflow

AWS Step Functions' callback pattern (waitForTaskToken) is a clean way to implement this: the workflow reaches the sensitive step, emits a task token to a human (email, Slack, a ticket), and pauses — for up to a year — until someone approves or rejects:

{
  "ApproveHighRiskAction": {
    "Type": "Task",
    "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
    "Parameters": {
      "FunctionName": "notify-approver",
      "Payload": {
        "action.$": "$.proposedAction",
        "taskToken.$": "$$.Task.Token"
      }
    },
    "Next": "ExecuteAction",
    "TimeoutSeconds": 86400
  }
}

The approver's decision resumes the workflow:

# Approve
aws stepfunctions send-task-success \
  --task-token "<token>" --task-output '{"approved":true}'

# Reject
aws stepfunctions send-task-failure \
  --task-token "<token>" --error "Rejected" --cause "Not authorized by reviewer"

Decide what needs a human

Route only high-consequence actions through the checkpoint — deletes, financial transactions, permission changes, external communications, anything irreversible. Let the agent classify a proposed tool call and branch: low-risk actions execute directly; high-risk ones enter the approval state. Keep the classification server-side, not something the model can talk its way around.

Fail closed and record the decision

If the approval times out, the workflow should fail closed (do not execute), not proceed. Log who approved or rejected, when, and the exact action — this is both your audit trail and your evidence that autonomy stayed bounded.

Naming note: set TimeoutSeconds to a window that fits your operations; on timeout the action is not taken. Send the approver enough context (the exact operation and its parameters) to decide safely.