What not to automate with AI: my approval rule for money and irreversible actions
 |
I let AI handle all of my reading, summarizing and analysis automatically. I let it create and edit files too. But anything that spends money or publishes something to the outside world still waits for me to press approve. I decide where a task goes with two questions: if this goes wrong, does money leave my account? And can I undo the result? If a mistake could cost money, or the result can't be undone, the task needs my approval before it runs.
That line hasn't moved since I drew it. What did change was how I approve things. I built a one-tap approve button for my phone, and I took it down myself 26 days later. That story gets its own post. This one is about the line itself.
Contents
The worry that grew as the AI got better
I had already split my AI helper into three roles: planning, doing the work and checking the result. That left me with much less to review, but one worry remained.
A checker can be thorough, but a check can't pull money back once it has gone out, and it can't unpublish a post that strangers have already read. The more capable the helper became, the bigger this worry got. Capable means it gets more done without asking me first.
So I wrote the rule down in three sentences. Reading and analysis: fully automatic. Touching files: automatic, but done in two steps, so changes are recorded and I can look before anything is final. Anything that spends money or goes public: a human approves, every time. When I sorted every job I had handed over with this rule, it fit in one table.
| Task | How it runs | Why |
|---|---|---|
| Reading, summarizing, analysis | Fully automatic | If it's wrong, I run it again |
| Creating and editing files | Runs automatically, then I check the record | I can undo it |
| Publishing posts, payments, changes pushed to outside services | A human approves first | Once it leaves, I can't take it back |
I thought I had drawn these 3 rows by gut feeling. Later I found that the company behind the AI coding tool I use draws it in almost the same place in its official permissions docs. Read-only actions such as reading files and searching need no approval. Running commands needs approval. Editing files needs approval. Reading needs no approval; running commands and editing files do. That is close to my three rows.
Why "just send me a notification" wasn't enough
Before I added the button, only the notification was automatic. When a task needed my OK, the AI sent a notification to my phone. I read it, then typed a "go ahead" message back in a chat window. It worked. The longer I used it, the more holes I saw.
There was no ledger linking each alert to its task. If two alerts arrived back to back, the system couldn't tell which one I meant by "go ahead".
I also worried that I'd get tired of typing, skim an alert and type yes just to get it over with.
That isn't only my laziness. A public experiment collected more than 40,000 plays of a game where people approve or reject commands proposed by an AI. It found that people let 1 in 3 risky commands through (average accuracy 66.3%). A threat disguised under a familiar-looking command name was approved 64.7% of the time (source). Even with an approve button, I could get used to pressing it without thinking. I only found these numbers after I had built my button.
I hadn't separated receiving information from approving a task. Approval needed a record of who allowed what and when.
What my approval step looked like
So I added Approve and Reject buttons to the alert, letting me respond with one tap.
- When the AI hits a task that needs approval, it first writes a request into a ledger: a request number, what the task will do, and which job asked for it.
- Then my phone gets an alert with two buttons, Approve and Reject.
- When I tap Approve, the alert message changes to show "approved", and the same decision is written to the ledger and to the run log. That left a record of my tap in all three places.
On the first test, I pressed the button and nothing happened. When I dug in, the program receiving the button press was treating it as an unknown signal and throwing it away without logging anything. I fixed that part by taking the AI's judgment out of it completely and making it follow fixed rules only.
After it was built, I had the design reviewed from three separate angles. All three rejected it at first and between them found more than eight flaws, and I closed every one. I added two checks. The system checks the account to block approvals from anyone else, and rejects an approval if the button doesn't contain the correct value.
I thought I'd built the system properly, so the failure confused me. It took me 26 days to accept that the problem was elsewhere, outside this code.
The line you can draw today
Skip the button for now. Start with the line, in 4 steps. This is the order I actually followed:
- Write down every job you've handed to an AI or a script.
- Ask two questions of each one: if it goes wrong, does money leave? Can I undo the result?
- If either answer is a yes to money leaving or a no to undoing, make that job stop right before execution and ask you.
- Make sure a record shows who approved it and when.
Common questions
Doesn't a one-tap button make people approve without reading?
That's why every approval request carries the details: what the job is and where its result will land. A request without details should be rejected. The button only makes a decision faster. It doesn't replace reading.
What if the AI skips the approval step and just runs the task?
Put the check on the side that executes, not the side that sends the alert. Right before running, the executor looks up the approval record in the ledger and stops if there isn't one. Then no request, wherever it comes from, goes ahead without a sign-off.
Can I build this if I can't code?
The button takes some work. The principle doesn't depend on any tool, though. Turning the last step of an automation into a human check prevents most accidents. If you use automatic payments or scheduled runs, start by making the final step wait until you press something yourself.
Comments
Post a Comment