To a Hammer, Everything Is a Nail

"To a hammer, everything is a nail." That's how I feel about agents right now.

Agents are being quickly adopted, and personal agents in particular are taking off, but not every job is actually suited well to an agent.

My inbox labeler

My own inbox labeler was eating nearly half of what my AI chief of staff spent on tokens. Then it ran out my quota and took my other agents down for about 3 days. (From the outside it looked like a provider outage.)

Everybody that connects Claude or any of these other agents to their personal email likes to set up a routine that will label emails, archive emails, and reply to emails. The way that's typically done is a recurring routine that checks every hour, 30 minutes, 15 minutes, 5 minutes, or 1 minute, depending on how many tokens you want to blow on this.

That's what I had. A typical run burned about 895,000 tokens to label messages. What a waste.

Plain code

So what's the better option? Plain code. Turns out that these agents are incredibly good at writing code, so you can create a pretty simple script that does the labeling on your emails. Anything that it can't label based on its rules, it will summon an agent to help it.

I rebuilt mine that way. Rules sort the obvious stuff first (promotions, notes to myself, known senders like Airbnb). Only the leftovers go to a model, 10 at a time. It costs about 500x less, and when I checked it, it hadn't lost a single message.

Here are the 5 steps the script runs, in order:

1. Check for new unlabeled mail. If there's none, exit.
2. Promotions go to noise and get archived.
3. Notes to myself go to info.
4. Known senders (Airbnb, alerts, digests) get their labels.
5. Everything left goes to a model, one call per 10 messages.

Only step 5 uses a model. Here's the labeling job before and after the rebuild:

It runs every 10 minutes, so a typical message waits about 6 minutes for its label (10 at worst). Every 30 minutes was about 24K tokens a day, so the cadence is a speed decision, and both numbers are noise next to 21.3M.

An added benefit is security. You can filter out many potential prompt injections before they ever reach an agent.

Plain code alongside an agent

Let's expand on the concept of using plain code alongside an agent. I built a wealth tracking app that handles all of my bookkeeping.

One way to build a bookkeeping app that's replacing QuickBooks is to have an agent look at every line item that comes in and classify the transaction by the type of expense it is, and tie it to the right entity based on which credit card it landed on or which bank account. Sure, an agent could do that, but that's a complete waste of tokens, and it's exposing yourself to potential mistakes.

A better method is to have the agent do it once, have another agent validate it, and then it becomes a rule so that every transaction that looks like this becomes a piece of plain code that automatically gets routed every time. Now the agents are only being called to make judgment decisions on new, novel transactions, and they're writing rules instead of making direct edits to the general ledger.

That's another example, and there are countless more. My support tickets, for example, used to be handled directly by an agent, but now the support ticket flow is integrated between plain code that I've written to handle support issues and agents on top of that to execute.

The agent proposes an action, code validates it, I approve by reply, and code makes the change through one narrow route. The agent never writes to the database directly.

Every time I run across something pretty complicated to do for an agent, the performance just goes up significantly when you add in the right boundaries with old-fashioned code.

So if you are out there and are having trouble with an agent being reliable on a given task, then ask yourself if you need some plain code embedded with the agent to improve its performance.

The hybrid model of code plus agents is what provides the best blend of reliability, cost and performance from my experience.

Now, time for me to go back to looking for more nails to hammer.