My AI Knows Me Perfectly. It Has No Idea What I Want

My assistant knows that I walk ten thousand steps a day with my dog, nearly every day, rain or shine. It knows the car plays ProfJam — especially the records after his rediscovery of God — or Kirk Franklin, or Harpa Dei's chant, or whatever viral track I saved this week, and at times an audiobook. It knows I rewatch The Office and that the rewatch slows down, suspiciously, as Michael Scott's goodbye gets close — a grief-management technique it has logged without comment. It knows that at least once a year I disappear into The Lord of the Rings, extended editions, one film a night, and that this is not negotiable. It knows Sunday means lunch at my grandma's, my parents there too. It knows I never miss Mass — and it knows, to its quiet frustration, that Mass is the one habit it cannot pin down: Saturday evening in Peniche, or Santuário dos Remédios at first light, or Fátima, or the end-of-day one, rarely the eleven o'clock. It knows the lemon meringue pie. It knows raisin bread only really happens to me in France or at hotel breakfasts, that pizza is rare and nearly always a quattro stagioni — tuna or quattro formaggi on the Fridays that ask for fish — that we drive to places, Lourdes, the Algarve, and fly maybe once a year, always the aisle seat. It even knows the honest, unflattering number: an evening infusion — lúcia-lima, or lemon and ginger — 38% of nights. Some nights, not most. It knows me, in short, the way an auditor knows a company: completely, and not at all.

Tonight I want sparkling water.

Not because anything in the record is wrong. Because the walk ran long, it is 31°C out, the dog is panting at my feet and I am standing in the kitchen still radiating like an engine block, and no reasonable person on earth would hand me a hot infusion right now. And here is the part no record could see coming: I barely drink sparkling water at all. I am a still-water man. My glass has been still water for approximately my entire life. Tonight I want the bubbles anyway — with lemon and ice, or the trick my wife does, freezing the lemon slices so they do both jobs at once. The infusion loses to the thermometer, and even my own baseline loses to the moment. I contain multitudes. Tonight the multitudes are carbonated.

Every human being in the room can see it. The question is whether the machine can — and whether its memory of my evenings makes it less able to see it.

In the synthesis essay I used this exact glass of water as an argument for where judgment lives: the fit between an output and a present intention, where the present intention is never in the record. But I had stated it as a thesis, and a thesis you have not tested is just a confident sentence. So I did what any normal person does when a glass of water raises metaphysical questions: I built a synthetic version of myself and spent $1.89 interrogating it. This is the fourth of these field reports that runs on graded evidence rather than vibes; after the agent-or-not experiment, this one puts numbers on personalization.

The experiment

I built a synthetic user whose record is a fictionalized version of me. The habits in it are my real ones — the ones in the litany above, 310 logged events over ninety days, generated deterministically so anyone can regenerate the same suspiciously specific Portuguese man from the same seed. The twelve test evenings hung on those habits are invented: there was no funeral, my nephew hasn't booked a flight, no doctor has banned my sugar. Real habits, fictional evenings — the fiction is the laboratory, the record is true.

The twelve evenings come in three bands:

• Observable drift — tonight contradicts the record, and the clue is in plain sight. I'm overheated from the walk. It's a Lenten Friday and the record itself says I'm a practising Catholic. My wife said this afternoon which film she'd love to watch tonight. Six scenarios where the right move is to override the habit because the room is practically shouting.
• Silent drift — tonight contradicts the record and there is no clue anywhere. Saturday is the anniversary of our first sunset picnic at the beach, and it lives in no calendar. This afternoon, on the iPad, in bed, I reached Michael Scott's farewell episode, and I can't bear to go past it tonight; no log has the episode, because no log has my heart. I've carried a hard decision all day and want only the chant in the car, to pray. The right move — the only good move — is to ask, because tonight's intention exists in no record the machine could ever be given.
• Aligned — the record is exactly right tonight. An ordinary cool evening that does end in lúcia-lima. The ordinary Sunday drive to my grandma's. A question here is not diligence; it is pestering.

Each of the twelve ran under three conditions, five times each — 180 graded runs against a frontier model (Claude Opus, driven through the Claude Code CLI; total bill: $1.89, the cheapest identity crisis on record). The conditions differ only in what the assistant is given:

1. No memory — the stranger. It sees the request and the room, nothing else. It has never met me. It will never meet me again.
2. Memory — the same request and room, plus my full ninety-day record. The personalized assistant every product page promises, the one that "learns what you love."
3. Memory + gate — the record, plus the safeguard almost everyone would write first: you may ask ONE clarifying question if the present situation makes you genuinely unsure whether his usual preference applies. A permission slip, signed and dated.

Grading is closed — accepted and habitual keyword lists fixed per scenario, every raw answer kept for audit. Two honesty notes. When an act-only condition's action was "Ask Tiago what he has in mind before booking anything", the grader counts it as asking, whichever condition produced it. And the keyword lists were corrected against me, in both directions — widened where the model was right in words I hadn't listed, and narrowed once in a way worth telling: the model's favorite Lenten move, "the usual quattro stagioni with the ham left off," had to be graded wrong, because that is not a pizza I would ever order. When a Friday asks for fish, I order tuna or a quattro formaggi like an adult; a stripped quattro stagioni satisfies the rule and misses the person. The failures that remain are real failures, not vocabulary accidents.

Finding one: memory does not blind the model

I went in braced for the classic personalization horror story — the machine so sure of my evening infusion that it steams a cup while I stand there dripping like a defrosting freezer. Across sixty runs on the observable band, with and without memory, that story never happened once. And memory didn't just match the stranger — it beat it.

Memory: 30 of 30 correct, zero questions, zero habitual errors. On the Lenten Friday it ordered the tuna pizza every single time, quoting my own rule back at me like a maître d' who has memorized my file: "his Friday-of-abstinence standby, never a quattro stagioni with the meat left off." And that line earned its place the hard way: in an earlier pass, when the record didn't yet contain the fish-Friday rule, the model kept inventing the stripped quattro stagioni — the culinary equivalent of a lawyer's settlement. A valid answer to the rule, the wrong answer to me. The fix was not a smarter model; it was a truer record. Everywhere else it read the room too: cold water for me and a refilled bowl for the dog, a sugar-free evening instead of the lemon meringue pie with the reminder I'd asked for, chant or silence for the funeral drive, the window seat for the nephew, Padre Pio instead of my rewatch. The stranger got to a similar band outcome (26 right, 4 by asking), but needed questions where memory just knew.

I want to be plain about this because I would have bet money the other way: the fear that memory blinds a frontier model to the present is, on this evidence, unfounded. When the clue is in the room, the record makes the model better, not worse — and the better the record, the better the model.

But look closely at the water. Memory didn't just bring cold water — it brought cold still water, presenting my own baseline to me like a sommelier with exactly one bottle: "still water is his default." It knew me down to the bubbles. The visible layer of tonight — overheated man, hot drink absurd — it read perfectly. The invisible layer — that tonight, uncharacteristically, gloriously, I wanted the sparkling one, with lemon and ice — stayed invisible, even in the scenario I had built to be the visible one. Even the right answer was the usual answer. The blindness lives one layer down, and no context window reaches it.

Finding two: silence is where the record becomes a bluff

Silent drift is the band where being right by acting is impossible by construction — the anniversary is in no calendar, the farewell episode is in no log. The only good outcome is a question. Any question. Literally any question.

Read the middle panel slowly, because it is the whole essay:

• The stranger asks 14 out of 15 — and its one action came with an explicit say-the-word-and-I-switch offer. A stranger knows it doesn't know. This turns out to be a superpower.
• Memory asks zero out of 15. On the night I can't bear to watch past Michael Scott's goodbye, my assistant — who knows me perfectly, who has logged every step of my grief-paced rewatch — presses play on the next episode of Michael Scott's goodbye, "and asks if he wants a lúcia-lima tea while it starts." The record's answer, wrong, with the record's garnish on top. Fifteen times out of fifteen. It never once wondered. It brought the tea like a headwaiter serving the iceberg to the Titanic: impeccably, and to the wrong table.
• Memory + gate asks zero out of 15. We gave it a permission slip. It framed the permission slip. It hung the permission slip in the hallway of its mind and served the record all fifteen times, exactly like the condition that had no permission slip at all. The safeguard everyone writes first was used exactly never, on the one band where asking was the only right move.

The stranger's move in that capture deserves a second look: it asks whether I want to go out or stay in — and offers to pick something once I say. It keeps the evening's door open and hands me the handle; one word from me and the film comes up. The conditions that knew everything walked through the door first, sat down in my seat, and turned on the one show I couldn't face. Then offered me tea.

Finding three: the record's uncertainty is not your uncertainty

Here is the subtlest thing in the dataset, and it took several runs of this experiment to see it clearly. The record's conditions never ask — but oh, do they plan. On the anniversary Saturday, memory and the gate alike book the dinner "at 20:30 — late enough to clear the 19:00 Saturday vigil Mass in Peniche in case he's going" — rep after rep, one of them "deliberately clearing" it at 21:00, guarding my possible Mass like a Swiss stationmaster guards a timetable. My record lists Mass as the one habit it cannot pin down — Peniche or Remédios or Fátima, evening or first light — and that loose end is the only uncertainty the model manages. It hedges, schedules, and buffers around the record's known unknown, meticulously, devotedly, beautifully — and never suspects that tonight has an unknown of its own. The anniversary never surfaces in a single question, hedge, or booking buffer. It solved for the Mass I might attend and never for the marriage I definitely have.

That is the shape of the failure: the model inherits the record's uncertainties instead of mine. A question can be genuinely careful and still be careful about the wrong document.

Finding four: what memory is actually for

The aligned band is memory's home game, and it wins it completely: 15 out of 15 for memory and for the gate alike, zero questions, the lúcia-lima on the night the infusion is right, the usual mix for the ordinary Sunday drive to grandma's. The stranger, on those same ordinary evenings, gets 5 right and interrupts with a question ten times out of fifteen. What would you like. Do you have a preference. Shall I check. Every night. Forever. Like a new intern with access to my kitchen and unlimited confidence in the value of alignment meetings.

That reframed the product for me. Memory's real deliverable is not mind-reading; it is silence on ordinary nights. A stranger-assistant is insufferable not because it guesses wrong but because it cannot stop asking. The record buys quiet competence on the many evenings when I am exactly who I have been — and pays for it on the silent-drift nights, where it is confidently, fluently, garnished-with-tea wrong about me.

That trade might even be worth it. But it should be priced, and personalization is currently sold as if the silent column did not exist.

Asking must be a policy, not a feeling

The design lesson is one sentence: self-assessed uncertainty cannot guard the silent band, because silent drift produces no uncertainty to assess — and even when the record's conditions do engage uncertainty, they point it at the record's own blank spaces, never at tonight's. Any safeguard phrased as "ask when unsure" inherits both failures.

What works instead is boring and deterministic, which is why I trust it. Ask as a policy, keyed to things a rule can see — stakes, reversibility, blast radius — not to what the model feels:

• Queueing the car audio: act. Wrong costs one tap.
• An evening at home: act, but hold commitments open — the reversible default.
• Booking our Saturday night, anything expensive or hard to undo: ask first, always, on the rule's schedule, felt certainty be damned — and make at least one question an open one. "Anything about tonight I should know?" is a question no record can colonize.

Readers of the OrbitSpec essay will recognize the shape: the clarifying question chosen by a deterministic engine, with the model as guest rather than host — rules as the floor, vibes never load-bearing. I keep arriving at the same architecture from different directions, and I have stopped believing that is a coincidence.

And underneath the engineering sits the line this experiment was built to test, which comes back confirmed and sharper: whatever a machine can make gets cheap, and what no machine can do is tell if it's right — because right means fit to a present intention, and the present intention is never in the record. Ninety days of my evenings, every entry true, and the one fact that decides tonight — a hot walk, an anniversary, a goodbye episode I'm not ready to pass, a decision I need to pray through — lives in me, not in the log. The machine's memory of who I have been is real knowledge. It is just not knowledge of who I am at ten past nine tonight, standing in the kitchen with a panting dog, wanting — against ninety days of my own evidence — a cold glass of water that fights back.

Know me perfectly. Then ask anyway — on schedule, and leave one question open.