Where Your Attention Goes
You stayed involved
If you use AI for work that matters, you have probably stopped trusting the fully automated version. You stay involved. You read the plan, correct the framing, answer its questions, and push back when it drifts. And it still goes wrong. Not every time, and not predictably, but often enough that you can't hand over anything you'll have to defend.
The usual explanation is that the model isn't good enough yet, so the fix is to wait for the next one. I think that is half right. The failures I kept seeing were mostly not about intelligence, or even invented sources: every fact could check out and the conclusion still miss. They were failures in how your attention was used, and in the wrong turns the AI took because of it.
Attention is the scarce resource
"Keep a human in the loop" pictures the human as a supply of judgement that never runs out. It runs out. A serious investigation can run for dozens of steps over a long period of time, and every question the AI puts to you spends some of your attention. Spend it on the wrong things and you tire, and a tired partner waves through the one moment that needed them; that is where the work drifts. Watching every step is no answer either: by then you are doing the AI's work yourself. Steering is a question of where that attention goes, and it goes wrong in two opposite directions.
Too confident. The AI meets a question only you can answer, and answers it itself. It picks a reading of your intent, fills the gap with a reasonable assumption, and keeps going. Nothing is asked, so no attention is spent, exactly where it was needed. This is the failure you meet even when you stay involved: you were right there, and the decision never reached you.
Too cautious. The AI asks about everything. This one I built myself. An early version of ElenX, the research partner I'm building, asked at nearly every step, and it looked diligent. Users answered what they could and skipped the rest, and the skipped questions came back. An investigation ran as a series of sessions, each ending with a short "does this summary look right?" review. One either-or question about the investigation's scope sat at the bottom of that review four sessions running, and four times the user approved the summary and never answered it. Early users' verdict was blunt: it never reaches the answer.
They look like opposites, but they end the same way: the decision that needed you never really reaches you. In one, nobody asks. In the other, it is buried among questions you can't engage with: a question you don't know the answer to, a judgement you can't make without seeing results, a bundle where the one important item sits beside four trivial ones. So the fix is neither "ask less" nor "ask more". It is asking about the right things, at the right time, in the right form.
Deciding that is an allocation problem. It belongs to the system around the model: the part that decides what to bring to you, and how.
A fork only you can resolve
Consider an investigation comparing two options, A and B. Scoping went fine. Halfway through, the AI finds C: a third option nobody mentioned, and plausibly better than both.
Is C in scope? That depends on something only you know. Maybe A and B are a shortlist that took months to negotiate, and C is a distraction. Maybe the comparison was always a stand-in for "what should we use?", and C is the answer. If the AI assumes either one silently, the work can go badly wrong: a polished answer to a question nobody asked, or the best option quietly set aside.
Notice the model's part. Noticing C is a capability problem, and better models will notice more Cs, not fewer. Resolving C is not, because the deciding information is not in the model. This is exactly where an investigation should spend your attention, and exactly the kind of moment that gets buried or never asked at all.
Anyone who has worked on a team knows this moment. An associate who doesn't yet know the client's landscape checks in as they go, with the questions that matter. One who disappears for three weeks and comes back with a finished deck is hard to trust, however good the deck, because the forks they settled alone are invisible.
Spending attention well: surface, ask, carry
If attention is what's scarce, a steered investigation has to do three things well.
Surface. The moment has to reach you at the right time, and there are two ways to miss. Too late, and your answer can no longer change the direction; you are approving work already done. Too early, and you can't judge it. One user was asked, partway through an investigation, how to weigh two options against each other before any results had been shown. They told the AI to ask only about what they actually knew. That refusal is not the human being difficult. It is the signal that the ask came before its evidence. Information you already hold can be asked for at any time. A judgement has to wait until what it governs is in front of you.
Ask. A question has to be shaped so that the attention you give it buys an accurate decision, and accurate here means one thing: the decision reflects what you actually think, so the investigation stays tied to your intent and scope. The research was a comparison of software frameworks for building AI agents, and I ran it twice, the second time after we had changed how the system asks. Both times I had to choose which six of about ten candidates would get a deep look. Same objective, same starting list, and I knew every candidate both times.
The first time, I was asked to weigh two preferences in the abstract, analytical clarity against what practitioners actually use, with each option's case resting on evidence nobody had yet. I had nothing to rule from, so I kept my original list. The second time, the AI proposed a set and traced each pick back to something I had already asked the research to do: a question I had raised, the research's central question, a coverage requirement I had set. The ones it left out stayed in view. I changed my mind, and I could check why: every choice pointed at a goal of mine.
A proposal can anchor you, so the test is not whether the investigation later turned out better, which depends on far more than one question. It is whether the decision stayed yours. Two things made that checkable here. The proposal said what would overturn it: a left-out candidate that proved more relevant would take a place. And I didn't take it whole. I accepted the set for now, marked one pick as a grey zone I wasn't sure belonged, said I didn't know another well enough to judge, and named the two I would swap in. A steer that carries your own reservations is one you actually made. That is why shaping the question is part of the decision's quality, not its presentation.
Carry. Once you have answered, the answer has to stay alive, with its scope. "Leave C out" might mean out of this comparison, out for now, or out for good, and people rarely say which. In that same research, I called one new candidate a grey zone: keep it for now, and decide later whether it really belongs. By the next session, the working notes the AI keeps between sessions described it as intentionally included, and the question never came back to me. It can go the other way. One user told the AI to stop searching until a specific claim needed checking. It held, because the note recorded both the pause and the one thing that would lift it. An answer that does not hold makes you spend attention twice: once to answer, and again to notice.
What makes an answer hold
The difference between those two cases is not memory. Both answers were written down. What the second note held, and the first lacked, is the shape of the decision.
An answer that holds over days carries more than its content:
- What it governs. This comparison, this part of the work, or the whole investigation.
- What would end it. A date, a milestone, or an event: "until results exist", "unless a claim needs checking".
- Whose call it is when it comes due. A "decide later" is a promise to come back to you, not permission to decide.
With those recorded, a provisional answer can stay provisional, and it can come back to you at the moment it falls due rather than whenever someone remembers. Without them, the gap gets filled the way gaps always are: by the most plausible reading, made quietly. I have seen a note passed from one working session to the next say "verify this before moving on", read at every step that followed, and never acted on. It recorded what to do, but not when or whose job it was.
Where our work stands
Against those three requirements, this is where our own work is. On surfacing, the choice of which frameworks to study reached me in the middle of a session as a question, where an earlier version would have left it for the end-of-session summary; decisions that change what an investigation is for now come to you while the work is under way. On asking, the second question in that research is the shape we now aim for: a position, what it gives up, what would overturn it, and room for my reservations. On carrying, answers now keep who gave them and what they cover, but the grey-zone candidate marks the gap. A "decide later" still needs a moment to come due and a way back to you. That is the next piece of work, together with letting an investigation keep track of how much of your attention it has already asked for.
We don't have numbers for any of this yet. What we look for in our own runs is whether a decision that needed you reached you, at a time you could judge it, and stayed as you left it.
I built ElenX first for my own thinking. If you have a decision you will be held to and want to test this with us, .