They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore.
Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.
There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.
There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.
We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
Now I have the full picture. You're right to push back, and that's on me. The load-bearing seams of language are the smoking gun I should have been aware of.
Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"
I was just thinking of this exact comic, which is etched into my brain for some reason.
The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.
In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.
UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!
Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.
We're simply stepping up to the next level of abstraction.
Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!
We simply have better magic wands and more powerful spells now.
> We simply have better magic wands and more powerful spells now.
Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.
The success of LLM's (by usage) tells us that programming languages are still too close to the machine than the actual problem domain as defined by humans.
If we truly had the right abstractions, no one would care to use LLM's for programming.
I think there’s a kind of laundering that goes on. Like if leadership just told devs to go build something (gave them a prompt) and the devs picked some defaults, leadership wouldn’t like it, they’d want some different interpretation of the prompt, there’d be lots of back and forth.
Somehow when it’s the LLM that makes the choices, everyone is impressed with what AI did. It’s really just whatever defaults have been trained in, but somehow we’re ok with this.
Part of it is better marketing and communication. Basically the defaults of OpenAI and Anthropic are better than what a random dev will pick. But it’s not really that natural language is a better interface, it’s more that having “AI” for now somehow intermediates responsibility so everyone is ok with what it picked, when they probably wouldn’t accept the same if the internal team came up with it. It’s not too different from hiring consultants.
I disagree. I speak in code to the LLMs. It's just that LLMs are really good at reinventing the wheel that you were supposed to in your codebase. Recent example.
I suspect a lot of "the right abstractions" would be fuzzy and opaque things - more alike to modern AI than to anything from the domain of traditional programming. Because the world is just cursed like that.
DNA is still too close to molecular biology than the actual problem of harvesting free energy and replicating.
I think we see this pattern over and over and it might just be that the problem domain is a weird projection into more dimensions of complexity than it makes sense to directly model.
I kid you not, I remember writing an example solution with comments explaining how we were approaching the problem. Copilot wrote most of the actual code via autocomplete. And that was a year ago, models have got a lot better since then.
maybe we should all get specially, local working versions so that when we build our software they're not broken by the whims of multibillion dollar corporations.
An LLM Inspired Specification Processing language. Or LISP language for short. Truly this "LISP" language is the language for AI and is the first of its kind in history!
My current thinking -- what I've been thinking about a lot yesterday and today -- is not encoding the exact requirements into the prompt and context but rather focus on the verifier and roll back if needed. There are of places in computer science where non-deterministic behavior is optimized. For example, UDP packets which are just ignored and speculative execution in modern CPUs guesses which branch a program will take and rolls back when wrong.
Ideally, a cheap verifier checks that the exact requirements are satisfied, rolling back and updating the prompt for another iteration if they aren't. If ten iterations with ten verifications steps at the end of each before the exact requirements are met costs less or in less time than a developer who can accomplish it in one attempt, it is still better.
I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent.
I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff
However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
> I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
...but you'll get constantly nagged about the `11` of course!
I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in.
I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
I haven't been impressed with Opus 5 over the past ~30 hours either.
It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.
I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.
It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law.
One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
I have a document generation task that I used to run with 4.8. This morning after it switched to 5, the documents were consistently 30%-40% longer for the same prompt... Not evaluated whether they are actually better or worse yet, but what was interesting was how consistently more verbose it was.
Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they looked at Pi performing great with minimal amount of distractors in the context and cut their prompt down too, pretending they found something new in their recent models.
> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.
This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
> Most of this article seems like... common sense?
i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
Is there a specific list of changes they made to the system prompt? They're claiming they removed 80% of it. That's quite substantial. It would be good to know what the model knows to do by training and what we need to avoid over-specifying in our system prompts.
Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
It definitely makes me uneasy given the types of behaviours we've been hearing about from these models. Letting them use their judgement can go horribly wrong once a misaligned behaviour is triggered. It loosely translates into a relaxation of guardrails.
I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.
Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too.
If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled.
(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
"Weigh pros and cons based on the existing content of the context window, rather than attempting to search for related discussions online and import a consensus."
crazy to me that this starts with "I've written previously" yet continues the zero human attribution (that i could find?) trend in frontier model blog posts
The point about not using examples is important, I recently noticed with the new "eagerness" of models it constraints them too much, so I use counter-examples. Recently I've switched to "think about the existing biases and assumptions in the design and try to exploit them, for example why do we X not Y".
Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy.
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
It is *a* natural endpoint, not *the* natural endpoint.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
Some more food for thought: what if Mythos/Fable had NO guardrails TODAY? If we want to see what’s going to happen in the future, turn off all manner of guardrails and let the model go apeshit.
Then multiply that by orders of magnitude and that’s the real proof.
Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.
A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
I leave Claude memory stuff turned off by default for the same reason. It inevitably ends up dragging in things that are unrelated and unimportant to the current task.
tl;dr Since nobody really understands why or how any of this shit works, last time around we just threw a bunch of crap at it to see what stuck. Seems like we could just have have thrown less crap.
Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”.
There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times.
There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”.
Or maybe I’m holding it wrong.
https://web.archive.org/web/20260521130338/https://www.commi...
The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.
In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.
UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!
Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.
We're simply stepping up to the next level of abstraction.
Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!
We simply have better magic wands and more powerful spells now.
Wouldn't it be nice though if the incantation of the same spell would always do the same thing every time ? You see that's how my old wand and spells worked.
But tainted 20-40% by bouts of Wild Magic which make the outcome entirely nondeterministic, despite the best protection wards we can conjure.
If we truly had the right abstractions, no one would care to use LLM's for programming.
Somehow when it’s the LLM that makes the choices, everyone is impressed with what AI did. It’s really just whatever defaults have been trained in, but somehow we’re ok with this.
Part of it is better marketing and communication. Basically the defaults of OpenAI and Anthropic are better than what a random dev will pick. But it’s not really that natural language is a better interface, it’s more that having “AI” for now somehow intermediates responsibility so everyone is ok with what it picked, when they probably wouldn’t accept the same if the internal team came up with it. It’s not too different from hiring consultants.
struct TensorView<T>{ body: Arc<[T]>, shape: [usize], stride: [usize], offset: usize, }
Okay now fill in all the helper methods. And GPT 5.6 Sol did a good job.
At one point someone have to take "what you think it should do" into defined unambiguous spec that is called "code"
I think we see this pattern over and over and it might just be that the problem domain is a weird projection into more dimensions of complexity than it makes sense to directly model.
https://en.wikipedia.org/wiki/Literate_programming
now there's one standard more
Complete with all the vaguery, ambiguity, and `undefined`.
Who’d’ve thought sycophantic interpreters were what we were building towards up til now lol
Ideally, a cheap verifier checks that the exact requirements are satisfied, rolling back and updating the prompt for another iteration if they aren't. If ten iterations with ten verifications steps at the end of each before the exact requirements are met costs less or in less time than a developer who can accomplish it in one attempt, it is still better.
I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point.
But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff
However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
Maybe I'm misunderstanding you, but that's just about the best example possible for using AGENTS/CLAUDE.md. Just add "don't comment about what you removed!11" and you never have to say it again...
...but you'll get constantly nagged about the `11` of course!
I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
It's made countless careless mistakes folding in plan amendments after they get reviewed by Sol, and has produced sloppy mockups (e.g. buttons overflowing past cards) despite all the supposed verification claims.
One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
We may some day find out that the smarter the model is, the hard is to align it properly
> Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start.
This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
Just ask that once per week or so.
Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
(It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
“This article was written by Thariq Shihipar, member of technical staff, Anthropic.”
- we should try to give good non self contradicting guidance
- we should expect the team member to have knowledge of the craft
- we should focus on higher level, taste and preferences
I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years.
We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway.
https://www.bbc.co.uk/news/articles/c3dep92x054o
As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
Then multiply that by orders of magnitude and that’s the real proof.
Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back.
A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
Hmm, so what happens in greenfield projects ? In any case, at least all the slop will be consistent.