The Goldfish Student: Why AI Needs Better Context, Not More Context

When prompting an AI, giving it some background information and references helps it produce better answers. But giving it too much information is often counterproductive.

To understand why, it helps to imagine AI as a goldfish student. The goldfish is very intelligent, but its working memory is very limited. If you ask it to read a single-page document, it can keep the whole document in its working memory while answering your questions. But if you ask it to read a full book, it cannot remember it all. What the goldfish does instead is take notes as it reads. It’s as if it wrote a one-page summary of the book, because that is all it can fit on its desk when answering your question. And that summary, not the full book, is what it uses to answer.

Could it take five pages of notes instead of one? Yes. But when answering your question, it would still need to condense those five pages into whatever fits in its working memory. Ultimately, the answer would still be based on a compressed version of the information.

As a general principle, more context helps until the goldfish’s working memory is full. Beyond that point, additional context often becomes counterproductive.

Modern AI systems work similarly. Not fully so, but partially so. Even models with very large context windows still compress and prioritize information internally, and this has profound implications.

If you give an AI a book to read and then ask it to provide advice based on it, it will often answer based on something like a one-page summary containing a paragraph per chapter. If your questions are about the book in general, that may be fine. But if your questions are about the contents of a specific chapter, the answers will often be superficial.

So, if only one chapter matters, only give the AI that chapter. Or give it the full book, but tell it that only one chapter matters. Or, give it the full book, but tell it it should only care about the information it will need to advise on this or that specific matter, so that the AI focuses on using its scarce working memory on retaining what matters.

You can also explicitly tell it to revisit the book rather than relying on its notes. Something like: “First, read the book looking for references about X. Then advise me on Y.”

The key is to ask the AI to spend its limited working memory on information relevant to what you want it to help you with.

What About Prompts Without Documents?

The same problem appears even when there are no documents involved.

Suppose you ask the goldfish to write a marketing email. Instead of giving it a few clear instructions, you write a ten-page prompt containing your company history, product descriptions, customer profiles, writing preferences, examples of previous emails, caveats, edge cases, and formatting requirements.

The goldfish faces the same problem as before. Its memory is still the same size, and it cannot fit all you gave it. So, it will try to identify the most important information, fill its limited working memory with that, and start working. But every additional instruction competes for desk space. A useful instruction may push out another useful instruction. An irrelevant detail may push out an important one.

As a result, longer prompts often suffer from what I call context degradation.

The more information you add, the harder it becomes for the AI to keep all the important information simultaneously present while reasoning.ShareCopy

This does not mean that longer prompts are always worse. Sometimes a complex task genuinely requires a lot of context. But it does mean that every sentence should justify the space it occupies in the goldfish’s memory. In many cases, a one-page prompt that clearly states the objective, constraints, and relevant background will outperform a ten-page prompt that includes everything remotely related to the task.

When in doubt, ask yourself: if the goldfish could only keep ten pieces of information on its desk, which ten would I choose? Then give it those.

Or, if you do not want to do the triaging work yourself, ask the AI to do it. Give it a clear purpose and criteria, such as: “Here is a collection of documents about my company. Retain only the information that will be relevant to writing a marketing email for this product and this customer.”

Practical Techniques

A useful technique is to separate remembering from reasoning. The reason is that each task uses working memory space. If the AI is trying to remember, summarize, search, and reason simultaneously, those tasks compete with one another.

Instead of asking, “Read these ten documents and tell me what strategy I should follow,” first ask the AI to extract the relevant information, and only then, ask it to build a strategy from that information. Separating the two steps often produces better results.

Another technique is to fully separate the ingestion of resources from the answering. First, give the AI a book and ask it to take notes on specific topics. Then, ask it to show you those notes, and review a sample of them. This step is important because it lets you verify the quality and depth of the notes. If the notes are superficial, the answers will be superficial too. Only after you have validated the notes should you ask questions based on them.

A more advanced version combines several techniques:

1. Tell the AI that you are about to give it some reference information, but before you do so, tell it what the information will be used for, so that it knows what to retain.
2. Give the AI one or more books and, for each book, ask it to create both a high-level summary and more detailed notes for each chapter.
3. Ask it to show you a sample chapter note so you can evaluate the quality of the note-taking.
4. Ask your question and instruct it to answer by consulting the summaries first and then digging into the more detailed notes when necessary.

The idea is that the AI uses the book summaries as a routing table to know which chapter notes to dig into before answering each of your questions. This ensures that only relevant information occupies its working memory.

A further technique is to use multiple agents. This is equivalent to the goldfish recruiting more goldfish and assigning a subtask to each. One reads the documents. Another analyzes them. A third reviews the answer. Each goldfish has fewer things to manage and can focus on a smaller, more relevant subset of information.

Note that some of these techniques require AI tools that can create and manage files, such as Claude Code, Claude Projects, or Codex, and are therefore unavailable in a simple web chat interface. If all you have is a web chat, you can still approximate the process by asking the AI to create notes, saving them, and then uploading those notes as the basis for future conversations.

For a complementary set of techniques on getting better results from AI, see 5 Techniques for Better AI Outcomes.

Final Thoughts

A common mistake is to assume that AI behaves like a perfect reader that remembers everything it has seen. A more useful mental model is the goldfish student: intelligent, capable, and diligent, but only able to keep a limited amount of information in its working memory at any time.

Once you understand that, many best practices become obvious. Give it a purpose before giving it information. Help it take notes. Review those notes. Break large tasks into stages. And whenever possible, reduce the amount of information competing for space in the working memory.