The new skill in AI is not prompting, it's context engineering

Here's a summary of the themes from the Hacker News discussion:

The Centrality of Context Engineering in AI Agents

A primary theme is the shift in focus from "magic prompts" or model updates to the meticulous engineering of context for AI agents. This involves understanding the specific business use case, defining desired outputs, and structuring information so an LLM can effectively perform a task.

• "Building powerful and reliable AI Agents is becoming less about finding a magic prompt or model updates. It is about the engineering of context and providing the right information and tools, in the right format, at the right time." - baxtr
• "The LLM is just an oven; a fantastical oven. But for it to produce a good product still depends on picking good ingredients, in the right ratio, and preparing them with care. You hit the bake button, then you still need to finish it off with presentation and decoration." - CharlieDigital
• "in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step." - Andrej Karpathy (as quoted by simonw)

The Human Analogy: A Divisive Comparison

The discussion features a notable debate on the usefulness of comparing AI behavior to human cognition or workflows. While some find these comparisons insightful for understanding LLM limitations, others dismiss them as inaccurate or a distraction from the technical aspects.

• "That’s actually also true for humans: the more context (aka right info at the right time) you provide the better for solving tasks." - baxtr
• "I am not a fan of this banal trend of superficially comparing aspects of machine learning to humans. It doesn't provide any insight and is hardly ever accurate." - root_axis
• "I've seen a lot of cases where, if you look at the context you're giving the model and imagine giving it to a human (just not yourself or your coworker, someone who doesn't already know what you're trying to achieve - think mechanical turk), the human would be unlikely to give the output you want." - furyofantares
• "I don't see the usefulness of drawing a comparison to a human. 'Context' in this sense is a technical term with a clear meaning. The anthropomorphization doesn't enlighten our understanding of the LLM in any way." - root_axis

The Nuance of "Better" Context Over "More" Context

A key refinement on the context engineering theme is the emphasis on the quality and relevance of context, rather than simply increasing the quantity.

• "Not 'more' context. 'Better' context." - lupire
• "Context is often incomplete, unclear, contradictory, or just contains too much distracting information. Those are all things that will cause an LLM to fail that can be fixed by thinking about how an unrelated human would do the job." - furyofantares
• "It's easy to forget that the conversation itself is what the LLM is helping to create. Humans will ignore or depriotitize extra information. They also need the extra information to get an idea of what you're looking for in a loose sense. The LLM is much more easily influenced by any extra wording you include, and loose guiding is likely to become strict guiding" - EricMausler

The Evolving Role of Tool Use and Agent Capabilities

The ability of LLMs to interact with tools and search for information is seen as a significant advancement, blurring the lines between simple prompting and more complex agentic behavior.

• "These days, so can LLM systems. The tool calling pattern got really good in the last six months, and one of the most common uses of that is to let LLMs search for information they need to add to their context." - simonw
• "The same patterns is increasingly showing up in coding agents, giving them the ability to search for relevant files or even pull in official document documentation for libraries." - simonw
• "My guess is the main issue is latency and accuracy; a single agent without all the routing/evaluation sub-agents around it that introduce cumulative errors, lead to infinite loops and slow it down would likely be much faster, accurate and could be cached at the token level on a GPU, reducing token preprocessing time further." - storus

The "Buzzword" Phenomenon and The Rebranding of Prompt Engineering

Several users express skepticism and mild annoyance regarding the emergence of "context engineering" as a new term, viewing it as a rebranding or a repackaging of existing concepts, potentially to generate hype or justify new roles.

• "It is somewhat bothersome to have another buzz phrase. I don't why we are doing this, other than there was a Xeet from the Shopify CEO, QT'd approvingly by Karpathy, then its written up at length, and tied to another set of blog posts." - refulgentis
• "The way I see it we're trying to rebrand because the term 'prompt engineering' got redefined to mean 'typing prompts full of stupid hacks about things like tipping and dead grandmas into a chatbot'." - simonw
• "This is no secret or suspicion. It is definitely about avoiding (more accurately, delaying until legislation destroys the business model) the wrath of copyright holders with enough lawyers." - Zopieux (regarding why companies might obscure search results)
• "rediscovering basic security concepts and hygiene from 2005 is also a very hot AI thing right now, so that tracks." - Zopieux

The Ongoing Debate on LLM Determinism and Reproducibility

A recurring concern is the non-deterministic nature of LLMs, which contrasts with traditional programming. This leads to discussions about whether AI can truly replace or augment software engineering without deterministic outputs and the implications for reliability and reproducibility.

• "building your own agents, it is also critical when using agentic coding right now. These limitations/behaviors will be with us for a while." - the_mitsuhiko
• "This is like telling a soccer player that no change in practice or technique is fundamentally different than another, because ultimately people are non-deterministic machines." - csallen
• "A half baked programming language that isn't deterministic or reproducible or guaranteed to do what you want. Worst of all worlds unless your input and output domains are tolerant to that, which most aren't." - nimish
• "So without verification of some kind, or an expert (human or AGI), how could one in general cases use that code safely and usefully? There could be cases where one could do other kinds of mitigation, but there are at least a large proportion of cases where I am skeptical that sole mitigation strategies would be sufficient." - inspectorwadget

The Critical Role of Evaluation and Validation

The importance of rigorous evaluation and testing for AI projects is highlighted, with some users expressing surprise at its perceived absence in many projects.

• "One thing that is missing from this list is: evaluations! I'm shocked how often I still see large AI projects being run without any regard to evals. Evals are more important for AI projects than test suites are for traditional engineering ones." - crystal_revenge
• "The quackery comes in the application of these techniques, promising that they 'work' without ever really showing it. Of course what's suggested in that blog sounds rational -- they're just restating common project management practices. What makes it quackery is there's no evidence to show that these 'suggestions' actually work (and how well) when it comes to using LLMs. There's no measurement, no rigor, no analysis. Just suggestions and anecdotes: 'Here's what we did and it worked great for us!'" - ModernMech

The Future of AI and Skill Evolution

There's a general sentiment that skills related to interacting with AI are evolving rapidly, with some predicting that "context engineering" will also become a commodity or be superseded by more advanced techniques or internalized model behaviors.

• "I don't know what prompt engineering will look like in the future, but without AGI, I remain skeptical about verification of different kinds of code not being required in at least a sizable proportion of cases." - inspectorwadget
• "Cool, but wait another year or two and context engineering will be obsolete as well. It still feels like tinkering with the machine, which is what AI is (supposed to be) moving us away from." - hnthrow90348765
• "If you're not writing your own agents, you can skip this skill." - tptacek (in response to the idea that context engineering is a necessary skill for everyone)

The Ethics of Data Usage and Copyright

The discussion touches upon the ethical implications of how AI models are trained and how they use external data, particularly concerning copyright and the transparency of information retrieval.

• "This is no secret or suspicion. It is definitely about avoiding (more accurately, delaying until legislation destroys the business model) the wrath of copyright holders with enough lawyers." - Zopieux
• "I find this very hypocritical given that for all intents and purposes the infringement already happened at training time, since most content wasn't acquired with any form of retribution or attribution." - Zopieux
• "I don't understand how you can look at behavior like this from the companies selling these systems and conclude that it is ethical for them to do so, or for you to promote their products." - NoraCodes