Some thoughts on LLMs and software development

The Nature of Technological Bubbles and Historical Parallels

A central theme is the debate over whether technological advancements inevitably lead to economic bubbles, and if so, whether the current AI boom fits this pattern. Some argue that historical parallels, such as canals and railroads, demonstrate a recurring cycle of over-investment and speculative bubbles.

• "All major technological advances have come with economic bubbles, from canals and railroads to the internet." (krainboltgreene)
• Others challenge this by questioning whether "airflight bubble" or a "car bubble" existed at the technologies' invention. (krainboltgreene)
• The debate extends to whether exuberance in sectors like airlines constitutes a "bubble." (mmmm2)
• Another perspective suggests that an "AI bubble" might not be about the technology itself. (Marazan)
• There's a consensus that railroad history includes "a long series of huge bubbles." (tptacek)
• The existence of bubbles, even if not directly tied to initial invention, is acknowledged in other sectors like electricity and steel production. (insane_dreamer, cake_robot)
• One participant recalls the "Lindberg Boom" as an example of over-speculation in early aviation. (avation)
• The comparison to the internet bubble is noted, with a question about whether data centers will be valuable infrastructure or written off like disused railroad tracks. (anthem2025)

"Hallucinations" as a Core Feature or Mischaracterized Problem of LLMs

A significant portion of the discussion revolves around the concept of "hallucinations" in LLMs, with differing interpretations: are they fundamental to how LLMs work, a misnomer for a different issue, or a sign of a broken "filter"?

• A provocative framing suggests, "hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful." (sebnukem2, oo0shiny)
• This perspective is countered by the idea that an "agent" is not just layered hallucinations but includes non-LLM code and tools that do not hallucinate. (tptacek)
• The definition of hallucination in AI is tied to the generation of false or misleading information presented as fact. (ants_everywhere)
• One viewpoint states that the term "hallucination" is misused because LLMs don't intentionally lie but rather cannot distinguish truth from falsehood, or they "bullshit" by not caring about truth. (ninetyninenine, BlueTemplar)
• A comparison is drawn between LLM "hallucinations" and human thought processes, particularly in conditions like schizophrenia where the "filter" for unreasonable ideas might be broken. (armchairhacker)
• This is further elaborated with the concept of predictive processing, where perception is distinguished from hallucination by its grounding in physical reality. (keeda)
• The accuracy of LLM outputs is debated, with some arguing that a majority of hallucinations are actually true. (ninetyninenine)
• It's also suggested that "hallucination" might be an inaccurate term when the LLM simply runs the same process for everything, and the output doesn't "work" as expected. (anthem2025)
• An interesting historical parallel is drawn to Martin Fowler's renaming of "Inversion of Control" to "Dependency Injection," suggesting a potential misunderstanding of the underlying concept. (Scubabear68)
• The discussion also touches on the idea of "positive hallucinations" and the potential for LLMs to generate outputs that seem plausible but are fundamentally flawed due to a lack of true understanding. (fencepost, vkou)

The Shift Towards Non-Determinism in Software Engineering and its Implications

A core concern is whether the increasing reliance on LLMs in software development represents a move away from the deterministic nature of traditional computing, and what the consequences of this shift might be for knowledge preservation, education, and the very nature of programming.

• There's a sentiment that LLMs mark a point where software engineering might be entering a "world of non-determinism," unlike other engineering fields. (ares623)
• This is contrasted with the idea that software engineers have always strived to introduce determinism and that LLMs push in the opposite direction. (ares623, delusional)
• The beauty and value of programming are seen in its deterministic nature, allowing for step-by-step tracing and understanding, which is feared to be lost. (didericis, pton_xd)
• The difficulty in producing byte-for-byte deterministic builds is highlighted as evidence of underlying non-determinism that might be more prevalent than realized. (ants_everywhere)
• The introduction of LLMs is likened to a "five years old child transcribe[ing] the experimentally measured values without checking," leading to errors. (Viliam1234)
• The argument is made that LLMs introduce chaos monkeys or randomness, rather than just error tolerances, and that this is a fundamental difference. (makeitdouble)
• The analogy is drawn to the shift from Newtonian determinism to quantum mechanics, questioning whether this is a regressive step. (AaronAPU)
• A strong counterpoint is that software engineers have always dealt with non-determinism (e.g., web requests, network load) by applying engineering principles to enforce it. (skydhash)
• There's resistance to the idea that relying on LLMs undermines the foundational deterministic principles of computing, potentially throwing away decades of hard work. (delusional)
• Similarly, there are concerns that asking LLMs to "do X" repeatedly instead of writing deterministic scripts is wasteful and counter to progress. (sodapopcan)

The Role of LLMs as Collaborators and their Impact on Junior Developers

The discussion frequently uses the analogy of LLMs as "junior colleagues" or "senior colleagues" with specific flaws, exploring how this impacts developer productivity, learning, and the future of the profession.

• LLMs are often compared to junior colleagues, readily producing code with apparent "green tests" that fail upon execution. (insane_dreamer)
• This behavior is contrasted with human junior engineers, where such unreliability would lead to HR involvement; however, the speed of LLMs makes the comparison complex. (CuriouslyC, xmprt)
• A more nuanced view likens them to "extremely experienced and knowledgeable senior colleague[s] – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted." (nicwolff)
• The potential for LLMs to make junior developers "screwed" is raised, as skipping the fundamental step of writing code and debugging breaks the learning feedback loop. (sfink)
• Conversely, some argue that LLMs can help juniors onboard faster by performing tasks that would otherwise be time sinks, allowing them to focus on higher-level concepts and learn through experience. (skhameneh)
• The analogy of LLMs as a "junior colleague" is challenged because LLMs can generate code much faster, but they also lack the self-awareness of a human making a mistake. (tricky_theclown)
• LLMs are seen as shallow but broad, typing incredibly fast but requiring more careful instruction than humans. (furyofantares)
• There's a concern that the pressure to be "productive" with LLMs might make it feel irresponsible to spend time learning fundamentals, potentially harming junior developers' skill acquisition. (sfink)
• The idea of "vibe coding" is discussed, with differing interpretations of whether it's about intuitive iterative development or precisely planned instruction to an LLM. (skhameneh, epolanski)
• The notion that "written code is no longer a time sink" is disputed, with the argument that figuring out what code to write and fixing it remain critical, and LLMs only help with a part of this. (sfink)
• It's suggested that LLMs, like junior developers, can be easily swayed and might deviate from tasks or implement instructions incorrectly despite outward agreement. (Jcampuzano2)

The Nature of Prediction and the Uncertainty of the Future

The discussion touches upon the difficulty of predicting the future, especially regarding technological advancements, and the motivations behind such predictions.

• The difficulty of making accurate predictions about the future is acknowledged. (koolba)
• A cynical view suggests that predicting the future is less about accuracy and more about "selling something to someone today." (Towaway69)
• Frustration is expressed with futurists who don't account for social and economic network effects, or those with overly narrow predictions. (th0ma5)
• The uncertainty surrounding future cash flows for AI is identified as a key element, though distinct from the foundation of a bubble itself. (crawshaw)
• The unpredictable trajectory of AI development is acknowledged, with rapid progress in areas previously thought impossible within a short timeframe. (atleastoptimal)