HN Distilled

Essential insights from Hacker News discussions

RustGPT: A pure-Rust transformer LLM built from scratch

Original Article

Hacker News Discussion

Here's a summary of the themes discussed in the Hacker News thread, presented in Markdown format with direct quotes:

Rust Dependency Management and the Cargo Ecosystem

A significant portion of the discussion revolves around Rust's dependency management system, Cargo, and its ease of use versus potential downsides. The original post highlighting a simple dependency tree for an LLM project in Rust sparked this.

  • Praise for Cargo's Simplicity and Robustness: Many users appreciate Cargo's straightforward approach to dependency management, especially when compared to other language ecosystems.

    • "As someone who has spent days wrestling with Python dependency hell just to get a model running, a simple cargo run feels like a dream." - untrimmed
    • "Cargo has less friction than pypi and npm. npm has less friction than pypi." - jokethrowaway
    • "Cargo is literally the reason why people coming to Rust from languages like C++ where the lack of standardized tooling is giant glaring bomb crater that poses burden on people every single time they need to do basic things (like for example version upgrades)." - dev_l1x_be
    • "I'm a “Python guy” in that I write Python professionally, but also am like you in that I’ve been extremely underwhelmed by Portry/Pipenv/etc. Python dependencies are still janky, but uv is a significant improvement over existing tools in both performance and ergonomics." - jhardy54
    • "I'm one of you. I'm about the highest tier of package manager nerd you'll find out there, but despite all that, I've been struggling to create/run/manage venvs out there for ages. Always afraid of installing a pip package or some piece of python-based software (that might muck up Python versions). ... uv has truely been a game changer. Try it out!" - hobofan
    • "As a Ruby guy: uv makes Python feel like it finally passed the year 2010." - tinco
    • "I am not a huge fan of Go, but if all the world's "serious" Python became Go, the average code quality would skyrocket, so I think I can agree to this proposal." - DiabloD3 (referring to the general sentiment about Python tooling before mentioning uv)
    • "I mean I would understand that comment in 2010, but in 2025 it's grossly ridiculous." - Galanwe (referring to dependency hell in Python)
    • "Yeah, because of a tool written in Rust, copying the Rust way of doing things for Python developers." - adastra22 (referring to uv)
    • "As a mainly Python guy (Data Engineering so new project for every ETL pipeline = a lot of projects) uv solved every problem I had before with pip, conda, miniconda, pipx etc." - OoooooooO
    • "uv has been amazing for me. It just works, and it works fast." - shepardrtc

  • Concerns about Over-Reliance and "Dependency Clutter": Conversely, some users express a concern that the ease of adding dependencies via Cargo can lead to excessive reliance on external libraries, potential security risks, and inflated build times.

    • "Rust projects can really go bananas on dependencies, partly because it's so easy to include them" - stevedonovan
    • "lowkey ppl who praise cargo seem to have no idea of the tradeoffs involved in dependency management" - taminka
    • "the difficulty of including a dependency should be proportional to the risk you're taking on, meaning it shouldn't be as difficult as it in, say, C where every other library is continually reinventing the same 5 utilities, but also not as easy as it is with npm or cargo, because you get insane dependency clutter, and all the related issues like security, build times, etc" - taminka
    • "how good a build system isn't equivalent of how easy it is include a dependency, while modern languages should have a consistent build system, but having a centralised package repository that anyone freely pull to/from, and having those dependencies freely take on any number of other dependencies is a bad way to handle dependencies" - taminka
    • "My thesis is that negative externalities of build systems are important and I don't know how to convince of importance of externalities someone whose value system is built specifically on ignoring externalities and only factoring in immediate convenience..." - taminka
    • "Dependency management should most definitely be one of the main criteria for evaluating how good a build system is. What's misguided is intentionally opting for worse dependency management in an attempt to solve a people problem, i.e. being careless about adding dependencies to your project in circumstances when you should be careful." - CodeMage
    • "A cargo build that warms up your CPU during winter while recompiling the whole internet is better?" - zoobab

  • Dependency Resolution Nuances: The discussion touched on how Rust handles multiple versions of dependencies and the role of semantic versioning.

    • "linking both rand-core 0.9.0 and rand-core 0.9.3 which the project could maybe avoid by just specifying 0.9 for its own dep on it" - cmrdporcupine
    • "It doesn't link two versions of rand-core. That's not even possible with rust (you can only link two semver-incompatible versions of the same crate). And dependency specifications in Rust don't work like that - unless you explicitly override it, all dependencies are semver constraints, so "0.9.0" will happily match "0.9.3"." - Diggsey (This correction clarifies a common misunderstanding.)
    • "If A depends on B and C - B and C can each bring their own versions of D, I thought?" - eximius (This highlights another common question about transitive dependencies.)

AI-Generated Code and Project Authenticity

A significant thread emerged concerning whether the LLM project discussed was generated by AI, based on its README, code style, and comments.

  • Indicators of AI Generation: Several users pointed to specific elements as strong indicators of AI involvement.

    • "It's AI generated" - koakuma-chan
    • "I know because this is how an AI generated project looks. Clearly AI generated README, "clean" code, the way files are named, etc." - koakuma-chan
    • "To me it looks like LLM generated README, but not necessarily the source (or at least not all of it)." - cmrdporcupine
    • "Or there's been a cleaning pass done over it." - cmrdporcupine
    • "I think pretty clearly the source is also at least partially generated. None the less, just a README like that already sends a strong signal to stop looking and not trust anything written there." - koakuma-chan
    • "The repeated Impls are strange." - GardenLetter27
    • "All the emojis in the README are a hint as well." - vlovich123
    • "The over-commenting?" - Revisional_Sin (questioning a potential AI trait)
    • "// Increased for better learning" and "// Use the constants from lib.rs" followed by comments about constants already being defined were pointed out as "sus." - ramon156

  • Skepticism and Nuance: While many suspected AI generation, others offered more nuanced perspectives or questioned the certainty.

    • "How do you know? The over-commenting?" - Revisional_Sin
    • "Commit messages look pretty human. But the emojis in readme and comments like "// Re-export key structs for easier access", "# Add any test-specific dependencies here if needed" are sus indeed." - magackame
    • "For the constants is it possible the author didn't know how? I remember in my first week of Rust I didn't understand how to name things properly, basically I was overthinking it." - tialaramex
    • "The code looks like a very junior or a non-dev wrote it tbh)." - Snuggly73

Perceptions of Rust Code Readability and Style

The readability and typical style of Rust code were also discussed, with opinions varying.

  • "Readable" Rust: Some found the code to be surprisingly readable, contrasting it with a perceived complexity in other Rust projects.

    • "Absolutely love how readable the entire project is" - Charon77
    • "Never knew Rust could be that readable." - yieldcrv
    • "jmaker: Not sure what you’re alluding to but that’s just ordinary Rust without performance or async IO concerns." - jmaker
    • "Most Rust code looks like this - only generic library code goes crazy with all the generics and lifetimes, due to the need to avoid unnecessary mallocs and also provide a flexible API to users. But most people aren't writing libraries." - GardenLetter27

  • Critique of Rust Style (Procedural/OO Focus): One user suggested the project's style was not idiomatic Rust, favoring more functional approaches.

    • "It is very procedural/object oriented. This is not considered good Rust practice. Iterators make it more functional, which is better, more succinct that is, and enums more algebraic. But it's totally fine for a thought experiment." - emporas

  • "Vibe-Coded" Rust and Quality: There was a brief exchange about the potential impact of "vibe-coded" Rust (writing code based on intuition or analogy) on the overall quality of Rust code.

    • "Do you think vibe coded rust will rot the quality of language code generally?" - ericdotlee
    • "For AI you definitely need to clean up and I think even targeted learning on some practices would be beneficiary ; for users ; it depends on the people, and I'd argue that vibe-coded rust can be better than just 'written-rust' IF the important details and mind of the user are actually focused on what is important ; Eg ; I could vibe-code a lock-free well architected s3 - focus on all the important details that would actually make it high perf - or write some stuff myself 10x slower - which means I will have 10 x less time to work on the important stuff. However what you asked is wether the vibe coded rust will rot the quality of language ; this is a more difficult to answer to, but I don't think that people who are uninterested in the technics are going to go for rust anyway - from the signals I feedback people are actually not really liking it - they find it too difficult for some reason and prefer to blanket with stuff like C# or python. Can't explain why." - 6r17

The Role of Python in AI and ML Development

A significant portion of the conversation turned into a debate about Python's dominance in AI/ML and the potential for other languages, particularly Rust, to replace it.

  • Python's Dominance and Ecosystem: Many acknowledged Python's entrenched position in AI/ML due to its extensive libraries, community support, and researcher accessibility.

    • "There's not really another game in town if you want to do fast ML development :/" - airza
    • "But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling. I think ML will fade away from Python eventually but right now it's still everywhere." - nkozyra
    • "The disease is the cargo cult addiction (which Rust is full of) to micro libraries, not the language that carries 90% of all peer reviewed papers, datasets, and models published in the last decade. every major breakthrough, from AlphaFold to Stable Diffusion, ships with a Python reference implementation because that is the language researchers can read, reproduce, and extend, remove Python and you erase the accumulated, executable knowledge of an entire discipline overnight, enforcing Rust would sabotage the field more than anything" - WhereIsTheTruth
    • "People saying 'oh those Python libraries are just C/C++ libraries with Python API, every language can have them' have one problem - no other language has them (with such extensive documentation, tutorials etc.)." - wolvsechoes

  • Arguments for Rust/Other Languages in AI/ML: Counterarguments focused on performance, safety, and maintainability benefits that Rust (or potentially other languages) could offer.

    • "The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default." - DiabloD3
    • "Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based." - DiabloD3
    • "The sole exclusion I've seen to my social circle is those working at companies that don't directly do ML, but provide drivers/hardware/supporting software to ML people in academia, and have to try to fix their cursed shit for them." - DiabloD3
    • "Also, fwiw, there is no reason why Triton is Python. I dislike Triton for a lot of reasons, but its just a matmul kernel DSL, there is nothing inherent in it that has to be, or benefits from, being Python.... it takes DSL in, outputs shader text out, then has the vendor's API run it (ie, CUDA, ROCm, etc). It, too, would benefit from becoming Rust." - DiabloD3
    • "If someone wrote a Triton impl that is all Rust instead, that would do a lot of the heavy lifting on switching... most of their hard code is in Triton DSL, not in Python, the Python is all boring code that calls Triton funcs. That changes the argument on cost for a lot of people, but sadly not all." - DiabloD3
    • "Very interesting to already see rust based inference frameworks as well." - ericdotlee

  • Critique of Python's Performance: Some participants pushed back on the idea that Python is "performant" for AI, arguing that heavy lifting is done by C/C++ libraries.

    • "'Python' is perfectly performant for AI and this demonstrates a deep lack of understanding. Virtually every library in python used for AI delegates to lower-level code written in C++." - leoh
    • "This is incredibly cool, but I wonder when more of the AI ecosystem will move past python tooling into something more... performant?" - ericdotlee

The "From Scratch" LLM Project and its Characteristics

The original project being an LLM built "from scratch" in Rust was met with mixed reactions and interpretations.

  • Appreciation for "From Scratch" Effort: Some users appreciated the effort and educational value of building an LLM from the ground up.

    • "Extra credit for "from scratch" in the title." - sloppytoppy
    • "this is a very cool exercise. I did the same with Zig and MLX a while back, so I can get a nice foundation, but since then as I got hooked and kept adding stuff to it, switched to Pytorch/Transformers." - Snuggly73
    • "Very cool project, always nice to see deep learning built from scratch in Rust without heavy frameworks." - capestart

  • Technical Discussions on Implementation: Users discussed specific technical aspects of the LLM, such as transformer blocks, numerical gradients, and specific implementation choices.

    • "Congrats - there is a very small problem with the LLM - its reusing transformer blocks and you want to use different instances of them." - Snuggly73
    • "You know that tensorflow was written in C++ and the Python API bolted on top?" - adastra22 (in response to a question about C equivalents)
    • "great job! which model does it implement? gpt-2?" - amoskvin
    • "Very nice! Next thing to add would be numerical gradient testing." - yobbo
    • "Is that where you approximate a partial derivative as a difference in loss over a small difference in a single parameter's value? Seems like a great way to verify results, but it has the same downsides as forward mode automatic differentiation since it works in a pretty similar fashion." - tripplyons

  • Performance of Rust LLMs: The speed of Rust-based LLM implementations was also a topic.

    • "That time to first token is impressive, it seems like it responds immediately" - bionhoward

General Software Development Philosophy and Tooling

Beyond the immediate Rust/Python debate, broader philosophical points about software development, tooling, and community practices were raised.

  • The Value of Simplicity and Education: Some emphasized the importance of teaching good practices rather than relying solely on tooling.

    • "The solution has always been education, teaching juniors to value simplicity, portability and maintainability" - WhereIsTheTruth

  • Critique of Language "Cargo Cults": A criticism was leveled against what was perceived as a "cargo cult" mentality within some language communities.

    • "The disease is the cargo cult addiction (which Rust is full of) to micro libraries..." - WhereIsTheTruth

  • Maturity of Ecosystems: The maturity and breadth of documentation and tutorials were highlighted as key differentiating factors between language ecosystems.

    • "Python community is constantly creating new, great, highly usable packages that become de facto industry standards, and maintain old ones for years, creating tutorials, trainings and docs. Commercial vendors ship Python APIs to their proprietary solutions. Whereas Rust community is going through forums and social media telling them that they should use Rust instead, or that they "cheated" because those libraries are really C/C++ libraries (and BTW those should be done in Rust as well, because safety)." - wolvsechoes

  • The "Right Tool for the Job" Debate: The perennial discussion about choosing the appropriate language or tool for a given task surfaced.

    • "Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative." - trklausss
    • "Rust is not a viable replacement for Python except in a few domains." - wavemode

  • The "Abuse" of Semver: A user briefly touched upon the potential for misuse of semantic versioning.

    • "abusing semver to break without breaking semver is unfortunately also common." - cmrdporcupine (While this comment is within the context of dependency management, it touches on a broader tooling concern.)