HN Distilled

Essential insights from Hacker News discussions

Show HN: Haystack – Review pull requests like you wrote them yourself

Original Article

Hacker News Discussion

This Hacker News discussion primarily revolves around the utility and perception of AI-generated content, particularly in the context of software development workflows like pull requests (PRs) and meeting summaries.

Skepticism Towards AI-Generated Content Consumption

A significant theme is a general distrust or lack of patience for AI-generated content when it's intended for human consumption. Users express a feeling of "time wasted" when reading AI-generated summaries, code, or other text, especially if the quality is not guaranteed.

  • "As I work more with AI, I’ve came to the conclusion that I have no patience to read AI-generated content, whether the content is right or wrong. I just feel like it’s time wasted." (tkiolp4)
  • "I think that's because I know not all AI generated stuff is equally created and that some people are terrible at prompting/or don't even proofread the stuff that's outputted, so I have this internal barometer that screams 'you're likely wasting your time reading this' and so I just learned to avoid it entirely." (volkk)
  • "I don't want you to send me a AI-generated summary of anything, but if I initiated it looking for answers, then it's much more helpful." (shortcord)
  • "If they don't give a good summary, I ask them to write one. Exactly; if people can't be bothered to describe (and justify) their work, or if they outsource it to AI that creates something overly wordy and possibly wrong, why should I be bothered to review it?" (Cthulhu_)

AI as a Tool for Augmentation, Not Replacement

Conversely, there's a strong sentiment that AI is highly valuable when used as a tool to assist human analysis and workflow, rather than replacing human effort. This is particularly relevant in the context of understanding complex codebases or distilling information for specific lookup needs.

  • "I like AI on the producing side. Not so much on the consuming side." (tkiolp4)
  • "Using an LLM on the backend to discover meaningful connections in the codebase may sometimes be the right call but the output of that analysis should be some simple visual indication of control flow or dependency like you mention. At a first look the output in the editor looks more like an expansion rather than a distillation." (ray__)
  • "AI can generate far more text than we can process, and text treatises on what an AI was prompted to say is pretty useless. But generating text not with the purpose of presenting it to the user but as a cold store of information that can be paired with good retrieval can be pretty useful." (mediaman)
  • "I would really want to use this, maybe about once a week, for major PRs. I find it absurd that we all get AI help writing large features but very little help when doing the approx same job in reviewing that code." (kanodiaashu)

Utility of AI for Meeting Summaries and Information Retrieval

While some expressed skepticism about AI-generated content, a notable portion of the discussion highlighted the practical utility of AI for summarizing meetings and creating searchable knowledge bases, especially in large organizations or for specific retrieval purposes.

  • "For me, AI meeting summaries are pretty useful. The only way I see they're not useful for you is that you're disciplined enough to write down a plan based on the meeting subject." (gobdovan)
  • "I'm in a large company, sometimes we have long incident meetings running for hours and new idiots join in the middle 'what happened?'. Now at least we can get summaries of the past hours during the meeting to catch up without bothering everyone !" (xwolfi)
  • "Meeting notes are useful in two ways, for me: I'm reviewing the last meeting of a regular meeting cadence to see what we need to discuss. I put it in a lookup (vector store, whatever) so I can do things like 'what was the thing customer xyz said they needed to integrate against'." (mediaman)
  • "For me, AI meeting summaries are pretty useful. The only way I see they're not useful for you is that you're disciplined enough to write down a plan based on the meeting subject." (gobdovan)
  • "But to be blunt / irreverent, it's the same with Git commit messages or technical documentation; nobody reads them unless they need them, and only the bits that are important to them at that point in time." (Cthulhu_)

AI in Code Reviews and PR Organization

The core of the discussion centers on the application of AI to improve the pull request review process. This includes AI assisting in understanding the structure of a PR, identifying dependencies, and facilitating better organization of code changes. There's also a debate about whether using AI to manage PR complexity indicates underlying organizational issues.

  • "I agree that there is a real need here and a potentially solid value proposition (which is not the case with a lot of vscode-fork+LLM-based starups) but the whole point should be to combat the verbosity and featurelessness of LLM-generated code and text." (ray__)
  • "Non-trivial PRs need two passes: first grok the entrypoints and touched files to grasp the conceptual change and review order, then dive into each block of changes with context." (irrationalfab)
  • "This nails a real problem. Non-trivial PRs need two passes: first grok the entrypoints and touched files to grasp the conceptual change and review order, then dive into each block of changes with context." (irrationalfab)
  • "This is where I feel like we've solved a third-order problem. If you're sorting all PRs into those two buckets then you should probably take a step back and redefine what a PR is for your organization, as both 1 and 2 make the assumption that the PR is too big to review in a single sit down or that the author didn't put in enough effort to craft their PR." (Ethee)
  • "I think organization helps the 1st case and obviates the need for the author to spend so much time crafting the PR in the 2nd case (and eliminates messy updates that need to be carefully slotted in)." (akshaysg)
  • "I love the AI of adding context and logical ordering to PRs! Really cool concept" (koinedad)
  • "If AI writes all of the code, we will need to max out humans’ ability to do architecture and systems design." (ripped_britches)

Technical Feedback and Concerns (UX, Privacy, Pricing)

Users also provided direct feedback on the product being discussed, touching on technical aspects like performance, user experience, privacy policies, and pricing models.

  • "There is QUITE a delay doing the analysis, which is reasonable, so I assume as a productionized (non-demo) release this will be async?" (mclanett)
  • "Failed to load resource: net::ERR_BLOCKED_BY_CLIENT" (mclanett)
  • "Any ideas what pricing will look like? What is your privacy policy around AI? Any plans for a locally-runnable version of this?" (sea-gold)
  • "1. Pricing would be $20 per person and we'd spin up an analysis for every PR you create/are assigned to review 2. We don't train or retain anything related to your codebase. We do send all the diffs to Open AI and/or Anthropic (and we have agentic grepping so it can see other parts of the codebase as well)" (akshaysg)
  • "Missing browser navigation is the problem. The virtual back buttons you put in the top left are working as I'd expect browser nav to do. I keep trying to go back, it would feel so natural." (gobdovan)
  • "It could be just fatigue over the technology itself, but “like you wrote them yourself” sounds too much like a dog whistle for a user base of programmers working with AI generated PRs." (tolerance)