The Hacker News discussion revolves around several key themes related to machine learning, dimensionality, and the nature of AI research.
The Misapplication or Redundancy of ReLU in Loss Functions
A central point of confusion and debate stems from the initial question about the purpose of a ReLU (Rectified Linear Unit) function applied within a loss function, given that loss function inputs are typically non-negative.
- Initial Question and Misunderstanding: The original poster, "bigdict," questioned the utility of ReLU in a loss function, stating, "What's the point of the relu in the loss function? Its inputs are nonnegative anyway."
- Abs Function as an Alternative: "GolDDranks" echoed this sentiment, suggesting that “abs has that property already!” indicating that the absolute value function might serve a similar purpose without the perceived redundancy of ReLU.
- ReLU's Effect on Loss: "fancyfredbot" clarified the behavior of ReLU, explaining that it would create a "flat below zero" loss without the absolute value, and that adding
absfirst would render ReLU's effect moot: "RELU would have made it flat below zero ( _/ not \/). Adding the abs first just makes RELU do nothing." - Joking about Redundancy and AI Safety: "fancyfredbot" humorously suggested the redundancy might be a "belt and braces approach to AI safety," adding, "Better safe than sorry with these troublesome negative numbers!"
- Skepticism about Understanding: "naniwaduni" expressed skepticism about the authors' understanding, stating, "Well, I guess it's helping to distinguish authors who are doing arithmetic they understand from ones who are copying received incantations around..."
- Hypothetical Cosmic Ray Defense: "meindnoch" offered a humorous, albeit highly improbable, technical justification: "Sometimes a cosmic ray might hit the sign bit of the register and flip it to a negative value. So it is useful to pass it through a rectifier to ensure it's never negative, even in this rare case."
- Joking about Idempotent Functions: "lblume" extended the humor by suggesting a general principle: "Indeed, we should call all idempotent functions twice just in case the first incantation fails to succeed." This was followed by a serious dismissal of the cosmic ray idea as a practical resilience method.
- Modern LLM Architectures: "andy_ppp" brought the discussion back to current practices, noting, "In reality it’s probably not a RELU modern LLMs use GeLU or something more advanced."
The Nature and Application of Orthogonality in Vector Spaces for Concepts
A significant portion of the discussion centers on how concepts are represented in high-dimensional vector spaces, the role of orthogonality, and the implications of the Johnson-Lindenstrauss (JL) lemma.
- Orthogonality in Binary Vectors: "js8" introduced the idea of orthogonality in binary vectors, stating, "You can also imagine a similar thing on binary vectors. There two vectors are 'orthogonal' if they share no bits that are set to one." This was presented as a way to encode many concepts efficiently.
- Misunderstandings and Clarifications on Orthogonality: The initial definition of orthogonality for binary vectors sparked several clarifications and corrections.
- "phreeza" questioned the definition, suggesting only one vector would be orthogonal if the definition was strict: "If they are only orthogonal if they share no bits that are set to one, only one vector, the complement, will be orthogonal, no?"
- "yznovyak" and "prerok" engaged in a debate about the number of mutually orthogonal vectors possible in a given dimension, with "prerok" stating, "by definition, an n-dimentional space can have at most n vectors which are all orthogonal to one another."
- "js8" refined their point, clarifying that the initial remark was not about dot product but about shared set bits, and introduced the concept of "quasi-orthogonality" (small dot product/overlap): "It's not correct to call them orthogonal because I don't think the definition is a dot product. But that aside, yes, orthogonal basis can only have as much elements as dimensions. The article also mentions that, and then introduces 'quasi-orthogonality,' which means dot product is not zero but very small."
- "asplake" offered a definition of orthogonality based on shared zero bits: "By the original definition, they can share bits that are set to zero and still be orthogonal."
- "js8" provided an example: "For example, 1010 and 0101 are orthogonal, but 1010 and 0011 are not (share the 3rd bit)."
- "henearkr" pointed out that the definition of orthogonality for binary vectors might involve XOR and finite fields: "Your definition of orthogonal is incorrect, in this case. In the case of binary vectors, don't forget you are working with the finite field of two elements {0, 1}, and use XOR."
- Johnson-Lindenstrauss Lemma and its Implications: The discussion frequently referenced the JL lemma and its relevance to AI.
- "dwohnitmok" linked these intuitions to the JL lemma and its role in Sparse Autoencoders (SAEs) for AI safety, referencing an Anthropic paper.
- "niemandhier" provided a detailed explanation of the JL lemma, emphasizing that it preserves pairwise distances for a finite set of points but doesn't necessarily preserve the underlying topology of continuous manifolds. They warned about "folding" when embedding into lower dimensions without sufficient capacity.
- "sdl" analogized this to map projection, which "niemandhier" countered was "Worse" due to topological violations like self-intersecting surfaces.
- "airstrike" and "rossant" engaged in a meta-discussion about the "ChatGPT vibe" of the article, with "rossant" identifying overuse of certain phrases.
- "jerf" argued that AI-generated prose signals a lack of care for the content and potential unreliability of technical details.
- "cgadski" critically analyzed the article's use of the JL lemma, questioning graph interpretations and suggesting the article might be LLM-generated, stating, "seems to be written in large part by a language model… and has a bunch of basic inconsistencies that make me doubt its conclusions."
- "jvanderbot" defended the critique of LLM generation as a way to indicate errors, while "Blackthorn" saw it as a signal that the content shouldn't be read.
- "cgadski" shared resources on high-dimensional geometry and information theory, highlighting the concept of "spherical codes."
- "stogot" and "prerok" clarified the meaning of "dimensions" in this context.
- "mallowdram" presented a contrarian view that LLMs do not aim for orthogonality, that concepts are not well-defined, and that LLMs are designed for Western concepts, lacking interdependence and holistic understanding. This was met with counterarguments about context and probabilistic relationships.
- "WithinReason" countered that non-linearities in neural networks negate the need for strict orthogonality.
- "djoldman" highlighted the importance of representation learning and how abstract concepts can be combined in embedding spaces.
- "gibsonf1" argued that there are not billions of concepts, but rather a limited number of concepts with many instances.
- "jgbuddy" used an analogy to computer representations of numbers to explain how dimensions increase representational capacity.
- "twiggy" suggested that LLMs might not be space-efficient in their current packing of information.
- "jvanderbot" offered a more mathematical link to JL and "exponentially many vectors" with small inner products.
Critique of AI Research Practices, Transparency, and Peer Review
A significant portion of the discussion critiques current practices in AI research, particularly concerning transparency, the quality of widely circulated blog posts, and the perceived shortcomings of the peer review process.
- The Anthropic Paper and its Publication: The Anthropic paper cited by "dwohnitmok" became a focal point for this critique.
- "emil-lp" questioned if the paper was peer-reviewed, as it was published on
transformer-circuits.pub. - "yorwba" defended the paper, stating that its high citation count indicated a form of "post-publication peer review."
- "emil-lp" countered that citation is not the same as peer review and that actual reviewing is time-intensive.
- "emil-lp" questioned if the paper was peer-reviewed, as it was published on
- Sloppy Writing and its Impact: "adroniser" strongly criticized the "circuits thread blog posts" for their "hand wavy language," "vague analogies," and lack of concrete definitions, arguing this sloppiness damages the field's health and encourages a similar style among new researchers. They questioned if such posts would be publishable in mainstream conferences.
- Peer Review Ineffectiveness: Several users expressed cynicism about the effectiveness of traditional peer review.
- "PeterStuer" agreed that theoretical ideals of peer review might not hold in practice.
- "sdenton4" described receiving "useless" or "actively promoting scientific dishonesty" reviews from tired PhD students, contrasting this with the better quality (though slower) reviews from specialized journals and important work that bypasses the conference circuit.
- "adroniser" reiterated that even if reviewers don't improve work, submission to peer review would force authors to be more precise.
- What Constitutes Peer Review: A definitional debate arose about what "peer review" truly means.
- "LolWolf" argued that peer review is simply "review by your peers" and involves reproduction and critique, not just formal publication.
- "emil-lp" maintained a distinction between a quick glance for citation and a thorough review.
- Reproducibility and Crackpottery: "mutkach" raised concerns about the reproducibility of research in the field, particularly for complex models and synthetic data, warning against "crackpottery" and unsubstantiated claims (like LLMs being quantum).
- Link Review Farms: "golem14" briefly cautioned against assuming all citations are organic, suggesting link review farms as a possibility.
- Community and Validation: "yorwba" defended the citation count as a form of validation. "l1n" pointed to replication and reviewer comments on the linked page as evidence of review.
The "AI Writing" Phenomenon and its Societal Impact
A substantial portion of the thread devolved into a critique of writing styles that are perceived as having been influenced or generated by LLMs, highlighting potential downsides of AI-assisted prose.
- Distracting AI Tone: "rossant" found the "ChatGPT vibe" of the article distracting, mentioning it felt like generic AI prose with phrases like "delving," "fascinating," and emphasis on collaboration.
- AI Writing Indicators: Users identified various characteristics of AI-generated text:
- Overuse of adjectives ("superlative adjectives and too many adjectives in general").
- The "rule of three" (e.g., "adjective, adjective, and adjective" or "short phrase, short phrase, and short phrase").
- Generic, often positive framing of discoveries ("remarkable," "fascinating," "profound").
- Certain phrasing patterns ("When I shared these findings with Grant, his response exemplified the collaborative spirit...").
- Social Signal and Trust: "jerf" framed the preference for AI-like writing as a "social signal of not really caring about the content," suggesting it also implies the output might be unreliable for deep technical understanding due to subtle errors.
- Impact on Prose and Uniformity: "rossant" elaborated on finding this "tone" distracting as a non-native speaker, lamenting the "uniformity of style" from reliance on AI prose improvement.
- Examples of AI Writing: Links to Wikipedia's guidelines on AI writing and specific examples of AI-generated text were shared.
- Spiteful Dash Usage: "airstrike" humorously noted how using en and em dashes correctly now might lead to being misidentified as an AI, prompting them to use them "out of spite."
- "GPT'y" Feeling: "rebolek" pinpointed "tangential" and "effectively" as phrases that triggered the "GPT feeling."