Here's a breakdown of the key themes discussed in the Hacker News thread, with supporting quotes:
The Relationship Between Weight Functions and Orthogonal Polynomials
The discussion kicks off with the connection between weight functions and orthogonal polynomials, highlighting how the choice of weight function influences the integration domain and the properties of the resulting polynomials.
- "What is also worth pointing out and which was somewhat glanced over is the close connection between the weight function and the polynomials. For different weight functions you get different classes of orthogonal polynomials." - constantcrying
- "So the choice of weight function also influences the choice of integration domain." - constantcrying
- "Like, is it possible to infer that Chebyshev polynomials would be useful in approximation theory using only the fact that they're orthogonal wrt the Wigner semicircle (U_n) or arcsine (T_n) distribution?" - creata
Orthogonal Projection vs. Chebyshev Interpolation: A Point of Contention
A significant portion of the discussion revolves around whether Chebyshev interpolation can be considered an orthogonal projection and the practical differences between them. sfpotter argues that while orthogonality is crucial, Chebyshev interpolation isn't a direct L2 projection, while constantcrying initially contends they are essentially the same thing. The core disagreement seems to be on whether the "computation" method changes the underlying principle.
- "The fact of their orthogonality is crucial, but when you work with Chebyshev polynomials, it is very unlikely you are doing an orthogonal (L2) projection! Instead, you would normally use Chebyshev interpolation" - sfpotter
- "What you do not understand is that this is the same thing. The distinction you describe does not exist, these are the same things, just different perspectives. That they are the same easily follows from the uniqueness of polynomials, which are fully determined by their interpolation points. These aren't distinct ideas, there is a greater principle behind them and that you are using some other algorithm to compute the Approximation does not matter at all." - constantcrying
- "In general, f_N and p_N are not the same polynomial...In practice, people do not compute the coefficients of f_N, they compute the coefficients of p_N. Nevertheless, f_N and p_N are essentially as good as each other when it comes to approximation." - sfpotter
- "If you want to compute the L2 projection of a function onto the orthogonal subspace of degree N Chebyshev polynomials, you would need to evaluate a rather expensive integral to compute the coefficients. It's expensive because it requires the use of adaptive integration... many function evaluations per coefficient! Bad!" - sfpotter
- "And, again, since the the polynomial so constructed is not the same polynomial as the one obtained via L2 projection mentioned in paragraph 3 above, this interpolation procedure cannot be regarded as a projection! I guess you could call it an "approximate projection"" - sfpotter
Practicality and Utility of Chebyshev Polynomials in Approximation
The discussion highlights the practical reasons for using Chebyshev polynomials in approximation, especially their minimax properties and the existence of fast transforms. Legendre polynomials are presented as an alternative for L2 approximation but are less convenient for numerical computations.
- "Chebyshev polynomials are useful in approximation theory because they're the minimax polynomials." - sfpotter
- "Normally you'd use Legendre polynomials, since they have w = 1, but they are a much less convenient basis than Chebyshev for numerics." - sfpotter
- "This is why Chebyshev polynomials are so useful in practice for approximation, and why e.g. Legendre polynomials are much less useful (they do not have a convenient fast transform)." - sfpotter
The Semicircle Distribution and Chebyshev Polynomials
creata raises an interesting question about the deeper, perhaps abstract, reasons for the semicircle distribution being connected to Chebyshev polynomials, drawing an analogy to the central limit theorem and the normal distribution.
- "But I guess what I was asking was: is there some kind of abstract argument why the semicircle distribution would be appropriate in this context?" - creata
- "I guess the semicircle might more-or-less be the only way to get something where interpolation uses the DFT (by projecting points evenly spaced on the complex unit circle onto [-1, 1]), but I dunno, that motivation feels too many steps removed." - creata
- "If there is, I'm not aware of it. Orthogonal polynomials come up in random matrix theory. Maybe there's something there?" - sfpotter
Book Recommendations for Numerical Analysis
Several participants ask for and recommend books on numerical analysis covering topics like minimax approximation and quadrature.
- "Could you guys recommend an easy book on this topic?" - efavdb
- "I'd suggest: Trefethen, Lloyd N., Approximation theory and approximation practice (Extended edition), SIAM, Philadelphia, PA (2020), ISBN 978-1-611975-93-2." - 4ce0b361
- "I would check out "An Introduction to Numerical Analysis" by Suli and Mayers or "Approximation Theory and Approximation Practice" by Trefethen." - sfpotter
Gaussian Integrals and Related Concepts
A tangent arises regarding the Gaussian integral and its significance in mathematics and physics, particularly in quantum field theory.
- "I thought when I first saw the title that it was going to be about the Gaussian integral[1] which has to be one of the coolest results in all of maths...That is, the integral from - to + infinity of e^(-x^2) dx = sqrt(pi)." - seanhunter
- "Gaussian integrals are also pretty much the basis of quantum field theory in the path integral formalism..." - niklasbuschmann
- "It's the gateway drug to Laplace's method (Laplace approximation), mean field theory, perturbation theory, ... QFT." - sream
- "There is a relationship here, in the case of Gauß-Hermite Integration, where the weight function is exactly e^(-x^2) the weights have to add up sqrt(pi), because the integral is exact for the constant 1 polynomial." - constantcrying
Critique of Figure 1 in the Original Post
Several users criticize the visualization in the original post (Fig. 1), suggesting alternative ways to present the data more effectively.
- "Fig 1 could use a rethink. It uses log scale, but the dynamic range of the y-axis is tiny, so the log transform isn't doing anything." - mturmon
- "It would be better shown as a table with 3 numbers. Or, maybe two columns, one for integral value and one for error, as you suggest." - mturmon
- "Anyhow for anyone interested the values for those 3 points are 2.0000 (exact), 1.9671 (trapezoid), and 1.9998 (gaussian). The relatives errors are 1.6% vs. 0.01%." - rd11235
Clarification Regarding Gaussian Integration
A user points out that Gaussian integration evaluates the integrals of polynomials of degree 2n+1 exactly, rather than just estimating them.
- "What I'm getting at is that the Gaussian integration is not estimating the integrals of polynomials of degree 2n+1, but it's evaluating them exactly." - tim-kt
The author of the blog post then acknowledges this clarification and updates the blog post accordingly.