How Ruby executes JIT code

Here's a summary of the themes expressed in the Hacker News discussion:

Ruby's Dynamic Nature and JIT Friendliness

A central theme is the impact of Ruby's dynamic nature on its ability to be efficiently compiled by Just-In-Time (JIT) compilers. Ruby's flexibility, particularly its ability to dynamically add methods and alter classes at runtime, poses significant challenges for JIT compilers that rely on making assumptions about code.

• "Any web request in my rails app could theoretically add new functions to any other class, in ways that would just not work in most other languages. This wreaks havoc on JIT compilers, because they constantly need to check if the assumptions they made when compiling certain bits of code still hold or if the entire world has shifted out from underneath them." - WJW
• "It's not being able to add new methods or new member variables after initialization would help." - jbritton
• "One reason is that the JIT needs info on the types passed to the method, which are only available after a few calls. Per the article, a+b could refer to any type of a and b. The only way to know what was used is profiling." - zamalek

However, there's a counterpoint that while these dynamic features can disrupt JIT, they don't necessarily do so frequently enough to prevent all optimization.

• "While it's true that CAN happen, it doesn't mean it happens often enough to disrupt JIT compiling the code. It does mean there is an invalidation when code paths are modified." - jemmyw

The Role of Investment and Maturity in VM Performance

A significant factor identified for the performance gap between Ruby VMs (specifically CRuby) and the JVM is the vast difference in investment and development resources. The JVM has benefited from decades of dedicated, well-funded engineering from multiple major corporations, leading to sophisticated JIT compilers, garbage collectors, and tooling.

• "I think the real answer why the CRuby VM lags behind the JVM is more pedestrian: there's been orders of magnitude more investment made into the JVM than any Ruby VM. Sun and then Oracle have hired world class VM developers and researchers to work on multiple JIT compilers, multiple GC implementations, fast tooling, a standard memory model, world-class concurrency library, and so on. OpenJDK sees heavy investment by many others, too (RedHat probably leading the charge). And then you have other vendors advancing the state of the art with alternative VM implementations (Azul, IBM, Amazon, etc.)." - nirvdrum
• "Ruby as a language has seen loads of research done on compilation and alternative VMs [2], but the CRuby VM itself hasn't seen anything close to OpenJDK's investment. That's not to say there isn't excellent work being done to improve the CRuby VM. But, production VMs are large, complex projects and not the sort of thing easily moved by a volunteer working nights and weekends. More investment by companies using Ruby would help close the gap." - nirvdrum

Alternative Ruby Implementations on the JVM

The discussion frequently brings up JRuby and TruffleRuby as solutions that allow Ruby code to run on the JVM, leveraging its performance characteristics. This highlights that "Ruby" as a language can achieve JVM-level performance, albeit through a different execution engine.

• "JRuby is a thing and it runs on the JVM as more than a simple interpreter." - jemmyw
• "So you could actually say Ruby can already run as fast as the JVM because it runs on it, what it cannot do is run as fast as a static language because, at the very least, it has to deal with additional checks that things haven't changed." - jemmyw
• "Are you aware of jruby? https://www.jruby.org/ It is ruby running on a Java Virtual Machine." - bashkiddie
• "You could use Ruby on top of JVM such as TruffleRuby and JRuby." - ksec
• (Referencing paper) "TruffleRuby applies many of these same techniques to Ruby to great effect." - nirvdrum

Comparison with Other Dynamic Languages and JIT Challenges

The conversation extends to comparing Ruby's situation with other dynamic languages like JavaScript (V8) and even C (via dynamic loading), as well as the JVM's own strategies for handling dynamism. This provides context for the challenges faced by JIT compilers in dynamic environments.

• "Even V8 isn't as fast as JVM :) I think only LuaJIT is comparable in some ways." - Tiberium
• "Java Spring applications pretty much do the same thing (reflections, proxy beans, aop, JSON/Date serializers/deserializers, ORMs, etc are all compiled and changed dynamically upon route hits)." - rdsubhas
• "These things are theoretically possible (and happen) in any JIT language (node, java). Only statically compiled languages seems to be immune to this." - rdsubhas
• "While probably less common in Java, the language supports dynamic class loading, reflection, runtime bytecode generation, invokedynamic, and other features that can break static assumptions. The JVM does need to deal with all of this and does so with speculative optimization and deoptimization." - nirvdrum
• "Maybe like Racket. Simply not being able to add new methods or new member variables after initialization would help." - jbritton

Potential for Advanced JIT Techniques in Ruby

There's speculation and acknowledgment of advanced JIT techniques, such as tiered compilation and background compilation, which could potentially improve Ruby's performance, especially for long-running applications.

• "I wonder if there's ever a point at which you could run a thread at very low "niceness" that just slowly keeps compiling more and more methods into native code as the program goes on. Surely this would be worth it for long lived server programs. You could even keep it around to periodically recompile methods if runtime information shows that some paths are hotter than others." - WJW
• "This technique is used by some existing VMs. .NET does this with a background thread and calls it 'tiered compilation', where methods are queued for recompilation at a higher (more expensive/thorough) optimization level in the background. It's pretty helpful." - kg
• "I believe most browser JS runtimes do it too." - kg