Rendered at 11:43:35 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
MaxBarraclough 3 hours ago [-]
Reminds me of the 2024 blog post Look ma, I wrote a new JIT compiler for PostgreSQL [0]. Both articles lament that Postgres's LLVM-based JIT [1] takes a while to generate code.
> The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile.
That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM.
The problem with the approach is that it's not real JIT-compilation, it's just assembly templates with basic substitutions.
By not using LLVM, you're missing all the optimizations it does.
the-lazy-guy 21 seconds ago [-]
This is absolutely a JIT-compiler. It compiles code into machine code. This is a surprisingly efficient way to get noticeable speedup relative to interpretation. Also it is much safer than proper optimising compiler. Say ebpf jit-compiler functions very similarly, because it is fast and _secure_ way to jit. (well, there's a bit of cheating because before emitting bpf bytecode it goes through gcc/clang pipeline).
LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.
mort96 17 minutes ago [-]
A non-optimizing compiler is a real compiler.
IshKebab 7 minutes ago [-]
This absolutely is real JIT compilation. Copy and patch is a very well known JIT compilation technique.
glum64 4 hours ago [-]
Uhm, Common Lisp, where JIT is not only available but is also manageable: the programmer can decide what deserves to be compiled and what does not.
Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...).
clbrmbr 1 hours ago [-]
is an xtensa lx7 (esp32-s3) target available that does not use llvm?
fweimer 1 hours ago [-]
Not in typical builds of SBCL: all code is compiled before evaluation.
ligarota 38 minutes ago [-]
Tcc be like
hamilyon2 2 hours ago [-]
It uses copy-and-patch compilation to archive that
glenjamin 4 hours ago [-]
pgrust sounds very interesting, but with the deep changes there’s no viable path to upstream it - is the end goal to be robust enough that it’ll get wide adoption?
FiberBundle 2 hours ago [-]
Is it really interesting though? It's essentially just vibe-coded by people who are unqualified for this kind of work. One of the authors claimed that what qualified them was having worked on a large-scale postgres cluster; they never actually worked on databases or compilers.
varjag 2 hours ago [-]
There’s been a meme circulating about how AI doesn’t help because “code was never the hard part.” I think that’s true in some domains, but in others, writing the code absolutely was the hard part. JIT compilers are a great example of that.
asdfsa32 2 hours ago [-]
Anyone who thinks AI is good with writing code that is hard to write for the operator, not due to lack of basic software engineering know how but complexity of the domain, either has access to models beyond what is available to the public or is completely lost.
I believe this because every time I use AI for domains that I consider myself above competent, if it is anything beyond UI components or a simple CRUD endpoints, I cringe at the quality of what it generates.
This has made me to be extremely cautious of starting working in a new domain with AI if I want anything beyond throw away quick hacks or junk, shy of quick bug fixes perhaps.
mgaunard 1 hours ago [-]
Fable (and even Opus if kept tightly under reigns) does generate high-quality code even for highly complex tasks.
It generally performs better if the tasks are broken done into small manageable pieces, and the person is actually reviewing and calling out problems, which usually requires the person to be a competent engineer in the problem domain to begin with.
But yes, I have personally used it to build what the OP calls a JIT. I would usually write that by hand and it would take me one week. The AI does it in an hour.
asdfsa32 48 minutes ago [-]
You can't make assertions about "quality" of code that was generated under an hour while it would have taken a human 40 hours, unless you put substantial amount of work into reviewing it.
I used Fable on a Zephyr project with time sensitive code for LR-WPAN and it broke everything. Literally made the code worst to the point that the devices stopped connecting.
Mawr 6 minutes ago [-]
Your second paragraph invalidates your first.
If I need to be a domain expert anyway, the value of the tool goes down by orders of magnitude. Same if I need to first break the task down into pieces and keep reviewing all the output. That sounds to me like >80% of the work I'd need to do anyway.
Personally, I've found the greatest value in asking for simple, easy to verify tasks, like wiring up APIs, generating boilerplate, etc. Anything remotely complex and the LLM cakes its pants.
grebc 1 hours ago [-]
Three quarters of my CS class at university could barely code and/or understand code.
That’s not a joke. A lot went on to be programmers professionally. And judging by the quality of closed & open source code I witness daily those figures from university accurately depict people’s capabilities.
Now that said, if you can’t really code then using AI will be a godsend to said individuals.
mgaunard 1 hours ago [-]
I'd say that is the problem we're observing. A lot of decent code is being written with weird inconsistencies, because it's actually written by AI driven by people who don't really understand what they're doing.
the tell-tale mark of AI code is highly over-engineered local solutions to trivial problems that don't matter, or that were already solved better elsewhere and that no sane human would ever duplicate.
asdfsa32 1 hours ago [-]
Fair, but I fear that now even more people who can't code will code, and code that is not any better than what people who could barely code write. Growing cabbages starting to look more and more interesting.
IshKebab 7 minutes ago [-]
When did you last try? They've improved a lot.
Also often the difficulty with writing code is simply knowing where to start - getting past the blank page. AI can help a lot with that. Often there's a task where I've got kind of writers block, but you can ask AI to do it and suddenly it's like "ah yeah, sort of but actually that's not quite right we should do it this way".
roschdal 4 hours ago [-]
JIT compilation is unsecure.
stevefan1999 54 minutes ago [-]
So what, are you willing to go away from von-neumann architecture where instructions are data and data are instructions, i.e. the instruction-data hominocity that underpins JIT compilation? Are you willing to go to a pseudo-Harvard architecture where the ability of JIT compiling is soft locked by other means like VM or strong code authentication or policy protection, which is what Apple is doing.
Fun fact: even Apple themselves have JIT. JavaScriptCore on iOS has JIT, it's just that the App Store policies forbid any application submissions with JIT or trying to mmap/mprotect an executable region. There used to be apps on TrollStore that runs JIT
asdfsa32 3 hours ago [-]
You're entirely correct because JIT requires violating Write xor Execute security policy. This is the reason on iOS, it is limited to Apple shipped software.
> - Android Runtime Just-In-Time (JIT) compilation/profiling is fully disabled and replaced with full ahead-of-time (AOT) compilation. The only JIT compilation in the base OS is the V8 JavaScript JIT which is disabled by default for the Vanadium browser with per-site exception support.
> - Dynamic code loading for both native code or Java/Kotlin classes is blocked for nearly the entire base OS. […]
> - Dynamic code loading for both native code or Java/Kotlin classes can be disabled for user installed apps via 3 exploit protection toggles: […]
W^X is typically per mapping, not per memory page and does not interfere with JIT compilation.
asdfsa32 2 hours ago [-]
Sure, but it still means that the OS has to decide who is allowed to do it and to what extent. Sophisticated worms like Stuxnet would be much harder with strict W^X for example, since CVE-2010-2568 and the like would be much harder to execute.
orf 2 hours ago [-]
> Sure, but it still means that the OS has to decide who is allowed to do it and to what extent
It has to do that anyway?
asdfsa32 1 hours ago [-]
Only if it wants to allow Writable Memory to become Executable, or basically, allow JIT.
pjmlp 2 hours ago [-]
Signed binaries with the proper assigned OS capabilities.
asdfsa32 1 hours ago [-]
Yes, but with JIT, you can't really verify what the application does upfront. That is the entire point.
pjmlp 1 hours ago [-]
Capabilities are a way to control that, and the point being that only responsible proven applications get the certificate, hence how it all goes on iOS.
asdfsa32 46 minutes ago [-]
You're making the assumption that "responsible" is something provable, but that is not the case, it is specially not easy to prove software is secure from tampering its behaviour.
1 hours ago [-]
dennis16384 4 hours ago [-]
It is the core of ClickHouse for example, for many years. Is it secure enough in your opinion?
JackSlateur 2 hours ago [-]
In rust, is jit equivalent to an "unsafe" block ?
brabel 17 minutes ago [-]
Read the code in the post. Everything is written in unsafe Rust. The assembly itself knows no memory safety at all and is completely up to the programmer skill whether it can be trusted to not mess up.
pjmlp 2 hours ago [-]
Machine code is insecure, we should all run interpreted code in a formally verified interpreter.
Alternatively, only allow for the execution of cryptographly signed static linked binaries, this naturally includes the interpreter above.
> The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile.
That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM.
[0] https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-c... , discussed: https://news.ycombinator.com/item?id=39742916
[1] https://www.postgresql.org/docs/current/jit-reason.html
It was the limits of 8 bit home computers hardware that made the interpreter version be more widely known.
Same to Lisp, Smalltalk, and many other languages.
Fully agree with you.
It is very relevant
By not using LLVM, you're missing all the optimizations it does.
LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.
Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...).
I believe this because every time I use AI for domains that I consider myself above competent, if it is anything beyond UI components or a simple CRUD endpoints, I cringe at the quality of what it generates.
This has made me to be extremely cautious of starting working in a new domain with AI if I want anything beyond throw away quick hacks or junk, shy of quick bug fixes perhaps.
It generally performs better if the tasks are broken done into small manageable pieces, and the person is actually reviewing and calling out problems, which usually requires the person to be a competent engineer in the problem domain to begin with.
But yes, I have personally used it to build what the OP calls a JIT. I would usually write that by hand and it would take me one week. The AI does it in an hour.
I used Fable on a Zephyr project with time sensitive code for LR-WPAN and it broke everything. Literally made the code worst to the point that the devices stopped connecting.
If I need to be a domain expert anyway, the value of the tool goes down by orders of magnitude. Same if I need to first break the task down into pieces and keep reviewing all the output. That sounds to me like >80% of the work I'd need to do anyway.
Personally, I've found the greatest value in asking for simple, easy to verify tasks, like wiring up APIs, generating boilerplate, etc. Anything remotely complex and the LLM cakes its pants.
That’s not a joke. A lot went on to be programmers professionally. And judging by the quality of closed & open source code I witness daily those figures from university accurately depict people’s capabilities.
Now that said, if you can’t really code then using AI will be a godsend to said individuals.
the tell-tale mark of AI code is highly over-engineered local solutions to trivial problems that don't matter, or that were already solved better elsewhere and that no sane human would ever duplicate.
Also often the difficulty with writing code is simply knowing where to start - getting past the blank page. AI can help a lot with that. Often there's a task where I've got kind of writers block, but you can ask AI to do it and suddenly it's like "ah yeah, sort of but actually that's not quite right we should do it this way".
Fun fact: even Apple themselves have JIT. JavaScriptCore on iOS has JIT, it's just that the App Store policies forbid any application submissions with JIT or trying to mmap/mprotect an executable region. There used to be apps on TrollStore that runs JIT
https://en.wikipedia.org/wiki/W%5EX
> - Android Runtime Just-In-Time (JIT) compilation/profiling is fully disabled and replaced with full ahead-of-time (AOT) compilation. The only JIT compilation in the base OS is the V8 JavaScript JIT which is disabled by default for the Vanadium browser with per-site exception support.
> - Dynamic code loading for both native code or Java/Kotlin classes is blocked for nearly the entire base OS. […]
> - Dynamic code loading for both native code or Java/Kotlin classes can be disabled for user installed apps via 3 exploit protection toggles: […]
https://grapheneos.org/features
https://en.wikipedia.org/wiki/W%5EX
It has to do that anyway?
Alternatively, only allow for the execution of cryptographly signed static linked binaries, this naturally includes the interpreter above.