Foojay Podcast #98: The End of JNI Pain: How WebAssembly Is Quietly Replacing Native Libraries in Java

Transcript

Automatically generated from the audio, so it can contain errors in names and technical terms. Suggest a correction.

[0:00] Web Assembly is already running in your Java app, but you just didn’t know it.

[0:05] Welcome to the Foojay podcast. All your news about OpenJDK.

[0:11] Welcome to another episode of the Foojay podcast. Today we’re diving into one of the most quietly exciting shifts happening in the Java ecosystem right now. Web Assembly becoming the invisible glue that lets Java developers use libraries written in Rust, CG go or JavaScript without touching GNI without shipping platform specific binaries and without leaving the GVM boundaries. I’m joined by Andreo Peruo who works at IBM and enables web assembly on the GVM. He wrote a great article on Foojay about exactly this topic. So let’s get started.

[0:49] Thanks Andrea for joining.

[0:51] Thank you for having me

[0:53] And thank you for the article on Foojay and I know web assembly I know it exists but can you explain in what sentence what it exactly is?

[1:03] Yeah web assembly is quietly u being used by every one of us especially in the browser. So if you start from that story you are daily using web assembl it powers sites like Google maps Google excel or the G Suite or Adobe Figma they there are a ton of different products that are being distributed through the browser thanks to web assembly and they would have been impossible to be realized with that level of performance if it was not because of web assembly. So this is pretty much spread around in the browser area and there are a ton of new npm packages that are being shipped containing some some some sort of web assembly payload as well. So this is completely spread in the node ecosystem. but it’s something that you don’t notice. You don’t notice because you don’t have direct access to those web assembly payments. They are used to perform to actually trigger the performance of your browser and give you a great user experience.

[2:19] Is it a language? Is it a runtime a combination? Probably

[2:23] Web assembly is a binary format is a binary format for executable programs. So what it does mean is that is like another target architecture. So like you have x86 then you have a web assembly but web assembly is just on one target architecture that will work on any other architecture and operating system. So how do you compare it to the GVM? Is it a bit the same right once run everywhere that ID

[2:50] Is some kind of that idea. So web assembly starts in the browser because of speed and they wanted to have low-level languages like Rust C compiled to web assembly and be portable into into the web browser in a safe sandbox. the same is happening here. So basically you can take you can take those different programming languages like Rust, C++ or whatever compile it compile them to W asma and execute on any other web assembly runtime and web assembly runtime are everywhere be it your browser or was time or endive or zero there are multiple projects that are bringing web assembly and any other host runtime

[3:35] You say it’s performance so is it can you compare to JavaScript if I have some complex thing running in the browser with JavaScript and then with web assembly is it comparable in anyhow performance wise

[3:49] No in the sense that web assembly starts as a continuation of a project called ASMJS if you have heard about basically they were trying to take any performance bit out of JavaScript but JavaScript is a dynamic programming language so that’s not going to go fast eventually

[4:10] [laughter]

[4:10] So, so enabling system system programming languages like a C, Rust very low level where you have access to all the resources. You can directly control and manually control your allocation, manually control every every bit and piece that is moving you can achieve almost near native performance. So with web assembly they are claiming that they are in the order of magnitude of 1x 2x maximum the performance of a native binary on the same machine

[4:47] Which is pretty good because because the idea is that the execution of such program is completely so it what it means when I say sandbox I mean that it is a target event using a separate memory. Mor other than the host.

[5:05] U so not your browser your not not your system memory but a bite array that you reserve for it. So [clears throat and laughter] and having the memory in a bite array means that you are automatically stepping away from any CVE things that might happen on your machine. So paying a little price for performance but just to have an extremely good by design safety. Yeah.

[5:33] Is the ch is the choice here.

[5:35] So web assembly wom is the short version is the same thing.

[5:40] Yes. Yes. Same.

[5:41] It’s the same thing. Okay. So if people have heard about wasome that’s what we’re talking about. your article says if you’re running J Ruby G Ruby in production you’re already running web assembly. I had no idea. I think most Java developers had no idea. So how what is web assembly wasn’t doing there in J Ruby? that was a an extremely interesting story that started basically at the start of the project when it was called chory and I developed it with Ben. So the idea is that in parsing Ruby code is extremely difficult. they have been developing a ton of different parsers for the language because the language is very much designed for humans. So it’s very very hard to be understood by computers. so they have developed many different parsers and they couldn’t agree across different ecosystems. So C Ruby with J Ruby with truffle Ruby with other implementation of Ruby. They couldn’t agree on having one single parcel that will be consistent across the entire ecosystem. So they do

[6:55] What we usually do in our industry. They came up with another project, a new project, [laughter] a new project to tackle them all and to standardize over it. I have to say that seems that they are being successful in this effort is a multi-year effort but I see it going through right now. Basically what happened is that they rewrote the Ruby parser from scratch and this project is called the Prisma is under the Ruby organization on GitHub if you want to look up for it. is a super efficient Ruby parser written. But what it means is that it can very easily be compiled to web assembly.

[7:45] And what happen is that when you want to bootstrap Ruby on the JVM, the first thing that you should do is to be able to read some user input. So Ruby programs that are provided by by by the user. And to do so, usually what you will do is to link with JNI or some kind of for FFI interface the Prism native library and talk to it. But what happen when you are on an architecture where that is not supported or that is like a different operating system that has not been tested or that you don’t have access to. this would not work because JNI needs to be distributed for each architecture and operating system. so what happen is that they can run thanks thanks to web assembly in J Ruby they can run the prism parser as the first thing and they can bootstrap the entire ecosystem starting from the bare JVM

[8:56] And this is a great power because this means that they can eventually go wherever they want

[9:02] On the yeah you mentioned a few things so you were talking about JNI which J I a I always mess up how I have to pronounce it, which exists in Java since I think 1.1, so a very long time. Then we had the FFM API in Java 22 who wanted to bring a new way of interacting with libraries and external code and external memory. so it was an improvement of JNI. Now, now what is the difference in the approach between JNI, FFM and web assembly and using it in Java? I would say that FFM is a great improvement over what we had with JNI. But let’s start from the very beginning. what JNI is a JNI is you have a C library, a native library, you compile it and you have some kind of interfaces that you can call and receive data to and from Java. This is effectively running a native library on the user machine from the Java code. So when you are accessing any kind of functionality on the native side from Java, you are losing a lot of guarantees on the JVM. So what happen is that you are act effectively escaping the JVM. So you don’t have observability, you don’t have debugging.

[10:27] All the tools start stops to stop to work. And even worse, if you have a single little bug on the native side and something crash when you are coding invoking it, it will crash the JVM completely which is not acceptable in some scenarios and in some situations. So the and FFM the new for the new interface is a step on top of this which is extremely good. of a enable you to write in a much more expressive way the same concept of basically crossing this barrier across Java and a native library and it enable you to do the very same things with even less foot guns and is extremely good. But the pro the core problem remains you are anyway accessing some external resources that are not on the JVM that needs to be compiled and available at least at the very least on the on the machine for the specific architecture and operating system while when you are using web assembly is just one target one compilation target. So what it means is that you can directly distribute your web assembly and you can either interpret or compile and

[11:58] Execute it within the JVM boundary. So the code that has been written in web assembly becomes effectively Java byte code and this Java P code is completely observable. It doesn’t escape the JVM is super safe and it doesn’t arm host memory. So this is extremely safe and is even good to execute untrusted code. So you can ask your user to send you scripts of whatever and you will be safe executing it in a web assembly sandbox and at the same time it gives you all the power to access a lot of [clears throat] native functionalities if you want. Yeah, I know the switch from JNI to FFM because I’m involved in the Pi4J project where we interact with Raspberry Pi GPIO pins and indeed GNI was extremely complex to code and to maintain you have this five layers and compiled code in between and now with FM we directly go from Java code to kernel and trigger some GPO bit what you actually say is with W was actually staying even more in the Java code. Correct.

[13:09] That’s that’s correct. That’s correct. One one note that I wanted to do is that you mentioned that you are accessing kernel functionalities [clears throat] like a GPIO etc. I did embedded it in the past. and this is usually not possible with assembly given the layer or it is not possible out of the box with a plain web assembly payload given the sandbox in the safety story. You should give a specific access to those resources outside from the O programming language otherwise the web assembly module will not be able to actually trigger something of occur. Yeah, this is absolutely forbid.

[13:53] Okay. But yeah, it makes it clear where web assembly actually stands within this approach. In the article, you’ve listed several things like SQL, light forj, quick gs4j, pg lights, protocol buff. they all have some kind of wasm in there. What was the biggest surprise that you found there? Those are a bunch of examples of how the technology can be used to leverage real world applications basically.

[14:22] So

[14:24] They don’t have a bit of was they are fully web assembly libraries with a very very very thin wrapper on top of them to make them usable easily usable from the Java ecosystem. So I can go through a bunch of them. So when we talk about SQLite, SQLite is apart from the Java community is the most used database in the world. is used on any Android device is used on Android everywhere is used for desktop application for everything and is a C program is a C program. So what happen is that you can take SQLite compile it to web assembly you take web assembly you compile it to Java byte code and what happen is that you have a super safe environment where you can run the original database with all the functionalities so even even advanced functionalities of the database fully in your process fully in your JVM application without escaping it. So this is extremely good for distribution.

[15:33] For example, if you want to you can distribute things like with a jar and everything will work and we self-contained included the database and for SQLite what I did was to take the JNI version of the JDBC driver and adapt it to use web assembly instead of the JNI bindings. And easy peasy, you have a onetoone you can switch off the old JDPC driver. You can pull in the new one fully fully based on web assembly and nothing changes to your application but you have everything embedded.

[16:14] Actually we I did the same thing with PGite. PGite is a version a patched version of posgress. So you can actually run posgress the real database in pure Java bite code if you want and there is a JDPC driver there as well and you can go with it is a little on the heavy side as you can expect [laughter] because you are running a full possess database inside your application which is not designed for it but it was very cool to achieve together. Oh, there definitely use cases there if you have a small data set that you want to load and inquiry and you don’t want to deploy a database. So, there’s definitely use cases there.

[17:03] Yes, we used to have a H2 because because to go on the JVM, what we used to do is to rewrite our programs, rewrite our programs in Java.

[17:14] And what I’m saying is that it’s not needed anymore. Yeah. So we can take C R programs and just run them on the JDM as we are used to run Java and

[17:24] That’s pretty good is great for use. I don’t have to pay for maintenance. I keep updating the SQL light versions updating their so not being affected by their box and they are updating it for me just changing one version because I maintain a super super thin integration layer instead of maintaining the core functionality that is database internals which is arguably hard. [laughter]

[17:54] Yeah. Maintaining the installations. Now if we go to your work so in the article you talk about Chikor project and endive. Can you give us a bit of history there? In 2023, I have a friend called whose name is Edardovaki and he started to work for titrate at the time to maintain web zero which is basically the web assembly runtime for go to avoid the sego in invocation and such. So what happened is that I was fascinated about the concept and introduced me to the city of Dipso. Dipso was a company that was primarily doing products around web assembly and projects around web assembly and the CTO start broke his leg and started writing an interpreter in [laughter] you have a lot of free time and after probably a week or a week since he started Edward introduced me to Ben which was the original author. I was actually writing my own interpreter on the site etc. But it was much better to join someone else codeabase. So I joined the effort and from there on we started we completed from interpreter we got the a compiler working both at runtime and compile time that can

[19:33] Generates directly Java bite code and we almost or we cover a very very large chunk of web specification so that we can run u real world programs as we focused on the databases side, but we have runtimes like quick.js is a JavaScript engine or protobuff. You have libraries that are use that are useful in general and yeah that started as a chory and what happened is that in the meantime the story of dead changed a little bit. They are now focusing on different stuff around AI. As far as as I’m aware, we wanted to have the project in a foundation. So in a space where it was not birectionally connected with just one vendor of the technology I spoke with folks and they invited me to fork it under another organization and that’s what I did. I reached out to bite code alliance which is the foundation which is a sponsoring and steward stewarding all the development around web assembly those days and I joined that foundation because it’s the most natural fit for the project and from there on this project is called dive

[21:03] And it’s within the bite code alliance so that’s they have several projects within this web assembly space

[21:09] Oh they have a ton of project in this web assembly space. Yes, they have different runtimes which are for example west time which is a rust runtime which is the leading runtime for that always implements the latest versions of the specification. they have warmer which is for C and embedded devices. they have they are maintaining native compilers from web assembly to real assembly code etc. So there is a tone of very interesting projects under bite code alliance and not not just to execute web assembly but even to build around web assembly. So they have tools for working with web assembly they are they have various things around it. Okay. Now if we dive into that endive project. So that’s the project going forward. what is the goal me as a Java user? What can I do with it?

[22:13] I do want you to not need to rewrite anything coming from other ecosystem going forward. I do want to just reuse a library. I am I do not want to go deep into understanding the inn and outs of very very complicated pieces of software that are already written by great developers and I just want to be able to reuse them. I do want to be able to run JavaScript like with quickjs to run python you can compile the cython interpreter into java bite code and execute it. they are doing it in a distributed database for example based on Java and those kind of things. So I do not I do want to enable people to reuse as much as as possible and to not have the burden of maintaining very complex pieces of software that are already written in a very good manner in other ecosystems. So if I have a very good library which is in C for instance, I should look for a way to convert it to web assembly and then I can use it in my project.

[23:23] You

[23:24] With your tool correct?

[23:27] Yes, you can d Yes.

[23:29] Yeah. one of the first major milestones that you mentioned is the integration of the experimental red line compiler. What does that do?

[23:39] Everything that I told you sounds really nice. and executing web assembly within the JVM boundaries has been bringing us quite far. what happened is that people trying on very random payloads or very random kind of applications this technology they always come back with just one comment. They are not concerned about the sandbox. They are not concerned about the separate they are just focused on performance.

[24:12] [laughter]

[24:12] This is what ended up happening. and to give this level of performance, we couldn’t actually stay within the JVM boundaries. So what happened is that we take the web assembly code as user input. We are a compiler. So this is user input. I mean we don’t have much control over how this code is written because it comes out other another compiler tool chain the was SDK or rust or whatever and often what happened is that some of the functions that are contained in this web assembly payloads are very very big. You can imagine the CPython big switch for the interpreter group or you can think about the logic for parsing SQL like queries which is a huge you know like like a state machine with all the access to random pieces in the same functions and they and this comes out to become a super huge function and turns out that the GVM is great and almost exceptional for running very small functions but is really not fast when you try to execute a big function. when you go over 8,000 bytes of the function code the JVM gives up completely. So it stops try even trying to compile

[25:55] Those big functions and they are executed on the JVM interpreter which is slow. [laughter] But unfortunately what happened is that those coded path like the SQL parser the cython interpreter loop they are on the super hot path of a of an application and we couldn’t reliably overcome this limitation in [clears throat] any reasonable way. So we have tried we have tried to split those meters up in different chunks and it was extremely har extremely fragile. It broke every time [laughter] every time and it was so unreliable and also the performance gain was not reliable as well. So we got some very some positively positive results but not great results you know like like something that shows you another way. So what happened is that with Panama FFM what happen is that you can directly link a system library without having to distribute the code. In this case, what happen is that we are directly invoking lib C to be able to reserve memory memory areas and drop their assembly code. To do it, what we do is we take the compiler from web assembly to machine code that is beyond

[27:31] Time. I mentioned was time before but this compiler is called crane lift and this a compiler written in rust

[27:40] That can take web assembly code and translate it to machine specific code. So x86 Linux or arm Mac or whatever. So since this is a rust program it can be compiled in web assembly. compiled in web assembly you can use it on purely on the JVM. So now you are purely on the JVM and you can directly take web assembly again compile it to the machine specific architecture and without distributing any any native library you can just map this assembly code in memory and ask the ask your the machine to execute it. the security boundary is respected because it’s the compiler behind the wstein. So they do verify the security of this code. But in this case, you would be calling a code external of the JVM. But yeah, that’s where we gave up to just give the user as much performance as they like.

[28:43] And given all our benchmarks performance now are on the order of magnitude of west time basically for every benchmark that we tried which is quite good being at the same reference level of Rust runtimes.

[29:00] It it’s unbelievable how you mix Java Rust native calls into one package. I think if a rust developer hears this story he would say yeah why don’t you use rust for everything isn’t but isn’t yeah what is the goal of combining all this in Java

[29:20] We are still maintaining compatibility with Jav 11 best that’s with power I started saying but I don’t want people to rewrite stuff I haven’t started saying but I do want people to rewrite things in rust I like rust rust is a great language but I don’t want arrived my Android was stuck in rest one day with the other. You have mission critical applications in banks insuranceances and everywhere that are based on the JVM and even if you are on Java 11 I don’t care you can run Rust code written yesterday there I do want you to use in your legacy application whatever comes next and to use it in a secure way. That’s that’s the key message here. That’s that’s what we want to achieve. So we want to be able to just not not throw away old code or old things that are working for us but we want to be able to extend them and to get all the power and all the tools that are coming from even from different ecosystem other than the GBM.

[30:22] So indeed it’s it’s compose don’t rewrite. That’s a bit the central message of your article is improve what you already have and integrate better ways of doing calculations of yeah using the external dependencies that so that’s a bit your main goal that people can improve their existing applications. Correct.

[30:43] Yes. Yes. Yes. They can open up their application to have plugins written in different languages that will improve and give new functionality to their old applications and coming from different ecosystems and doesn’t really matter where they come from. Mhm.

[31:02] If it was Java was already useful but since we didn’t add a good bridge with the native story web assembly is one way of doing it and it enable to embed new functionalities in a very safe way in a legacy application.

[31:19] You mentioned performance performance performance being one of the critical things. so if you have yeah all these compilations in between and switching from Java to was some some rust in between. You also said you have this performance benchmarks. Do you see the system working? Is this as fast as as what you expected?

[31:41] Is pretty fast I would say. So it very much depends on the kind of workload that you have or the kind of program that you try to execute. But yes it’s pretty fast. So it pays a little bit from pure native combinations. So you do have to expect a little hit there but it’s not that big. So and usually it enable a lot of use cases and given the super enhanced portability and simplicity in redistributing running in web assemblies much better than any other native comp natively compiled library.

[32:24] Mhm.

[32:24] You have this Java run once write once run anywhere. we have seen a lot of improvements there also with the FFM API to integrate with other libraries interact with other libraries. Do you see more changes happening within Java GVM the ongoing projects there that will also benefit to integrate better with other things like web assembly. There is a very good very interesting story that is going on with project Detroit. If you have heard about the project Detroit, basically the target to make to implement the Java scripting API for CPython and note

[33:09] So using the new FFM they are linking native CPython and native JavaScript to enable the execution of purely CPython packages and purely node packages without restrictions but calling it from the JTM. So we always had like like weird X to call into Python or rewrite of parts of Python or changing headers or very very a lot of different stories to try to mix and match the two environments but this project Detroit is targeting to make the switch much smoother. So I’m trying a lot and I’m playing a lot with FFM with a new panel FM and I have to say that is really great. It is a really great step forward compared to JNI which I did a lot in the past. I do think that there is space also for a web web assembly story here. So what happened is they are saying that you need to do everything with FFM which is good of course but what if you can give also the safety story so the which is what is missing so what if the memory that your Rust module can use is just the bite array for you in Java so any CVE won’t affect your application you would be able to run untrusted code with the

[34:41] Rise of AI and a lot of new CVE being exposed. Running a payload in a safe sandbox often is the best choice that you can do if you want to provide a functionality without exposing each and every line of your code to a lot of different malicious attackers.

[35:05] You mentioned AI, one of the hot topics of course at this moment. Do you see specific use cases where your tool could be very handy?

[35:15] So there are a lot of very very interesting use cases and I want to tell you one as much as I have understood it but I have people in HubSpot that are taking and dive and using it to execute Python. But to execute Python what I mean that they have in AI which is generating Python scripts.

[35:44] Mhm.

[35:44] And those Python scripts can contain whatever. So they do need a secure a very secure environment to execute them. So they can dive and they are a Java shop. So everything is on the JVM for them. So they take a dive they have a C python version compiled to web assembly. They give those pieces of Python software to several agents that can run that can run several cyon C Python engines with super super fast startup. They claim that a Cython engine like this can start in 0.2 milliseconds which is pretty fast.

[36:27] [laughter]

[36:28] And is something that you cannot achieve with container technology unfortunately. So you give those python snippets to all your CPython execu exeutor in web assembly and you get the result and then you can decide for example if you are comparing different implementation of an algorithm. you can get the one that performed the best or the one that was written more clearly or whatever but whatever choice but you can actually execute completely untrusted code generated by AI in super fast and super safe in the staying in the JVM in this case.

[37:10] Okay, that sounds great. This is also very experimental but that’s yeah where the state of AI is at this moment. We are all trying out what can we do with these tools and how can we safely [laughter] integrate them in our applications. So that’s definitely something which endive can solve. what’s what’s the plan for end dive? Where do you want it to be in two years?

[37:33] Oh wow. Two year two years is

[37:36] Two two years is far away in software. I know. Let’s imagine [laughter]

[37:41] As the short term goes, we really needed to get a release out of a knife because I haven’t released just yet, but it will happen any day soon.

[37:52] And next we will integrate the new red line compiler, the one I talked to you about before, the Rust integration. And next we have to finish and I’m doing it and I’m need some garbage wi improvements. So we in web assembly to extend web assembly to garbage collected languages there is an entire part of the spec that has been developed. Now we com we completely comply with the API but there are some internals that needs to be tweaked and there’s a big quite quite large right internally to have maximum performance and to be able to actually let the Java garbage collector act as well on web assembly payload. And we need to improve a little bit that story to just make extra smooth to run even languages like a cotlin scalart on top of it which is possible today but can be improved. and after that we have component model by code alliance is basically sponsoring and driving forward the effort for component model. Component model is preview two and preview three. So is the way a web assembly module has to be composed with our other web assembly modules and to integrate

[39:25] With the system. So the idea behind the component model is that once you have a component you can describe its interface and you can plug it like a Lego brick

[39:37] With any other with any other Lego brick which are web assembly modules as well. And this gives full interoperability across languages. And this gives the possibility to just fully interoperate any kind of any any kind of application with any other kind of application given that they respect the same def defined API. And this is very likely the next thing that the next big thing that we will pick up. We after I opened up and dive I received a lot of mails and messages by people interested in the joining the effort of implementing component model. So please come to me. I’m super happy. I created a separate repository to start experimenting around it and to enable the community because while I don’t have so much time to work on component model as of today I really want to enable the community to start experimenting and start working on it because it’s a very very interesting next step surely for web assembly and we want to be as complete to cover as much of a spec web specification as we can. So to enable any kind of new payload that has been developed for

[41:04] Example for the web to be to make you able to just run it inside the JVM

[41:12] Without any effort and making things super smooth is a long long process that require combining a lot of different pieces.

[41:21] Yeah. Okay. I will add to the show notes definitely the links to the GitHub project and the other projects and where people can reach out to you. okay. Anything else you want to share about this project about web assembly in general?

[41:33] It’s a great technology. You should use it at least for plug-in systems. So you can extend your application. It was something that was hard to be done in the past and it is not anymore. And I’m demonstrating with all those project like how it how easy is to em to rot see JavaScript runtime and embed it in your application and making the this in a secure way so that user can actually execute untrusted code code that you don’t want even to know what it does

[42:10] Is an in ext is an extremely great power and has being exploited for example by all the CDN like cloudflare workers are running web typescript or web assembly because it’s based on nodes technology fast is running on one time a just bought another firmian that are running things in web assembly so this already demonstrated to be the key and killer application of web assembly when running on the edge on lower resources but this can be expanded to our entire stack and we plugging points in each of every application. And to steal the words of Ben Eckl, we want to make software programmable board. So [laughter] we really want to enable people

[43:00] To program their Java application even in different programming languages.

[43:06] Yes. In other languages and integrate everything. Okay. looking forward to your next articles on Foojay where you release notes, new versions, new features. please share them also on Foojay audience can find them there. Okay, really great. Okay, that’s a wrap for this episode of the Foojay podcast. Huge thank you Andrea for sharing such a clear picture of where Web Assembly on the GVM is today and where it’s heading and now I also understand what it exactly is and what it can do. if this episode opened your eyes to wasn’t the way it did mine, then go check out the repository on GitHub and read Andrea’s article on Foojay. The links are in the show notes. If you enjoyed this episode, please subscribe to the Foojay podcast on YouTube or in your favorite podcast app. You can find all our episodes, show notes, and links to the resources mentioned today on Foojay.io.

[44:01] And don’t forget to follow friends of OpenJDK on social media for the latest news and updates from the Java community. Until next time, keep coding and keep composing whether it’s Java, Rust, or something in between. Thanks for listening. Give me the friends of OpenJDK.