Build Universal Libraries with Rust

An explanation and tutorial of how to write libraries in Rust and submit them to the WebAssembly Package Manager such that they can be consumed by projects written in most major programming languages.
WAI Definition language documentation: github.com/wasmerio/wai/blob/...
A SHA2 package in WAPM that I mention in the video: wapm.io/wasmer/sha2
An overview of Wasmer and WAPM: • WebAssembly On The Ser...
-
Stuff I use to make these videos - I absolutely love all of these products. Using these links is an easy way to support the channel, thank you so much if you do so!!!
Camera: Canon EOS R5 amzn.to/3CCrxzl
Monitor: Dell U4914DW 49in amzn.to/3MJV1jx
SSD for Video Editing: VectoTech Rapid 8TB amzn.to/3hXz9TM
Microphone 1: Rode NT1-A amzn.to/3vWM4gL
Microphone 2: Seinheiser 416 amzn.to/3Fkti60
Microphone Interface: Focusrite Clarett+ 2Pre amzn.to/3J5dy7S
Tripod: JOBY GorillaPod 5K amzn.to/3JaPxMA
Mouse: Razer DeathAdder amzn.to/3J9fYCf
Computer: 2021 Macbook Pro amzn.to/3J7FXtW
Lens: Canon RF24mm F1.8 Macro is STM Lens amzn.to/3UUs1bB
Caffeine: High Brew Cold Brew Coffee amzn.to/3hXyx0q
More Caffeine: Monster Energy Juice, Pipeline Punch amzn.to/3Czmfox
Building A Second Brain book: amzn.to/3cIShWf

[@workflowinmind]

I had no idea wasmer could do that! That's awesome

[@codetothemoon]

it's a fairly recently introduced capability, so it's understandable. I'm pretty excited about it too

[@anush8]

So cool. Wanted to build something using WASM for like forever. Always hesitated because I felt the setup and build-processes aren't streamlined at all. Wasmer seems promising. Can't wait to try it. The WASM book and the MDN docs didn't give me the push to go on with WASM. This video did. Thanks a lot.

[@codetothemoon]

nice, really glad you got something out of the video! if you're willing, definitely let us know what you wind up building with it

[@anush8]

Sure.

[@gomi-hako]

That's exactly what I needed for the library project I'm currently building

[@TheJohndward01]

Wow this is really cool. Also I need to learn more about web assembly, definitely checking out that video!

[@codetothemoon]

I agree it's pretty exciting!

[@PaulSebastianM]

It's not enough that they are really talented developers, it's the fact that they work together as a team or just helpful contributors, most of the time! We often underestimate that kind of human relationship: working together towards a better future. We have a lot to learn from these kinds of interactions and their results in the real world, be it software related, NGO related, or just people in a local community building a better community for everyone.

[@xucongzhan9151]

Most NGOs are sponsored by some government with a clear but hidden agenda, so that should probably be excluded.

[@dmitriidemenev5258]

I'm the contributor of cross-lang-and-cross-platform project and I'm excited to see more people tackling the problem of cross-language interoperability!

[@dynamite-bud]

got your mail, Thanks Dimitrii If you would like to connect dm me in community slack

[@cinamonroll2701]

Very good product. Congratulations guys. Kisses from Romania looking forward to implement your awesome product.

[@codetothemoon]

I agree it shows great promise!

[@vsuryabhargava]

Great content as always.

[@apinanyogaratnam]

moving your face to show the code which otherwise woulda blocked the code surprised me and is much appreciated!

[@kajacx]

You get a reference to self when using a resource automatically? That is *so good*! Can't wait to try it out.

[@codetothemoon]

😎

[@slava_in3757]

Great video! Wonder how it treats the types for TypeScript, does it emit `.d.ts` files as well?

[@sergioromano116]

This is so cool, I need to try it

[@codetothemoon]

it's worth giving a try!

[@harikrishnanb7273]

i always dreamt of such things, using webassembly as common between language given every language is trying to integrate with webassembly in some way.

[@codetothemoon]

yeah it seems perfectly positioned to fill that need!

[@trumpetpunk42]

I'd be curious what the common known pitfalls of that are, and how to avoid them. Like the xkcd "one standard to rule them all" becoming "now we have yet another standard." As someone else said, we don't need yet another JVM or DCOM or CORBA. But if there's a way to make bi-directional FFI convenient and architecture independent, I'm all for it.

[@ariasalehi5665]

This is amazing, thanks for sharing

[@codetothemoon]

thanks for watching!

[@NathanHedglin]

Great! Rust + WASM = super powers

[@codetothemoon]

agree 💯

[@wfzyx]

I really like the idea, and believe that this can be used in the future by smaller/newer languages to have strong ecosystems, given they implement wasm integration. The only problem that I see right now, is that you don't have LSP for the WAI bindings on Rust for e.g, so this makes developing anything more complex kinda annoying.

[@yokebabjr3866]

Great video! It seems to have plenty of use cases for this. Do you know if we could also use the webassembly sandbox mecanisim with this approach? I could imagine a plugin system with restricted rights.

[@ayushdedhia25]

Informative 👏🏻 btw what font are you using it looks pretty cool 🙂

[@codetothemoon]

Monaco!

[@ayushdedhia25]

@@codetothemoon thank you!!

[@alexIVMKD]

Very cool!

[@codetothemoon]

thanks!

[@jeffg4686]

is this an implementation of the new "Component Model" in WASI? Great video, btw - thanks for sharing - packed with good info. When a language with a runtime publishes to WA, will that include a WA coded runtime for that language (like C#)? thx IE, will it perform like a WA module coded in Rust, or more like a garbage collected language.

[@ha5anmukhlis]

Are wai files standard format or is it something by wasmer? with how webassembly is, I'm scared it might turn into a mess of an ecosystem like javascript

[@-indeed8285]

The font he is using is 'Monaco'

[@codetothemoon]

yep!

[@Erhannis]

I've searched their main site, their docs site, and glanced at their github, and I still can't figure out what languages they support, which is MINDBOGGLING. D:

[@codetothemoon]

There used to be such a list, I think they recently did a major revamp of the website (and merged wapm.io into wasmer.io). It seems like there is a list of the major languages here wasmer.io/products/runtime but I'm also not sure if the comprehensive list exists somewhere. Is there a particular language you're curious about?

[@jfk1337]

Does autocompletion work for these libraries? Does the lsp know that the uc object has the method hash with it?

[@guilherme5094]

👍Really nice👍!

[@kajacx]

Damn, the "WASM Interface Types" dream is true. I will have to see if I can use this instead of fp-bindgen.

[@codetothemoon]

its worth a try!

[@timkilian7140]

This will be the future

[@moeruishi]

I have a question. I’m currently working on a personal project and I’m wondering if there’s a way to define 2 functions with same name but different argument types and return types in the wai file? Similar to what we can do with traits (think function overloading)

[@leviathan7477]

When using records, how do you instantiate an instance of the record type in the language using the library?

[@apinanyogaratnam]

so is it running rust code or is it generating code in the language you use it?

[@ordinarygg]

Awesome, more malware libraries))

[@codetothemoon]

why limit yourself and make your malware consumable from only one language? 😉

[@ordinarygg]

@@codetothemoon you dirty boy!!!!!)) Love your humour and channel, subscribed already!)))

[@michael-rommel]

"enough brainpower to center a or quit vim" - good one 😂

[@codetothemoon]

thanks 😎

[@Im_Ninooo]

this is awesome!! I wonder if it's possible to generate a Node package locally/without publishing it.

[@Im_Ninooo]

I found out it is, by using the wasmer-pack crate

[@DevvOscar]

Random off topic question. How does he have that shell command input on the bottom in emacs? I want to get the same in Neovim. I don't want to type :! every time.

[@codetothemoon]

you get that "out of the box" with DOOM - I press M-! to get it. One fewer keystroke than nvim I guess. I'm also curious about a quicker alternative in neovim.

[@dungeon4971]

which font are you using ? looks good to be honest

[@BrazenNL]

If you look at the lowercase 'i', they are different in the terminal and in the editor.

[@codetothemoon]

In emacs it is Monaco, in VSCode i *think* it is Menlo

[@sadramohh]

For anyone trying to recreate this on their own: *WAPM is deprecated*, and so `cargo wapm` has been replaced by `cargo wasmer`. This is how the metadata configuration looks like in `Cargo.toml`: [package.metadata.wasmer] namespace = "" # same as your wasmerio account handle abi = "wasi" # choose "wasi" for both javascript and python bindings bindings = { wai-version = "0.1.0", exports = "" } # should not contain `_` underscores

[@Nintron]

6:09 scared the heck out of me lol

[@eliasp.2759]

Great video! Do the wasmer functions only allow primitive types as parameters?

[@codetothemoon]

nope you can have what are essentially "classes" as parameters as well - or "records" in WAI terms. I go a bit more in depth into WAI and its capabilities toward the end of the video.

[@omofolarinajibade]

Is it possible to develop universal client and does it work with reqwest?

[@dulanjala]

rust compile is so slow even for this kind of project. anyway to speed up, caching or just direct linking binaries?

[@roughedge-machineworks]

How do i use wasmer as a dependency and include a wasm file to execute a function off that? Ive tried several ways and just get a wasi error .. :|

[@AK-vx4dy]

Oh man! I undestimated WASM, now i realized that you can send "binary" module but not have to compile it for my arch.... devastating. Is any standalone executor/sandbox for wasm exist excluding browsers and node/deno etc. ?

[@OldKing11100]

This could be amazing. I love using Rust built items, but I'm not a big fan of coding in it, yet. Right now I'm still heavily coding in Go and their dataframe handling package is unmaintained. I've been eyeing polars (and a lot of other crates) and have been wishing to use those in Go. I know the real solution is to just use Rust, but that's not always an option to rewrite everything. Maybe this will be the start for utilizing more Rust under the hood of other languages?

[@d-o-n-u-t]

You don't necessarily have to write everything if you use the web services architecture; I know it's kind of a buzzword, but part of why it's great is that you can implement your backend in multiple languages. You can keep all of your non-performance-critical code in Go and fallback to Rust when you really need it.

[@sebastiangudino9377]

Your entire app does NOT need to use one language of the other. "Interprocess Comunicaition" and "Foreign Function Interface" might be good search terms as general ideas to interoperate Rust and Go (Other than just using wasm, of course)

[@frittex]

wow

[@codetothemoon]

I was pretty impressed when I first learned about this!

[@ChetanBhasin]

What would be cool if you could use the generated WAPM package without publishing it to the registry. That's going to be particularly useful for Monorepos and private code base for people who do not want to publish everything to public WAPM registry.

[@shaunhamilton]

I am quite surprised the .wai file is not automatically generated based on the Rust code. I realise this might be a problem for a library written in a dynamically typed language, but it would be much easier if the file were generated where possible.

[@kajacx]

The Rust code is generated from the wit file, which is better IMO, because you define the protocol first, and then the implementation. But it would be cool if it would be possible the other way too.

[@hck1bloodday]

also the client code need to specify the funtion to call as a string, that seems weird, specially in strong typed languages like Java or C#. if they solve that, I'll take a look at this product, meanwhile i will evade it.

[@Stopinvadingmyhardware]

this means we can stable host COBOL applications remotely now We can replace old iron now.

[@harikrishnanb7273]

okay, i completed watching the video. now i want know about the drawbacks or limitations of this.

[@codetothemoon]

I think the only drawback would be that WASM may be slightly less performant than the equivalent logic running on bare metal (machine code). This will be a non-issue for some use cases, and a complete deal breaker for others.

[@dingus4138]

This is legit

[@codetothemoon]

I agree!

[@hikionori]

Ok, I like the idea of ​​webassemblies but there is some problem because I can't write libraries specifically for webassemblies, this problem is a problem with rust collections. If anyone knows how to solve this problem for example HashMap answer me

[@kc3vv]

I wonder how the performance stacks up.

[@codetothemoon]

as compared to native machine code? I'd be interested to see that as well, I'd be surprised if somebody hasn't done it yet

[@myronkipa2530]

In my experience wasm is considerably slower than native rust

[@codetothemoon]

interesting - how much slower?

[@techlifejournal]

Zig vs rust Can you please make a video ?

[@JosephCatrambone]

In the interest of assuaging my anxiety, you cycled all those credentials and tokens before publishing this video, right?

[@codetothemoon]

I did. Thanks for checking haha 😎

[@JosephCatrambone]

@@codetothemoon Whew. Thank you.

[@recklessroges]

I'm still waiting for WASM's obfuscation to be used maliciously. It feels like WASM has trouble ahead.

[@turolretar]

As soon as you opened vscode I died inside

[@sumansaha295]

I'll never understand the hate for vscode, no other general purpose editor even comes close to the features and extensibility of vscode.

[@cbbcbb6803]

Someone should re-write all of the Python libraries into Rust. That way, a collection of universal libraries, written in Rust, could be called from any other language that a programmer may prefer to use. Is there any software that will assist in converting C/C++ source code into Rust source code?

[@savire.ergheiz]

If only life also so simple 😂

[@arjix8738]

I don't like that the wai file has lowercase everywhere, but the code generated has camel case. Like, why is the "crab" record generated as "Crab" in rust?

[@Nan0MK_Old]

"Enoom"

[@avi7278]

It's pretty much expected that any library these days is in Typescript.

[@codetothemoon]

I think there are many use cases for which TS is not a good fit 😎

[@phredlane9081]

You are mistaken.

[@user-mg6yz6pp7u]

Actually, the WAI interface is not supported by the Bytecode Alliance, which is defining the officiel WASI protocols. WAI is a fork of wit-bindgen, because wasmer tried to integrate bindings to wasmer into wit, and got rejected, because those 3rd party bindings do not belong in the source project. But the Wasmer CEO got butthurt, and make his own fork from WIT, which he named WAI. You all should rely on the Compoment Model and WIT, worked on by the original WebAssembly designers : The Bytecode Alliance.

[@codetothemoon]

I'm aware of the WAI origin story, though I'm not sure anybody got "butthurt". I don't think it makes sense to stick with something solely because it's maintained by the original creators, if that was the case we'd all be using vi or ed instead of Neovim. If you can enumerate the advantages one would get by sticking with WIT, that might be more constructive - for example do they have an equivalent of WAPM?

[@biomorphic]

This is basically CORBA 2.0. It failed once, I wonder if it will fail again.

[@lbirkert]

Postgres on the thumbnail tho

[@babayaga4329]

sure, let's reinvent x86 platform, as a universal platform so we can run it on x86 platform. Now that's out of the box thinking

[@skyline00069]

FIRST

[@codetothemoon]

🥇

[@skyline00069]

@@codetothemoon mom i finally did it

[@skyline00069]

i'm up now.

[@thecoolnewsguy]

​@@skyline00069 🌄🥇

[@developerdeveloper67]

No bro, we don't want to use rust, C++ is fine.

[@0xngmi]

getting this error on cargo wapm: Error: 'AnonymousUser' object has no attribute 'is_email_validated' Error: Unable to publish "universal_crypto" my email in wasmer is verified...

[@juan-lukeklopper3922]

Did you run the wasmer login command?