Okay, it's 4 p.m. now, so we can start with the next talk.
So Dan Phillips is going to talk about deploying Python on Wasm.
And what's interesting about this is there's been a lot of talking about running Python
in the browser, and I think Dan is going to talk about running Wasm on a server, which
is something completely new, at least to me.
Thank you very much. Give him a big welcome.
Thanks. Can everyone can hear me, yeah?
Perfect. Yeah, so thanks very much for the intro.
Feedback, no? Yeah, a little bit.
Okay. Yeah, thanks again.
My name is Dan Phillips, and today I'm going to be talking about deploying Python on Web
Assembly, and the sort of tagline here is smaller, safer, faster, and universal.
The one of these that should have an asterisk is faster, and I'll get into the details on
that shortly.
Briefly about me, I am an engineer at a company called Loophole Labs.
I'm here with a few of my colleagues down front.
We do very specific cloud primitives where we work in specific areas, so I mostly focus
on Web Assembly stuff.
We have a project called the Scale Function Runtime, the Scale Plugin Runtime, which is
a Web Assembly-powered function runtime.
You can check it out at scale.sh.
I'll talk briefly about it at some other points.
On the internet, I'm mostly D-Unscore-Filla.
Of course, on GitHub, since they don't allow underscores, just D-Filla without an underscore
or a space.
And I'm from Chicago, so in Chicago, I run the Web Assembly Chicago group.
If you're ever in town, we'd love to have you stop by, or if you're interested, we also
do all of our meetings virtually as well.
Okay, so we'll just jump right into it then.
What is Web Assembly?
We're going to sort of start at the more abstract level and then get down into the weeds a bit,
not too far into the weeds, but into them enough so that we can get a good grasp on what the
constraints are and what the benefits are.
So firstly, this is directly from the spec itself.
Web Assembly-abbreviated WASM is a safe, portable, low-level code format, which is designed
for efficient execution and compact representation.
There's a debate about the pronunciation of WASM, WASM, doesn't really matter.
Technically, it's probably WASM because it's from the precursor, which was ASM.js, if anyone
has used that previously, which was a set of JavaScript primitives that allowed you
to run more performant JavaScript code, C and C++ code in the browser.
The WASM project sort of came from that.
It's a safe sandbox execution environment, a deny-by-default security model, and it makes
no assumption about languages or host.
The best analogy is that it's a virtualized CPU, right?
So we can think of it as just another compilation target.
To continue, just like when we have architectures like x86, etc., we compile things to those
architectures and we also compile things to WASM.
It's a virtualized ISA, right?
So when we think about ISAs, we have the things I just mentioned, but this is a virtualized
instruction set.
And it's virtualized because it needs a runtime, right?
And it uses a stack machine model for the execution.
So we're not going to go too much into the specifics, but this is kind of the high-level
overview.
So what does that really mean, right?
In the broad sense, it's really just another architecture.
The key differences are it's virtualized.
It needs a runtime to translate to machine code, right?
So in browsers, every major browser has a WASM runtime in it.
The four major ones do, at least, and they're all relatively up-to-date with the spec.
There are also server-side runtimes.
There are runtimes that you can run in kernel-free environments, etc.
We'll kind of talk about a few of those.
And it's universal, by which I mean it's universal in as much as you have a runtime on the machine
that you are going to run the WASM on, right?
So people have played with WebAssembly here.
We just mentioned that there's been some really interesting work with PyScript, PyDyde
to run Python in the browser.
Initially it started as a client-side tech, right?
We go back to the spec here.
This is right from the spec that says a safe sandbox execution environment,
which makes no assumption about languages or host.
And if we dig into that a bit more, we can understand some other benefits here.
It was designed to be extremely compact and to start up extremely fast and shut down quickly,
right?
So cold start times can be in the nano-to-microsecond range.
And again, a universal compilation target.
This is a joke in the WASM community that WASM is neither Web nor is it assembly, necessarily,
right?
In the spec, there's no mention of the Web itself.
There's no mention of assembly.
It's just, I think it's just kind of a nice combination of words.
So server-side WebAssembly is an interesting point that's come up recently.
And one of the things that I like to think of it as is, it was sort of cloud infrastructure
as penicillin moment, right?
Where there was this technology that was clearly designed to be extremely performance-safe
and be able to be overnight shipped to billions of different machines.
And that's because this is what the browser commands, right?
So this was something that if we took this same idea and we kind of squinted, it kind
of starts to resemble other technologies that we see in other areas.
Some people have made this argument that we went from things like bare metal to VMs,
to containers, and then possibly next, WebAssembly, right?
So we're going to call our safer faster and much more universal.
The faster asterisk, asterisk here I did remember.
And when we talk about faster, again, it needs a runtime.
So you might think that it's not as fast as native code, although sometimes it is.
And if it's not, it's pretty close.
The faster argument here really comes from the fact that you can start out very, very
quickly.
This is a somewhat interesting tweet from Solomon Hikes.
Oh, no.
Of course, I'm not connected to the internet.
I'm sorry.
It was difficult for me to find this room.
It was very, very quick.
If this works.
Yeah.
That's my plan.
Okay.
That's fine.
Well, at least the slides work for now.
Okay.
Well, basically, the founder of Docker.
Oh, no.
Okay.
No.
There we go.
Open source technology comes through.
There we are.
Excellent.
Okay.
So this guy, Solomon Hikes, he's a great developer.
He's a great developer.
There we are.
Excellent.
Okay.
So this guy, Solomon Hikes said this a few years ago, which probably got a lot of VCs
very excited in 2019 that said if Wazemann Wazey had existed in 2008, they would never
need to create Docker.
That's how important it is.
Right?
And this has to do with sort of the fundamentals of the technology, which allow the things
that Docker aims to do and aim to do.
Initially, it allows you to do those same things at a smaller, safer, faster level.
We'll kind of talk about that as we get into this.
All right.
Okay.
Maybe.
Good.
Blah, blah, blah.
Good.
Wazey.
Who's heard of Wazey?
Anyone here?
Great.
Oh, a lot of people.
Good.
So the Wazem system interface, this was initially started as a POSIX interface in 2019.
It's gone through a couple iterations.
The big thing with Wazey is that it's capability-based security.
They borrowed some things from Plan 9, the operating system that came after Unix, trying
to fix some of the mistakes with Unix, to sort of think about resources as things that are
granted permissions to use and act upon.
It's an evolving standard.
We've had Preview 1, Preview 2 actually just got released last week, and Preview 3 will
be in the future.
Preview 2 brings some big things like networking support, which is obviously a big thing if
you'd like to have real applications that run.
Preview 3, which should come out next year, is going to have ASync support, which is also
a very, very big one.
And pertinent to this talk specifically about Python.
And then we will get to 1.0, probably in the next year or two.
Is it required to run WebAssembly on the server?
Do you need Wazey?
You do not.
You just need a WebAssembly-compliant runtime that can at least be compliant with 1.0.
So we've done some things, not always using Wazey itself, just using a standard WebAssembly
runtime on the server.
Okay, so briefly, this is a project that we work on at my company.
This is the scale framework.
I'm just going to bring this up to sort of show what a good fit WebAssembly can be in
very specific areas before I jump into the Python stuff.
So scale is a plug-in framework.
You can also think of it as a serverless function runtime.
Serverless functions, serverless itself has kind of become popular in recent years.
A lot of serverless architectures use containers.
The problem with containers, especially with things like Python and maybe like Ruby and
things that have a slightly larger runtime is that they can be a little bit slow to start
up.
So there's a lot of trickery that has to go into bringing them up to speed, keeping
them hot, moving them around.
With WebAssembly, you can do all this very, very quickly at orders of magnitude of speed
faster and size smaller.
It also allows you to do some very interesting things, which means you can do polyglot
programming, different languages, in the same runtime environment.
So with scale, you can do things like run, rust, type script, go, all in the same WebAssembly
environments.
And we've sort of figured out what we think is a good UX for doing that without having
to do a lot of low-level programming to deal with types and passing them across different
environments.
Yeah, so if you'd like to check that out, feel free.
Basically, it's very simple.
Scale.sh, you do scale new, scale build, scale run, and you can do something as simple as
this.
I know this is go, but it's relatively simple, pretty straightforward.
Basically, it gives you a function.
You do stuff in that function.
You can pass it to a function written in a different language, and you can get information
back.
You can put this in front or behind HTTP requests.
You can use middleware from other languages in other languages.
So we did some interesting stuff with go, where we took go's regex library and swapped
it with rusts.
And using this plug-in framework, we found that the regex speed improved four times faster
than go's native library using rust this way.
And you didn't even have to think about the rust that was happening.
So that was a pretty fun example.
So to continue, blah, blah, blah.
Good.
So building Python.
A lot of people here probably build Python from scratch on native platforms, right?
So interesting thing about Python is you need Python to build Python, which is kind of fun.
Some assumptions that Python has is that it's going to be on a Unix or Unix-like operating
system.
There's going to be a file system.
This is very important to Python in particular.
Dynamic linking.
This is something that certain builds think of.
WebAssembly has no concept of this.
We'll talk about that.
Also, that there are going to be syscalls and a libc and or.
So some of the pain points with building WebAssembly, building a Python distribution with Wasm is
that there's a limited number of supported syscalls, right?
In Wasi specifically, there's no pthreads, right?
So green threads, that's kind of out the window.
That's a tough one.
There's no socket APIs.
This is also a very big one, right?
This makes it very, very difficult if you've ever used things like PyDyn and PyScript.
They can do some interesting things where they sort of overlay on WebSockets to sort
of emulate behavior.
This is done using a tool called MScripten, which is one of the earliest WebAssembly compilation
tools, which allows you to take C and C++ libraries, compile them to Wasm, and then have
bindings in JavaScript to sort of mimic some of the system behavior.
There's non-comprehensive signal support.
This is also a very, very big one.
So if you want more details on this, this could be a whole talk by itself, and indeed
it was.
You should check out Christian Heinz's talk from Wasm Day in 2022.
He is one of the four maintainers of the Wasm Python project, and he, I believe, also works
on PyDyn.
It goes into all these pain points in excruciating detail.
It's an excellent talk.
So, but this talk is about actually deploying something, doing this, right?
Right?
So if you've tried to use WebAssembly, you might have realized it's a little bit hard to use
because there's certain things that are very low level that you have to do, a.k.a. like,
you know, doing things with data, getting data in and out of a running Wasm module,
communicating between the guest and the host, depending on what your implementation is.
So there's a project that I put together.
It's called Boxer, and I decided to try to take something that is well known to most
people in this world, along with some other ones, which is a container declaration, a
Docker file, and plug it into this tool and spit out a Wasm binary, right?
Plus the runtime.
You can check it out, boxer.dev.
It's experimental right now, but I'm going to be demoing it here in a second.
So what is in one of these?
Originally it was called Wasm Boxer, but now I'm just calling it Boxer.
The, what is inside a box?
You have a base layer similar to a container image if anyone here is dug into how containers
work.
The base layer sort of sets up the imports and exports for the Wasm module.
This mimics the sort of interfaces that you might find with syscalls and libc in the traditional
operating system.
Then you also have a virtualized file system, virtualized syscode subs.
So you have actual POSIX based file system calls that actually work in a virtualized
environment, and then you also have things that aren't supported, stubbed out.
You have a compiled runtime, which in this case is Python.
And then you have the user source code, which also gets passed in.
Okay.
And this is very important.
WebAssembly modules only understand about the outside world, imports and exports.
That's it.
You kind of think of it like a very, very simplified inter-process communication from the Unix world.
Okay.
So, I kind of went through this before.
I used a tool called Wiser.
Wiser is a really, really great WebAssembly tool that allows you to combine WebAssembly
modules, link them together, do snapshotting, things like that.
But basically, this taking an example like this, the big caveat here is that the A.
This is from a C binary.
That binary must be a WebAssembly binary.
That's the key difference.
Right?
Cool.
So, for the sake of time, I'm going to kind of move a little quicker.
Python really, really needs a file system, right, when you want to build Python.
Okay.
So, one of the very important things here is that we need a POSIX-based FS.
And you could do this with Wazzy, where you can go down into the underlying host and use the host's file system.
But those could obviously be different across different distributions.
So, one solution to this was building a virtualized file system, which we did here.
This is a small project of mine, also very experimental.
This is the Wazzyn VFS.
And what this gives you basically is very familiar sys call slash live-c calls that you might see across different distros.
And the point here is that you could do this all in a virtualized space.
It could be in the same Wazzyn module, or it could be in a different one.
Right?
So, yeah.
Cool.
The demo.
Yes, let's do it.
So, Wazzyn Boxer.
So, here we have like one of the simplest, this is right from the container registry, right?
From the official Python registry.
So, the very, very simplest thing we could do is from Python 3, set up the working directory, copy from the host's OS to the guest,
which in this case is the WebAssembly build, and then use the command directive to run the actual script.
So, what does this look like?
Basically, looks like this.
This is a very familiar command.
Box build, passing the Dockerfile.
Ignore my Rust warnings, which I haven't fixed yet.
Cool.
So, yeah.
You'll see here, we have the build started.
It found the base image, which in this case is just the sort of interfaces.
It's building and bundling the runtime, the standard library, the source code, and the file system.
This can take a while.
It was cached, so that didn't take as long.
And it bundled it, and it's complete.
Cool.
So, we have that, which means we have our box set up and ready to go.
It's just like the sort of purpose of this, that's just like building a container image, you have it ready to execute.
Then we just do a very simple box run, and that's Python code in WebAssembly.
And what that Python code is, is just a very simple square root operation, right, and printing it out.
So, that's that.
So, from these perspectives, you may not even really notice a difference,
but we'll kind of talk about what the specific differences are in the few minutes I have left here.
So, there are caveats, and we're going to talk through why these caveats are going to sort of improve,
and why things are a little difficult right now, but there are some ways around these, but they are difficult.
Threads, like I said, threads out of the box, they don't work.
There are things you can do, and there are some things that some people have experiments with.
Wasm is single threaded right now by default, and it won't not be for at least a couple years.
But there are ways that you can do some really interesting things with stack switching.
You can do pause and resume on the stack, and you can make it almost as if it is in an async sort of programming environment.
Networking, like I said, there are ways to do networking.
You just have to do it all yourself.
So, you have to bring it all yourself, right?
So, you don't have an operating system, but people have done things like taking the kernel networking stack,
bringing parts of it in, allowing those to be exports as host functions, and then calling out to the underlying OS that way.
Native dependencies, this is a big problem.
This is a big problem that they faced with PyScript.
People who have done scientific computing in this room would probably understand that there are some really interesting languages
being used under the hood in Python libraries, things like Fortran, I believe, right?
And so, there is no official Fortran to WebAssembly compiler yet.
So, what people are doing is, but the thing is, is that if there could be, you could then, if you think about native dependencies
on different platforms, how much of a pain that is, there is a possible way forward in the future,
which could be a different talk maybe, that you have one dependency and it's in WebAssembly,
and that's the only one you ever need because you can have a WebAssembly runtime on every machine, right?
That's kind of an interesting thing that some people are exploring.
So, this is a cool thing I had chatGPT make for me, but it's a nice little illustration.
You got a container, that's an app, and then you have a box, that's an app.
That kind of highlights the differences in metrics, and what are those metrics?
Well, containers, so a Python distribution, a Python container might be anywhere from 800 to 900 megabytes or more.
The start-up speed of this container could be 800 milliseconds to two seconds.
A security model with container run times are shared kernels, and with boxes or with a WebAssembly distribution of this kind,
you can get closer to something like 16 megabytes for the size, right?
Even less if you don't use the entire standard library, right?
I've seen people do it as low as five, I think, when they cut certain things out.
Start-up speeds could be 100 microseconds to about one millisecond for this exact build.
The security model is a virtualized sandboxed machine code execution, right?
So, you're at a different level of abstraction when you're talking about virtualization.
In fact, Docker really isn't a virtualization technology as much as it is sort of a sandboxing framework, in certain ways, right?
Again, that's probably a different talk, but also kind of interesting.
So, the future, what does it hold?
Full support for libc, Cisco interfaces.
There are some really interesting work going on with this, where people have taken parts of the Linux kernel
and made them available in Wasm, doing things like emulating signals,
doing things like emulating threads with, like I said, different techniques.
Then what that gives you is kind of a pluggable networking stack that you can use in different environments,
and maybe even a slight paradigm shift where you might not even need a kernel in some cases,
if it's all in WebAssembly and the WebAssembly runtime acts as the kernel in certain ways.
So, this is an interesting technique that people in the embedded space have kind of done some cool work with.
This is something where if you can compile the, if you can have a compiled Wasm runtime on bare metal,
everything you need is in Wasm.
This is great for people in the embedded space because they don't need to reflash their devices every time they update their code,
which is a very difficult thing to do.
You can just swap out a tiny WebAssembly module.
Wally is also a very interesting one.
This is the WebAssembly Linux interface.
This is a project that came out of Carnegie Mellon just last year,
and they're doing a lot of interesting work on sort of the emulation of Linux syscalls that make them available
so that you can run things sort of out of the box.
And like I said, with the embedded stuff, bare metal runtimes with a tiny little unicernel has allowed people to run
Wasm in really, really small spaces, doing some really, really interesting things too.
And what this does give you, then if we think all the way back to the beginning with browsers,
browsers have a Wasm runtime, servers have runtimes, your phone has the runtime,
you could have the runtime in an embedded device, in a controller, you could have it in all these different places.
So you kind of have this sort of new true isomorphism where you can actually run the same code everywhere,
and then what that does, it kind of pushes the problem to a new interesting space in distributed computing
where it's how do we orchestrate these things, and how do we move them around, and how do we make them available.
Cool, thanks very much.
I don't know if I have any time for questions, but...
Yeah, we have time for... Just a second.
We have time for a few more questions.
I see some over here.
Thank you for the presentation.
Not from a web application perspective, but from more like low level and server side side.
Some parts of the presentation reminded me of this recent thing called unicernels.
Could you comment on the similarities and conceptual inspirations between two things?
Yeah, certainly. I'm not really a unicernel expert, but unicernels kind of saw the similar thing of basically taking the space that you have
and only running what you need.
So from a theoretical perspective, it's kind of like WebAssembly can take the same approach,
and in fact they're being used together in certain cases where a unicernel only is provided for what the WASM runtime needs.
So yeah, it's cool. We can talk more about it later if you'd like.
Anybody else? I know you gotta go.
More questions?
Feel free to talk to me later if you'd like.
Yeah, this side.
Thanks again. Thank you.
Thank you.