[00:00.000 --> 00:11.000] Okay, so our next talk is by Ivan Sraka. [00:11.000 --> 00:18.480] Ivan has commercial experience working in 3D graphics and runtime design with Rust, Nix [00:18.480 --> 00:29.720] and Haskell, and he also does design algorithm competitions and children's coding workshops. [00:29.720 --> 00:34.800] He lives in Belgium, loves biking, hiking, climbing and vegan cooking. [00:34.800 --> 00:40.720] His topic today is on the path of better interoperability with Rust. [00:40.720 --> 00:41.720] Thank you, Ivan. [00:41.720 --> 00:42.720] Thank you. [00:42.720 --> 00:47.720] Hi, everyone. [00:47.720 --> 00:58.720] So this talk is about a thing I worked past few months and there is also a blog article [00:58.720 --> 01:07.040] on engineering.iog.io, I guess, which is basically the content of these slides with links and [01:07.040 --> 01:09.440] references and everything. [01:09.440 --> 01:18.320] And so the IoG issue is there is really large Haskell code base to maintain and some parts [01:18.320 --> 01:25.760] of the code base are C-bits mainly for cryptographic library like Kriptonite. [01:25.760 --> 01:33.880] And so there is a will to more easily integrate Rust library because for cryptography there [01:33.880 --> 01:39.880] is cool Rust library implementation. [01:39.880 --> 01:44.920] And that's not quite simple right now to interface Haskell and Rust. [01:44.920 --> 01:50.760] And so this talk is about how to make the experience easier. [01:50.760 --> 01:57.560] And there is a repository, yeah, so it's the links here, the subtitle where you can find [01:57.560 --> 01:59.760] the source and everything. [01:59.760 --> 02:08.720] And so when we try to integrate to and time, for example, Rust and Haskell in our case, [02:08.720 --> 02:19.440] there is a lot of way to interoperate different programming languages and one of the solution [02:19.440 --> 02:28.040] which is often used is to use, I don't know, sockets or writing on a file or a pipe or [02:28.040 --> 02:33.880] something like that and to use a protocol, for example, protocol buffer from Google or [02:33.880 --> 02:43.360] anything you can use, JSON or an HTTP API, all this kind of stuff, but if you do such, [02:43.360 --> 02:52.000] you will rely on these calls like EOS to make interoperability works and if the library [02:52.000 --> 02:59.840] on which you rely, you want really little overhead to call it to rely on what the library [02:59.840 --> 03:05.640] does, library does, you prefer to use something which is called FFI which is more close to [03:05.640 --> 03:10.880] a system programming which is foreign function interface which is basically jumping the memory [03:10.880 --> 03:20.960] of binary code generated by another languages and hope it will work the right way and to [03:20.960 --> 03:28.640] make it work, it requires special attention and the tools needed to handle all that is [03:28.640 --> 03:37.960] we will take special attention to that right now and if we look at what exists to FFI between [03:37.960 --> 03:44.000] REST and other programming languages because there is such a common way to interact, interoperate [03:44.000 --> 03:51.760] stuff, we can see that from C to REST there is a thing called REST bind gene, from REST [03:51.760 --> 03:57.600] to C there is a thing called C bind gene, REST is, I will not name all them but there [03:57.600 --> 04:04.160] are all about a thing which is bind gene, which is generating bindings, maybe you already [04:04.160 --> 04:10.080] know that but why we do binding generation? [04:10.080 --> 04:18.080] Because FFI is really something dangerous where if your two foreign function interface [04:18.080 --> 04:24.560] didn't match, you will not know, your compiler will not warn you about that, it will warn [04:24.560 --> 04:32.440] you if a symbol is missing but it will not check the types, the number of your argument, [04:32.440 --> 04:38.560] if you respect the same calling convention, it will, so you want this interface to be [04:38.560 --> 04:46.520] generated so you can ensure it match, something really common but it does not exist between [04:46.520 --> 04:53.760] REST and ASCAL so that basically what this token project is about and so here are a bit [04:53.760 --> 05:02.560] of REST code, the way I choose to generate bind gene using macro which work like a function [05:02.560 --> 05:08.920] decorator, so it is a function that does something really simple, just print a low name, it is [05:08.920 --> 05:14.240] string interpolation in REST, here I import my library and here I have my custom thing [05:14.240 --> 05:23.480] which is I tell my binding generation about what would be my ASCAL wanted function signature [05:24.440 --> 05:33.800] and so a macro will expand the code, so it is code generation, macro actually in most [05:33.800 --> 05:40.880] languages so here is a code expanded by the macro, I have no mongrel things because maybe [05:40.880 --> 05:46.800] you know mongling, some things that exist in C++2 is function symbol change, a change [05:46.800 --> 05:53.080] in binary and we want to preserve it, we want binary to have the same symbol to be called [05:53.160 --> 05:59.040] from the website from an ASCAL program and all the other things I want you to look at [05:59.040 --> 06:06.800] is I use extern C here, precise C, that means in REST I want to use the C calling convention, [06:06.800 --> 06:13.240] C, HBI, so HBI is the calling convention and memory type layout which is a bit of the [06:13.240 --> 06:23.040] same part of the same definition, the thing is when you define FFI in ASCAL, I will show [06:23.120 --> 06:28.560] your ASCAL code just after this slide, you don't have the choice, you use the C calling [06:28.560 --> 06:37.080] convention, in REST you have a bit more choice but you can't use, that's not true, what did [06:37.080 --> 06:44.920] you say, you have the choice in ASCAL, never mind, I want just to point out that REST, [06:44.920 --> 06:49.160] HBIs, REST calling convention, the CUL convention, I use REST function inside the binary between [06:49.200 --> 06:56.760] the same, isn't stable, which is things that exist in ASCAL 2, which is a way that REST [06:56.760 --> 07:07.880] internals, the REST core team keep to be able to break the mechanism inside REST C without [07:07.880 --> 07:12.960] changing the measure version of the compiler, so for example the CUL convention changed [07:12.960 --> 07:19.920] several times since the stable version of the language, which is REST 1.0 something, [07:19.920 --> 07:26.400] so if we build a thing on top of REST, HBI, CUL convention, it will not be stable, it [07:26.400 --> 07:32.680] will be an ACC, so maintaining it will be really laborious because it's working with [07:32.680 --> 07:39.640] internal documentation which is not made for people to use it as a public API, and so [07:40.640 --> 07:50.160] why REST macro? I want to point out that because there is a lot of way to the binding [07:50.160 --> 07:55.920] generation and often it works as an external tooling and the issue with the external tooling [07:55.920 --> 08:03.120] is really easy to forget to integrate the external tooling into your build suite and [08:03.120 --> 08:11.120] if you do that your binding will be out of sync with your codes, so your program will [08:11.120 --> 08:16.880] not work, so we want binding generation to be part of the create compilation, create [08:16.880 --> 08:24.480] is a REST module, and so I did it that way, and I also want to point out that there is [08:24.480 --> 08:32.280] other program in REST space like C bind genes that just try to integrate REST code like [08:32.400 --> 08:38.760] search and replace regex, so if you for example in the language have two identifiers in different [08:38.760 --> 08:46.080] namespace, the C bind gene that generates C bind gene for REST will not be able to understand [08:46.080 --> 08:51.000] their meaning, so when it's an undefined behavior, it's part of the limitation of that library, [08:51.000 --> 08:56.280] which is not the part of the bind gene library we present here, that understands REST semantics [08:56.280 --> 09:03.880] because it's implemented as a macro, and so it expands REST code and it has to generate [09:03.880 --> 09:10.800] as a set effect ASCAL code and ASCAL modules that just have the wanted signature, so same [09:10.800 --> 09:21.840] symbol, same signature, things that sync together, so if we relook the REST code, there's another [09:21.920 --> 09:28.600] thing I want to talk to you about, here you see you have the C, the compatible calling [09:28.600 --> 09:35.440] conversion type, the thing I want you to look at is I use trait, trait, rep C, trait, rep [09:35.440 --> 09:43.480] REST, and that's trait REST type classes, that's a way to define contracts for data [09:43.480 --> 09:49.880] structure, you mean a data structure should implement this and this method, so I want every [09:49.920 --> 09:56.560] data structure that has this method would implement those, internally it's implemented as virtual [09:56.560 --> 10:08.400] table, like in C++, and why I use that, I use that to be extensive programmable, so I have nice [10:08.400 --> 10:15.720] error, it's part of REST type system, users can add types to this framework, and for the [10:15.760 --> 10:20.760] types, which is part of the standard library, that's I implement myself, at the care of the [10:20.760 --> 10:27.960] memory management, and we talk about it a bit later, and also the thing is in a FFI in REST, [10:27.960 --> 10:36.520] you can use only what's called FFE safe types, it means types that have a memory layout in [10:36.520 --> 10:41.560] C calling convention, and most of REST type have undefined memory layout for C calling [10:41.560 --> 10:49.320] convention, so the thing is for example if I get REST strings, I cast it safely to a C string, [10:49.320 --> 10:55.440] so that I could represent as C string works, which is another way that REST strings are actually [10:55.440 --> 11:07.960] represented, and so what about GC, the thing is REST have a destructor mechanism based on ownership, [11:08.360 --> 11:15.400] a lot of rusty things, but the idea is more about destructors, like when you go out of a scope, [11:15.400 --> 11:23.640] the destructor of the value is called, which is dropped, and so what I do here is I tell REST [11:23.640 --> 11:32.840] type system to, does not call drop on REST value, because it will live on the Askel side, so that's [11:32.920 --> 11:40.520] the other side of the bindings that need to free the value, which is a thing that you can do with [11:40.520 --> 11:49.480] March Mald, I don't know how to pronounce that library, and the thing is also I learned during [11:49.480 --> 11:59.800] this project the real semantic of safe and then safe foreign call definition, because I finally [11:59.800 --> 12:10.200] understand that unsafe mean you want to play with any object on the Askel heap, which is not the [12:10.200 --> 12:18.200] default case of what we actually do when we pass basic data types, that's our, does not need the [12:18.200 --> 12:29.720] garbage collector to pause, because it doesn't know if we could get an inconsistent state by [12:30.360 --> 12:37.560] some things that the foreign call will do, and so few users in safety will have a warning [12:37.560 --> 12:45.400] that say a cautious, an unsafe call will slow down your war program, are you sure you want to [12:45.400 --> 12:56.840] do that or not and anything, and yeah, and that's a library, a REST library, but those are things I [12:56.840 --> 13:03.960] did and that's also what exists for all of our REST tooling is a key tool that helps you to set up a [13:03.960 --> 13:13.400] project, because you have to tweak your REST build file, you have to tweak your R-scale build file, [13:14.040 --> 13:21.320] actually you have to do a setup.hs build customization in the Askel unless you use NICS, [13:21.320 --> 13:32.760] which I could understand a lot of people who won't do, and on REST side I use build.hs, if you do a [13:32.760 --> 13:40.600] dynamic library, because dynamic library that GAC fetch needs the GAC version in the name of the [13:40.600 --> 13:47.960] dynamic library, so that's the thing that I have to get as a build customization, this kind of tweak, [13:49.160 --> 13:59.080] but overall this library is really small, it's, I'll put the next slide just a few minutes later, [14:00.520 --> 14:07.480] the whole library, the whole tooling I present here is less than 1000 lines of code, so [14:08.440 --> 14:19.160] okay, it's 1000 lines of code, so that's really small, minimalist, and kiss, so [14:23.000 --> 14:26.440] actually I, I forget what I was about to say, [14:27.400 --> 14:38.680] actually you, you, you, all that, all this plumbery is, is simple, and I'm not sure myself, I want to, [14:38.680 --> 14:45.080] to make it more complex, because it just works, and the thing is, it works, but you use really [14:45.080 --> 14:52.920] simple data types, the C data types that are representable in Askel and REST, and there is a [14:52.920 --> 14:59.720] lot of things that are not representable in C, for example, REST slice, a slice is a pointer [14:59.720 --> 15:04.600] and a size, and you have a guarantee that there is memory in it, you can iterate on it, [15:05.400 --> 15:11.800] you can represent it in C with a strip, but it's not really a C type, you will do a custom strip, [15:12.280 --> 15:22.440] and, and those things are targeted by the REST RFC, which is named interoperable ABI, [15:22.440 --> 15:27.240] which is creating an ABI which is stable, which have more REST [15:28.280 --> 15:36.280] C things than the C ABI, but which is flagged as stable, which is not the case of the REST internal [15:36.280 --> 15:47.080] IBI, and the other thing is cabal customization is a bit of a mess, doing the setup that [15:47.080 --> 15:55.000] hashes to find the library and anything to make it work, because of cabal bugs, so I really love to [15:55.000 --> 16:03.320] have a standard way to integrate, for example, REST chain into cabal, here you have to run the [16:03.320 --> 16:13.800] two-tile chain in two different states, and that's it, I can throw a lot of questions, [16:13.800 --> 16:22.840] or I can do a demo if my talk is shorter than what I was expecting, so that's the date to [16:22.840 --> 16:36.120] ask questions, or, and, yeah. Yeah, I can do, I can do a quick demo, sorry, [16:40.920 --> 16:45.960] it's not, it's not easy because my, it's not mirroring, so I have to look at the, the screen [16:45.960 --> 17:00.120] why I, I doing this thing, what I can do, what I can do, yeah, [17:00.120 --> 17:14.280] yeah, [17:14.280 --> 17:22.280] did you, up, [17:22.440 --> 17:30.440] yeah, [17:42.920 --> 17:49.080] it will be unsafe on the Askel side, if you do that, for example, because you, you, will it be, [17:49.080 --> 17:54.920] I, I, it's an interesting example, I didn't try [17:58.600 --> 18:07.640] passing a function pointer, the thing is, there is, the thing is, you, you, if you want to pass [18:07.640 --> 18:13.800] complex data types, you, it's always more meaningful to use serialization, because it's, [18:14.040 --> 18:22.600] it's, it doesn't match the two language model, but function, I didn't honestly think about it, but I, [18:25.720 --> 18:30.120] you mean a function pointer that will cross the FFI barrier? [18:30.920 --> 18:32.840] So that's the whole question, right? Yeah, yeah, yeah. [18:34.120 --> 18:40.360] In Erlen, for example, you have no concept of sharing pointers, you copy the whole data, [18:40.360 --> 18:45.320] and use it to the other node, including function, which are serialized, and then reinterpreted [18:45.320 --> 18:51.720] on the other node. I guess it will not, not really be an issue, [18:54.840 --> 19:01.080] because, because in fact, there's no really boundary at the end, that's the one [19:02.360 --> 19:09.960] binary, so the only things that I, I want my mental model to, to have a better grasp [19:09.960 --> 19:17.480] around it is how GC, as a GC will behave in this case, but I think there is no [19:17.480 --> 19:26.760] special issue for doing that, but I'm, I think I should experiment on it first to, to, to, [19:26.760 --> 19:32.680] before saying it's, it's completely okay to do that. Yeah, thank you for the question. [19:33.080 --> 19:35.880] Uh, yeah, I, [19:45.880 --> 19:46.840] did you see something? [19:48.200 --> 19:48.440] Fuck. [19:48.440 --> 19:52.200] Fuck. [20:01.320 --> 20:04.600] So, sorry. [20:07.000 --> 20:17.320] So if I go here, for example, I have a little resting, SRC. [20:19.160 --> 20:28.680] Um, yeah, I can show you the lib.rs, which do, uh, cryptographic primitive, for example, [20:28.680 --> 20:35.560] and you, what you, you manipulate is clearly, uh, rest types, it's not C types, [20:37.160 --> 20:44.760] and I have a warning, and I don't know why, uh, yeah, because I don't install it first, [20:44.760 --> 20:52.760] and if I do, I guess it will work. I hope so. Uh, I can check what's happened with, after [20:52.760 --> 20:59.160] some macro expansion, uh, will it work? [21:07.800 --> 21:08.200] Yeah. [21:10.840 --> 21:14.920] Sorry. Uh, DEMO effect. Uh, yeah. [21:15.880 --> 21:19.880] Hmm. Why? [21:26.840 --> 21:32.680] I, I don't know why you do that, but maybe I'm no internet connection and that's the issue here. [21:33.480 --> 21:38.680] Yeah. I guess that's the issue. So never mind. I can show you the, [21:39.640 --> 21:45.400] the askels things generated. So, no, that's a cabal file generated by the client, [21:46.440 --> 21:53.160] and so, uh, I have, it looks like that. Uh, and I have, [21:57.240 --> 22:02.040] yeah, it looked like that. I have my, my code generated. [22:02.280 --> 22:13.800] Uh, and what can I show you? I can show you the build customization. You can see what it looks like. [22:14.360 --> 22:19.640] Uh, it looks like it. It's not really interesting, but it do a few things. [22:22.280 --> 22:29.640] And what can I show you? Uh, I have a little file that's unsure that's a client you use to [22:29.640 --> 22:35.800] generate the whole stuff is, uh, compatible with the version of the library you use, [22:36.360 --> 22:40.520] due to, if I want to change the whole Bay of York in the future. [22:43.400 --> 22:47.240] And yeah, on the askel side, it would look something like that. So I have a project, [22:47.240 --> 22:51.960] I have a cabal project, right? Uh, that looks like that. [22:52.920 --> 22:57.960] Uh, and the test thing just work like, [23:01.800 --> 23:05.880] that I have, uh, a test dot cabal. [23:10.760 --> 23:19.320] Uh, it looks like that. Or I can use it like it was a normal askel dependencies and my, [23:20.120 --> 23:28.920] uh, askel code is quite simple, I guess. Yeah. Uh, so it's fixed me, but don't look. [23:31.800 --> 23:40.280] Yeah. Uh, yeah, you have something like that. It's, it's always a bit, uh, you, you, you manipulate, [23:41.000 --> 23:48.520] um, low level data structure, but that's, that's often what you want to do when you, you, you [23:48.520 --> 23:55.000] brought something in the system programming language. That's one reason to use rest over [23:55.000 --> 24:02.360] something else. Otherwise, uh, um, but I don't know. I, I, I got the input that's [24:03.240 --> 24:11.480] the, the bind gen don't really need, don't sorry, don't really need, uh, more advanced data types. [24:11.480 --> 24:18.440] For example, OCaml have an interop, uh, OCaml interrupts things with rust. That's, uh, helps to [24:18.440 --> 24:29.160] represent, um, ADT, both rust and OCaml. And I don't know if we really need to have ADT conversion [24:29.160 --> 24:37.160] between languages for most of use case. I don't know. Um, yeah, uh, that's, that's many. I don't [24:37.160 --> 24:43.640] know if you have other question or just curious about something or, or another. [24:46.920 --> 24:48.200] Yeah. Okay. There's a question. [24:51.960 --> 24:58.200] So, uh, you said at one point that, uh, rest will not drop the value and that's to, [24:58.200 --> 25:07.400] for our scale to, to free. So how does that work? Is that done by the, by the GC or does it need to [25:07.400 --> 25:18.040] be done manually? Um, you mean how I tell rest to not drop a value? No, if, if we do, if rest [25:18.040 --> 25:25.240] doesn't drop the value, who does? Is it the, the, uh, the scale GC or, or somewhere we need to, to [25:25.240 --> 25:31.720] advise, you know, you know, program? Yeah. Uh, the, the idea is because it's part of, uh, the [25:31.720 --> 25:39.400] marshmallow things, you will have to free it explicitly as I understand it. If, if we do not [25:39.400 --> 25:46.440] survive with leave, uh, maybe get some, somebody could tell me if I, if I'm mistaken in this [25:46.440 --> 25:55.240] point, it's really hard to, to, to, to debug what actually GC does. Uh, so far he didn't [25:55.240 --> 26:03.880] experiment so much on that, but, but yeah, the real thing is really that if you, if you do not [26:04.440 --> 26:09.560] force for us to, to not drop things, it will, because it's part of its type system, it will, [26:10.200 --> 26:17.880] it will put code to, to free memory, to free of the memory allocator you used, uh, straight into [26:17.880 --> 26:24.600] the code. Uh, rest, uh, statically decide where to put allocation and release of memory [26:24.600 --> 26:30.680] inside the bin and binary. It's a computed as, uh, statically at compile time. So you have to [26:30.680 --> 26:36.920] tell type system to, to not do it for specific types if you, there's an internals for that. [26:37.800 --> 26:45.880] Uh, but on Askel side, I, I, I'm not sure, I'm pretty sure, unsure that the GC will, uh, track it, [26:46.680 --> 26:54.600] um, as, uh, as a Gaboch collected object, I, I think, because you have to, to do it explicitly. [26:55.560 --> 27:02.840] Yeah. But that's one of the, the point, I'm a bit unconfident, so I want to, to check again, [27:02.840 --> 27:09.400] it's this kind of scenario. Yeah. Um, do you have, was there a question? [27:12.200 --> 27:16.840] What do you do if there's an external Rust library that you don't have control over, [27:16.840 --> 27:23.560] um, because they might not want to add HS, bind gen macros to their code base? Is there a way of [27:24.600 --> 27:32.600] generating a shim or something? Um, okay. In fact, it's really easy in Rust to do [27:32.680 --> 27:40.280] reexport. So you can always create a new crate, reexport a crate as you, you say, uh, I depend [27:40.280 --> 27:46.280] on this crate and reexport the thing I want to reexport and I, I, I, I decorate or not [27:47.000 --> 27:53.160] element. Yeah, that works on functions that exist in the other library. Sorry, you can you put [27:53.160 --> 28:00.360] those macros on the functions in the other library then? Yeah. Yeah. Yeah. Um, in fact, it's just, [28:00.360 --> 28:05.720] just wrapping function by function. So, so you can do whatever you want. Um, [28:07.800 --> 28:14.360] and yeah. Uh, and you can opt in or opt out some function of a library and some not, which is, [28:15.080 --> 28:20.920] to me, is a, is a advantage of using macro and function decorator over, uh, code, [28:21.880 --> 28:28.440] uh, wall code, uh, parsing. That's not free though, is it? That's not. So if you're, sorry, [28:29.160 --> 28:33.720] so if you're, you're reexporting functions but you, but are you actually reexporting the same [28:33.720 --> 28:42.040] function or are you exporting a wrapper around the reexport? Yeah, but I, I, okay, I, in Rust, [28:42.040 --> 28:50.200] there is a clear, uh, um, idea of, uh, what symbol I want to expose and what symbol I don't want to [28:50.280 --> 28:57.640] expose. So I, I'm pretty confident that the compiler has the library to inline or not things. [28:57.640 --> 29:03.800] So for example, most of the trade implementation I do, which is casting, are inline. I explicitly [29:03.800 --> 29:08.760] say I want it to be inline. And so, yes, it's a function that's called a function, but at the end, [29:08.760 --> 29:18.520] it's just, it's, uh, have no, no runtime cost. So yeah. Okay. We're out of time. Thank you. So [29:18.520 --> 29:26.840] thanks very much, Yvonne. Let's give him a round of applause.