Integrating some Rust in VLC media player
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Couprie, Geoffroy
VLC media player has a nice goal for users: handle almost any file or protocol you throw at it. Unfortunately, this results in a lot of parser vulnerabilities, because most of the parsing code is handwritten in C. By leveraging Rust and the nom parser combinators library, could we replace some security critical parts of VLC?
 I'm Jophar, I'm really happy to be here in Pittsburgh. It's my first time in Pittsburgh. I'm going to talk to you about a project that's been going on in my life for quite some time now and was the reason I initially came to Rust. So I'm pretty, pretty excited to talk to you about that. So first of all, I work at Clever Cloud, which is a French platform as a service hosting company. So just kind of like Iroku and others, you get push or code, it runs. We've been doing some Rust a lot these past few months. We have actually a lot of code in production that's using Rust. We have replaced big parts of our infrastructure. And I'm really happy to announce that now you can deploy Rust application on our platform. So it's just basically a cargo run on the cloud, push on as many instances you want. It's in virtual machines. It works with stable. I tested it this morning. It was really, really cool to use. So I'm really, really excited about that because it takes some time to support new platform and like Rust was so easy to get there. It's amazing. So let's talk about this little project that I really like in which I worked for a few years. So video run makes VLC media player, which is a media player. And the idea of VLC is basically you drag and drop any file and it will work. It should work with almost any format HTTP, HTTP as input or file, MP4, MKV, any codec, anything. It should work. So it's a pretty big goal, but it kind of works. But there's an issue with that when you try to handle as many formats as are existing that you got vulnerabilities. So those are few ones we got in the past few years. The MP4 demuxer, the MP4 parser is really interesting because it's not, it does not look like very complex as a format, but it's very, very ambiguous, very hard to get right because there are lots of competing implementations. So VLC is made in C. It's like hundreds, thousands of lines of C everywhere. And all of this, all of the passing code is written in C. So we get manual parsers where people just try to interpret the specification, do pointer arithmetic by hand. We have formats that are very, very ambiguous, very complex to get right. And so at some point, I thought maybe there's a better solution for writing parsers in VLC and make the whole application safer. So that's when I went looking for something like, you see where I'm going. I wanted something that's memory safe in which you can write correct code easily and that you can embed in C because otherwise I tried to propose Haskell at some point and they said it was not a good idea to have a garbage collected language in media replication. It's kind of cool, really. And that can work in streaming. You got data coming and coming and coming and you have to pass that very, very fast because it should not like block the rest of the decoding. So I already have one thing. We have memory safe stuff. Can we get good parsers with Rust? Well, yes. But I don't know if you've been going to the fuzzing workshop yesterday, but even if you use Rust, you will get crashes in parser when you write manual parsers because it's still hard to get right. So that's why I started to work on this little project called NOM with the idea that it should be easy to write a good parser and it should be a fun process. So NOM basically is two things. It's the approach parsing called the parser combinators. The idea is that you have a lot of small functions, very small, very testable, like you can unit test any part of your parser. And you combine them in larger parser, in larger recognizer, and then you get the whole format. And to do that, there are different approaches to parsing in Rust and long story short, when I started, Rust was very, very young and macros were a very, very good solution for that and I think they are still. But you will see how well it can get. So basically Rust, NOM, those are just functions that take an input type, an output type, and a error type. And when you get input into a function, it will get ever incomplete to say, okay, I need more data, which is very useful when you're doing streaming, like maybe you need to fill up the buffer a bit more, an error. And then, don contains the remaining input, what has not been consumed, and the output data. So it's really simple. All of the non-passers have this format. This is why I can combine them very easy because they all follow the same type. So yeah, macros are hard, right? Yeah, not so much. This is a parser that will recognize alphabetic characters that are terminated by digit characters and will return the alpha characters. So this will make a function that takes byte slices input and returns the byte slice. So it's quite easy to write. When you expand that, the macro generates a function that takes, as I said, byte slice as input, byte slice as output, with the same lifetime and everything. So I only get slice of the input data. I don't copy anything. Zero copy parsing, yeah, fast. And basically, it's just a big list of pattern matching. So it's not so bad, right? Well, actually, it looks not like that. But it's not that bad as well. It just needs to be a bit more explicit in the way it does the pattern matching. Lots of interesting features. So it works on byte slice. It works on strings, on bit streams. So like you have a format that works like on an even, an aligned size of a bit, like someone asked a few weeks ago, I want to parse a list of 11-bit integers. Yeah, Bitcoin. OK. And you have the combinators. So like there was just before terminated, parsed the first thing, parsed the second, returned the results for the first. And there are lots of different combinators. Like many will try to apply again and again and again and return a vector of the results there will generate a top-up with first result for second result. We will apply a parser on the input, see if it works correctly, but we'll not consume anything. Just take a look at the data and say, OK, it's all right. It's what I recognize. I use the regex create so we can put regex in some parsers. And there's some pretty cool error management stuff. So like here, basically there was this input from MP45 and I got some name of parsers and I see which one was applied on which part of the input. So this is something you can do in NUM and it's really, really usable. It works on all REST versions. Also in Stack Extension, no impaltrate. I would really like to have impaltrate when I started NUM, but it was, I don't know, REST 0.89. It was very, very old. There's no STD. It can work on par with C parsers. Like if you want, you can micro-optimize your parser and it will still be safe and it can be as fast as C parsers. It's really, really interesting to do that. It works with streaming. And as I said, it's just function, you can write your own. It's very easy. Now, I'm doing NUM 2.0. I'm working on a lot of stuff and I'm doing a lot of features. White space parsing. So you just wrap with WS macro and it will interspace space parsers between everything. So this is for JSON parsers. So it will try to pass a space, then a string, then a space, then a tag, then a space, then a value, then a space. And you don't have to write space everywhere because it was very, very annoying to do. So now it's automatic. There's a permutation parser. So you have a list of parsers and you can apply them in any order as long as all of the results are there and it will return the results in the order you specified them. So it's really, really cool to use for some formats like PNG. And you have just a new syntax for those who used NUM before. Because the chain stuff, which is very ugly to apply parsers in sequence. So I have do pass which can apply parser, take the results of parser and store it in variable that's usable elsewhere and that you can even return. So here we recognize the 42 integer and then we take the length, which is in our byte, and we take that many bytes and return those bytes. It's a very common pattern in binary formats to have attack that you recognize and then the length and then how many bytes you have today. And after that, a few things. We have custom input types. So no more byte slice or string limits. You can do that on anything else. And like you can work with ropes where you have a structure that has a contiguous buffer abstraction, but it is really a lot of different buffers. So NUM can work with that. And we can do some pretty vector stuff with this. And that's it. No. Case independent parsing and big, big performance gain for some parsers because I simplified your management if you don't need to do all of the interesting stuff where you want to know exactly which parser got to which part of the input. Most people don't read that. So I got a really simple way to get faster parser. So let's get back to the meat of the problem. VLC. So do you know how a media player works? Basically there's a common pattern of a pipe. You have the access. You get a file. You open the file or you open a network feed. You pass through the demuxer. The demuxers are the parsers. And you get multiple streams like an audio stream, a video stream, a subtitle or anything else. And you send them to decoders. The decoders will generate data that can be filtered like you can have post processing stuff on the video, grayscale or even scaling exactly that kind of thing. Then you send them to the output of your machine or re-encode and you can mix the streams back to another file or another network feed and you can send them, send that back. And most media players work like this with just a pipe of stuff. And the hard thing in that is synchronizing everything. Like you have the audio and the video and the subtitle stream and they all go at the same time but they don't decode as fast. Like it's faster to decode audio than video. And in the end you have to get everything at the same time when you present to the user. Sometimes we get that right. Sometimes we don't. So the way VLC is made is you have applications like VLC, VLMC which is video editing software. They call into LibVLC which is a nice user interface to build media players. And LibVLC core is the manager which does everything like load modules, provide the audio access, synchronizing everything, etc. and all of the modules linked to LibVLC for common features, they're all DLL. Like LibVLC, LibVLC core and all of the VLC modules, they're all dynamic libraries. And the modules linked to LibVLC for the features and LibVLC core loads all of the libraries to see how they work. So it seems to make a module in Rust we have to do like a bit of dance with the way DLLs are loaded. A module, so it's dynamic library loaded by VLC core. They have to expose three functions because like VLC core just try to load library, take that function, call it, it exposes some metadata on the module and then knows what the module can do. So the plan to start passing stuff with Rust in VLC is to make a DLL in Rust that acts just like a CDLL, you can load directly in the program that you can build with cargo and that will work just like a C module in VLC. So take the headers, reproduce the stuff we need, link to LibVLC core, get the functions we need, reproduce the module stuff, start writing a parser because that's kind of why I'm here and we'll start passing stuff. So VLC is written in C. The VLC command members is kind of macro stuff in C. Well, let's come back to macro I think. There's kind of object like interface in VLC where there's a command in a written stuff. So VLC command members see that like I inherit some attributes from the VLC object. Lots of things that can be a bit hard to represent like the union types and nested structures and everything. So the first thing you have to do is try to write that in Rust. So if you were at Rust 1 when I was talking about that, I said bind gen just failed on the VLC headers. Since then I tried with several bind gen and it's able to handle the VLC headers so I'll soon be able to generate the whole bindings like that with bind gen. It should be a lot easier to do. But in the meantime, I wrote those by hand. Basically you make Rust structs that kind of acts like C structs with pointers everywhere and you try to make that a bit safer. So you import functions like this one to take a stream and get a buffer of a specific size. You wrap those in functions that you can use from Rust correctly. When you try to do FFI, you want to isolate all of the unsafe parts because you don't want to sprinkle unsafe everywhere you could. So a bit of work to get the FFI running like all of the structures I need because like a stream T reference another struct with a reference another thing like you saw in there that if I want to have the VLC object T, I need to have the DMAX T, I need to have the libVLC and the VLC object T, et cetera. So lots of preliminary work. And then we can start writing our module. We have the interface, we can import code. And this is how you declare a module in VLC. Again, it's a macro to say, okay, this is the name of my module. This is what it can do. You can take an input stuff and you can parse it. And here are two functions, open and close, that you can use to interact with that module. And libVLC code just takes that and loads the module afterwards, knows what to do with that module. So when you expand that, it's got some C code that's big to know how to write. So yeah, basically it's just writing code. Let's just write that in Rust. It's easy to write, it's very, very easy to read, right? No, well, macro that stuff. Yeah, I think there's a common theme there that I really like, macros, I really can help you. But in that case, it's really useful to do. So you see, I declare the function VLC entry, like the way it's done in C, you have the VLC entry function there. So this is how the libVLC module will be loaded. And from there, okay, you can start passing stuff. Flash video, it's a format that's very simple, like it took me less than two hours to write most of the format. It got audio, video, a few codecs that are mostly outdated. And here's the beginning of the format, like you have FL and V, the skip charge at the beginning, then a version number, then some flags to indicate if you have audio, video and an offset that indicates when the video and audio streams start in the file. This is interesting because you can have the headers and the offset points to verify in the file and you can add some stuff between the header and the data. There has been some very, very funny stuff done like that. So yeah, I said earlier, okay, there was change in number one. Now we have do pass, which is a bit easier to use. So VLC, when it gets to file, it will try to call your open method in your demuxer and say, okay, I have some data. Tell me if you can pass that. So the first thing is you do a stream peak. You get, okay, I need nine bytes from the beginning of the data. And I will tell you if it's something I can pass. And then I call on that slice the header function that we just designed here. And if it passes correctly, it returns a header and we can say to the file, okay, now I know and I can pass that stuff. And we do a stream seek to go to the offset we want. So it's very easy to interact with VLC like that. And so since we know we can be the demuxer for that, we pass the demux and call. And control functions to VLC to say, okay, those are the callbacks you need to use. So call me again and again and again until I pass all the data. So when you reach that offset, most of the video formats, they have multiple streams in multiple blocks. Like one block of audio, one block of video, one block of subtitles, et cetera, that will be interspersed depending on how the encoder did the stuff. Like there's something for MP4 files where if you took a look at the way people were writing them at the beginning, they put just at the end of the file the header you need to see. So you had to go to the end of the stream to do anything. And now they saw that you can mess with the spec and put that header just at the beginning of the file and it will not run on anybody. And so lots of very weird stuff in video format. Like everybody has good ideas about how a video format should be and they're all wrong, basically. Like, yeah, I can't criticize anything but like if I design my own, I do something shitty as well. So, yeah. Something interesting there. I create a structure in Rust and I put it in the box and I'll be given back that structure when I'm called afterwards because I cannot want to store some data for my demuxer and this is the way I can interact with VLC. So when I get to the first block, I decode a tag that says, okay, it's audio, it's video or script. And I have the size of the data and a few things like the timestamp indicates when I need to present that part of the data. So if it's audio, it's at second one and if it's audio, it's at second one. I know I have to present both of them at the same time. Okay. So I declare my demux function, which is kind of the way you would do it in C. I read my header, so it's 15 bytes. If I have four bytes, I know it's the end of the stream because I have just a beginning saying the tag type and the size and then I will try to pass everything. So this is how audio header works. So you would have in the tag, you have first the type, then the data size and everything and then you have the format specific data. For the audio header, it's one byte and says the kind of codec you have, the size and everything. So bitstream passing in num. You can take your input, the backslice, you apply bits, which transforms that to backslice and an index and you go through that thing. So there you take four bits and it gives you an integer that you can use to see which kind of audio codec it is. And then you use that. So you take one byte with stream read and then you pass and then you have enough data to know, okay, you have this block of data size minus the header one that contains audio data that I can push to VLC. So the interesting thing in there is Rust never owns the data. Like I can ask VLC for some data but I don't manage its lifetime. I'm not the one allocating, I'm not the one deallocating, I'm just borrowing data from C and trying to not break stuff. It's very important because like most of the time when you interact with C code, the C code thinks it knows better. And I don't agree with that but we have to play nice. So, and don't worry about the unsafe stuff, it's because there are some functions that take VA list, VR, ARGs, stuff and we have to mess a bit with how functions are declared in Rust. But so this is really simple because like in that parser, I only read the headers. I take the file header, I jump to the next block, I get the block header which is like 10 bytes if it's audio and then I say, okay, there's this much data you can use and I'll be called again with the next block. So I never read much data with none in this. So it's quite fast. So don't take my word for it, I will show that it works. So I really like that GIF. Okay, maybe I'll be able to put that, oh yeah, just, if I get it. The thing is the more you see that GIF, the more it's funny. Okay, okay, stop. So yeah, I will just build and launch and run and copy that in a VLC installation. I get, it's a very old advertisement for Zelda in EFLV. You see that it's decoded with the Rust plugin there. The graphics are pretty outdated. So yeah, this is Rust, this is Rust in VLC. So it's very interesting because like, I've been spending, okay, I moved the stuff. So stop watching the GIFs. But spend so much time trying to pass stuff and trying to talk to see and everything and then it works in VLC. The first time you will get the video to launch is such a good feeling really. But yeah, now I have to integrate that really in VLC because it was just a plugin that I built separately and that copied just in a VLC installation. So now the build system. Can we build something with AutoTools because Cargo thinks it knows better how to build everything and I think it's right. But the AutoTools know better and you have to play nice again. So how does it work? First, you have to check if there's a Cargo and Rusty. So this is Autoconf stuff. It's pretty nice. I did not write that. I'm really bad with the AutoTools but it's all right. And then the idea is that you don't build the dynamic library yourself. You build an object file, the Rust plugin.o because there's also a live tool which knows better than you how to build a dynamic library. Like Cargo knows how to build a library for Windows, Mac, Linux and don't care, it works. But play nice with the build system. Play nice with C, play nice with the interface, play nice with the FFI. And it's annoying but like Rust makes it kind of easy to do. So I'm really happy with that. And on this, so the build system is the biggest issue. The unsafe APIs, you can work around them and then get a really nice code to use when you do FFI. The parser is the yes part and it does not really need to be fast because like video decoding is so much slower than parsing that you really don't care about that. And remember that you have to be a nice citizen. You're not on your land. You're using someone else's data so you have to be nice with the way the C handles it. So from now, if you're fixated to do, I have to see how to seek in a file like move back and forth and everything. I have to put it back in the VLC repository. I can build it now inside VLC so it's quite cool. Predominant dependency will be something that's pretty interesting because with VLC we have this big archive with all of the libraries we need. And so for Rust, I'd like to have that as well. And then to complete the imports, I have to play again a bit with bind gen and we'll be able to replace some plugins, some VLC plugins with Rust. And like you can do that with any C project. It's easy to replace a C project file by file. Well, easy. For different reasons, different definitions of easy, but it's doable. It's really doable. So a few people that helped me, Guillaume Gomez, maybe you talked a bit about documentation, Luca Barbato, who works at LibEvi on the LibEvi project and they helped me making this. So thank you. That's everything you need. If you want to make your own module, here's the library extracted from the project and you want to see the modules there. Play with Rust, play with C. Deploy C in production, it's cool. Do not deploy C in production, please. Because then I have to break it. Okay. Thank you. Thank you.
 I'm Jophar, I'm really happy to be here in Pittsburgh.  It's my first time in Pittsburgh.  I'm going to talk to you about a project that's been going on in my life for quite some time  now and was the reason I initially came to Rust.  So I'm pretty, pretty excited to talk to you about that.  So first of all, I work at Clever Cloud, which is a French platform as a service hosting  company.  So just kind of like Iroku and others, you get push or code, it runs.  We've been doing some Rust a lot these past few months.  We have actually a lot of code in production that's using Rust.  We have replaced big parts of our infrastructure.  And I'm really happy to announce that now you can deploy Rust application on our platform.  So it's just basically a cargo run on the cloud, push on as many instances you want.  It's in virtual machines.  It works with stable.  I tested it this morning.  It was really, really cool to use.  So I'm really, really excited about that because it takes some time to support new platform  and like Rust was so easy to get there.  It's amazing.  So let's talk about this little project that I really like in which I worked for a few years.  So video run makes VLC media player, which is a media player.  And the idea of VLC is basically you drag and drop any file and it will work.  It should work with almost any format HTTP, HTTP as input or file, MP4, MKV, any codec,  anything.  It should work.  So it's a pretty big goal, but it kind of works.  But there's an issue with that when you try to handle as many formats as are existing  that you got vulnerabilities.  So those are few ones we got in the past few years.  The MP4 demuxer, the MP4 parser is really interesting because it's not, it does not look like very  complex as a format, but it's very, very ambiguous, very hard to get right because there are lots  of competing implementations.  So VLC is made in C. It's like hundreds, thousands of lines of C everywhere.  And all of this, all of the passing code is written in C.  So we get manual parsers where people just try to interpret the specification, do pointer  arithmetic by hand.  We have formats that are very, very ambiguous, very complex to get right.  And so at some point, I thought maybe there's a better solution for writing parsers in VLC  and make the whole application safer.  So that's when I went looking for something like, you see where I'm going.  I wanted something that's memory safe in which you can write correct code easily and that  you can embed in C because otherwise I tried to propose Haskell at some point and they  said it was not a good idea to have a garbage collected language in media replication.  It's kind of cool, really.  And that can work in streaming.  You got data coming and coming and coming and you have to pass that very, very fast because  it should not like block the rest of the decoding.  So I already have one thing.  We have memory safe stuff.  Can we get good parsers with Rust?  Well, yes.  But I don't know if you've been going to the fuzzing workshop yesterday, but even if you  use Rust, you will get crashes in parser when you write manual parsers because it's still  hard to get right.  So that's why I started to work on this little project called NOM with the idea that it should  be easy to write a good parser and it should be a fun process.  So NOM basically is two things.  It's the approach parsing called the parser combinators.  The idea is that you have a lot of small functions, very small, very testable, like you can unit  test any part of your parser.  And you combine them in larger parser, in larger recognizer, and then you get the whole  format.  And to do that, there are different approaches to parsing in Rust and long story short, when  I started, Rust was very, very young and macros were a very, very good solution for that and  I think they are still.  But you will see how well it can get.  So basically Rust, NOM, those are just functions that take an input type, an output type, and  a error type.  And when you get input into a function, it will get ever incomplete to say, okay, I need  more data, which is very useful when you're doing streaming, like maybe you need to fill  up the buffer a bit more, an error.  And then, don contains the remaining input, what has not been consumed, and the output  data.  So it's really simple.  All of the non-passers have this format.  This is why I can combine them very easy because they all follow the same type.  So yeah, macros are hard, right?  Yeah, not so much.  This is a parser that will recognize alphabetic characters that are terminated by digit characters  and will return the alpha characters.  So this will make a function that takes byte slices input and returns the byte slice.  So it's quite easy to write.  When you expand that, the macro generates a function that takes, as I said, byte slice  as input, byte slice as output, with the same lifetime and everything.  So I only get slice of the input data.  I don't copy anything.  Zero copy parsing, yeah, fast.  And basically, it's just a big list of pattern matching.  So it's not so bad, right?  Well, actually, it looks not like that.  But it's not that bad as well.  It just needs to be a bit more explicit in the way it does the pattern matching.  Lots of interesting features.  So it works on byte slice.  It works on strings, on bit streams.  So like you have a format that works like on an even, an aligned size of a bit, like  someone asked a few weeks ago, I want to parse a list of 11-bit integers.  Yeah, Bitcoin.  OK.  And you have the combinators.  So like there was just before terminated, parsed the first thing, parsed the second,  returned the results for the first.  And there are lots of different combinators.  Like many will try to apply again and again and again and return a vector of the results  there will generate a top-up with first result for second result.  We will apply a parser on the input, see if it works correctly, but we'll not consume anything.  Just take a look at the data and say, OK, it's all right.  It's what I recognize.  I use the regex create so we can put regex in some parsers.  And there's some pretty cool error management stuff.  So like here, basically there was this input from MP45 and I got some name of parsers and  I see which one was applied on which part of the input.  So this is something you can do in NUM and it's really, really usable.  It works on all REST versions.  Also in Stack Extension, no impaltrate.  I would really like to have impaltrate when I started NUM, but it was, I don't know,  REST 0.89.  It was very, very old.  There's no STD.  It can work on par with C parsers.  Like if you want, you can micro-optimize your parser and it will still be safe and it can  be as fast as C parsers.  It's really, really interesting to do that.  It works with streaming.  And as I said, it's just function, you can write your own.  It's very easy.  Now, I'm doing NUM 2.0.  I'm working on a lot of stuff and I'm doing a lot of features.  White space parsing.  So you just wrap with WS macro and it will interspace space parsers between everything.  So this is for JSON parsers.  So it will try to pass a space, then a string, then a space, then a tag, then a space, then  a value, then a space.  And you don't have to write space everywhere because it was very, very annoying to do.  So now it's automatic.  There's a permutation parser.  So you have a list of parsers and you can apply them in any order as long as all of the results  are there and it will return the results in the order you specified them.  So it's really, really cool to use for some formats like PNG.  And you have just a new syntax for those who used NUM before.  Because the chain stuff, which is very ugly to apply parsers in sequence.  So I have do pass which can apply parser, take the results of parser and store it in  variable that's usable elsewhere and that you can even return.  So here we recognize the 42 integer and then we take the length, which is in our byte,  and we take that many bytes and return those bytes.  It's a very common pattern in binary formats to have attack that you recognize and then  the length and then how many bytes you have today.  And after that, a few things.  We have custom input types.  So no more byte slice or string limits.  You can do that on anything else.  And like you can work with ropes where you have a structure that has a contiguous buffer  abstraction, but it is really a lot of different buffers.  So NUM can work with that.  And we can do some pretty vector stuff with this.  And that's it.  No.  Case independent parsing and big, big performance gain for some parsers because I simplified  your management if you don't need to do all of the interesting stuff where you want to  know exactly which parser got to which part of the input.  Most people don't read that.  So I got a really simple way to get faster parser.  So let's get back to the meat of the problem.  VLC.  So do you know how a media player works?  Basically there's a common pattern of a pipe.  You have the access.  You get a file.  You open the file or you open a network feed.  You pass through the demuxer.  The demuxers are the parsers.  And you get multiple streams like an audio stream, a video stream, a subtitle or anything  else.  And you send them to decoders.  The decoders will generate data that can be filtered like you can have post processing  stuff on the video, grayscale or even scaling exactly that kind of thing.  Then you send them to the output of your machine or re-encode and you can mix the streams back  to another file or another network feed and you can send them, send that back.  And most media players work like this with just a pipe of stuff.  And the hard thing in that is synchronizing everything.  Like you have the audio and the video and the subtitle stream and they all go at the  same time but they don't decode as fast.  Like it's faster to decode audio than video.  And in the end you have to get everything at the same time when you present to the user.  Sometimes we get that right.  Sometimes we don't.  So the way VLC is made is you have applications like VLC, VLMC which is video editing software.  They call into LibVLC which is a nice user interface to build media players.  And LibVLC core is the manager which does everything like load modules, provide the  audio access, synchronizing everything, etc. and all of the modules linked to LibVLC for  common features, they're all DLL.  Like LibVLC, LibVLC core and all of the VLC modules, they're all dynamic libraries.  And the modules linked to LibVLC for the features and LibVLC core loads all of the libraries  to see how they work.  So it seems to make a module in Rust we have to do like a bit of dance with the way DLLs  are loaded.  A module, so it's dynamic library loaded by VLC core.  They have to expose three functions because like VLC core just try to load library, take  that function, call it, it exposes some metadata on the module and then knows what the module  can do.  So the plan to start passing stuff with Rust in VLC is to make a DLL in Rust that acts  just like a CDLL, you can load directly in the program that you can build with cargo  and that will work just like a C module in VLC.  So take the headers, reproduce the stuff we need, link to LibVLC core, get the functions  we need, reproduce the module stuff, start writing a parser because that's kind of why  I'm here and we'll start passing stuff.  So VLC is written in C. The VLC command members is kind of macro stuff in C. Well, let's come  back to macro I think.  There's kind of object like interface in VLC where there's a command in a written stuff.  So VLC command members see that like I inherit some attributes from the VLC object.  Lots of things that can be a bit hard to represent like the union types and nested structures  and everything.  So the first thing you have to do is try to write that in Rust.  So if you were at Rust 1 when I was talking about that, I said bind gen just failed on  the VLC headers.  Since then I tried with several bind gen and it's able to handle the VLC headers so I'll  soon be able to generate the whole bindings like that with bind gen.  It should be a lot easier to do.  But in the meantime, I wrote those by hand.  Basically you make Rust structs that kind of acts like C structs with pointers everywhere  and you try to make that a bit safer.  So you import functions like this one to take a stream and get a buffer of a specific size.  You wrap those in functions that you can use from Rust correctly.  When you try to do FFI, you want to isolate all of the unsafe parts because you don't  want to sprinkle unsafe everywhere you could.  So a bit of work to get the FFI running like all of the structures I need because like a  stream T reference another struct with a reference another thing like you saw in there that if  I want to have the VLC object T, I need to have the DMAX T, I need to have the libVLC and  the VLC object T, et cetera.  So lots of preliminary work.  And then we can start writing our module.  We have the interface, we can import code.  And this is how you declare a module in VLC.  Again, it's a macro to say, okay, this is the name of my module.  This is what it can do.  You can take an input stuff and you can parse it.  And here are two functions, open and close, that you can use to interact with that module.  And libVLC code just takes that and loads the module afterwards, knows what to do with  that module.  So when you expand that, it's got some C code that's big to know how to write.  So yeah, basically it's just writing code.  Let's just write that in Rust.  It's easy to write, it's very, very easy to read, right?  No, well, macro that stuff.  Yeah, I think there's a common theme there that I really like, macros, I really can help  you.  But in that case, it's really useful to do.  So you see, I declare the function VLC entry, like the way it's done in C, you have the  VLC entry function there.  So this is how the libVLC module will be loaded.  And from there, okay, you can start passing stuff.  Flash video, it's a format that's very simple, like it took me less than two hours to write  most of the format.  It got audio, video, a few codecs that are mostly outdated.  And here's the beginning of the format, like you have FL and V, the skip charge at the  beginning, then a version number, then some flags to indicate if you have audio, video  and an offset that indicates when the video and audio streams start in the file.  This is interesting because you can have the headers and the offset points to verify in  the file and you can add some stuff between the header and the data.  There has been some very, very funny stuff done like that.  So yeah, I said earlier, okay, there was change in number one.  Now we have do pass, which is a bit easier to use.  So VLC, when it gets to file, it will try to call your open method in your demuxer and  say, okay, I have some data.  Tell me if you can pass that.  So the first thing is you do a stream peak.  You get, okay, I need nine bytes from the beginning of the data.  And I will tell you if it's something I can pass.  And then I call on that slice the header function that we just designed here.  And if it passes correctly, it returns a header and we can say to the file, okay, now I know  and I can pass that stuff.  And we do a stream seek to go to the offset we want.  So it's very easy to interact with VLC like that.  And so since we know we can be the demuxer for that, we pass the demux and call.  And control functions to VLC to say, okay, those are the callbacks you need to use.  So call me again and again and again until I pass all the data.  So when you reach that offset, most of the video formats, they have multiple streams  in multiple blocks.  Like one block of audio, one block of video, one block of subtitles, et cetera, that will  be interspersed depending on how the encoder did the stuff.  Like there's something for MP4 files where if you took a look at the way people were  writing them at the beginning, they put just at the end of the file the header you need  to see.  So you had to go to the end of the stream to do anything.  And now they saw that you can mess with the spec and put that header just at the beginning  of the file and it will not run on anybody.  And so lots of very weird stuff in video format.  Like everybody has good ideas about how a video format should be and they're all wrong,  basically.  Like, yeah, I can't criticize anything but like if I design my own, I do something shitty  as well.  So, yeah.  Something interesting there.  I create a structure in Rust and I put it in the box and I'll be given back that structure  when I'm called afterwards because I cannot want to store some data for my demuxer and  this is the way I can interact with VLC.  So when I get to the first block, I decode a tag that says, okay, it's audio, it's video  or script.  And I have the size of the data and a few things like the timestamp indicates when I  need to present that part of the data.  So if it's audio, it's at second one and if it's audio, it's at second one.  I know I have to present both of them at the same time.  Okay.  So I declare my demux function, which is kind of the way you would do it in C. I read my  header, so it's 15 bytes.  If I have four bytes, I know it's the end of the stream because I have just a beginning  saying the tag type and the size and then I will try to pass everything.  So this is how audio header works.  So you would have in the tag, you have first the type, then the data size and everything  and then you have the format specific data.  For the audio header, it's one byte and says the kind of codec you have, the size and everything.  So bitstream passing in num.  You can take your input, the backslice, you apply bits, which transforms that to backslice  and an index and you go through that thing.  So there you take four bits and it gives you an integer that you can use to see which  kind of audio codec it is.  And then you use that.  So you take one byte with stream read and then you pass and then you have enough data  to know, okay, you have this block of data size minus the header one that contains audio  data that I can push to VLC.  So the interesting thing in there is Rust never owns the data.  Like I can ask VLC for some data but I don't manage its lifetime.  I'm not the one allocating, I'm not the one deallocating, I'm just borrowing data from  C and trying to not break stuff.  It's very important because like most of the time when you interact with C code, the C  code thinks it knows better.  And I don't agree with that but we have to play nice.  So, and don't worry about the unsafe stuff, it's because there are some functions that  take VA list, VR, ARGs, stuff and we have to mess a bit with how functions are declared  in Rust.  But so this is really simple because like in that parser, I only read the headers.  I take the file header, I jump to the next block, I get the block header which is like  10 bytes if it's audio and then I say, okay, there's this much data you can use and I'll  be called again with the next block.  So I never read much data with none in this.  So it's quite fast.  So don't take my word for it, I will show that it works.  So I really like that GIF.  Okay, maybe I'll be able to put that, oh yeah, just, if I get it.  The thing is the more you see that GIF, the more it's funny.  Okay, okay, stop.  So yeah, I will just build and launch and run and copy that in a VLC installation.  I get, it's a very old advertisement for Zelda in EFLV.  You see that it's decoded with the Rust plugin there.  The graphics are pretty outdated.  So yeah, this is Rust, this is Rust in VLC.  So it's very interesting because like, I've been spending, okay, I moved the stuff.  So stop watching the GIFs.  But spend so much time trying to pass stuff and trying to talk to see and everything and  then it works in VLC.  The first time you will get the video to launch is such a good feeling really.  But yeah, now I have to integrate that really in VLC because it was just a plugin that I  built separately and that copied just in a VLC installation.  So now the build system.  Can we build something with AutoTools because Cargo thinks it knows better how to build  everything and I think it's right.  But the AutoTools know better and you have to play nice again.  So how does it work?  First, you have to check if there's a Cargo and Rusty.  So this is Autoconf stuff.  It's pretty nice.  I did not write that.  I'm really bad with the AutoTools but it's all right.  And then the idea is that you don't build the dynamic library yourself.  You build an object file, the Rust plugin.o because there's also a live tool which knows  better than you how to build a dynamic library.  Like Cargo knows how to build a library for Windows, Mac, Linux and don't care, it works.  But play nice with the build system.  Play nice with C, play nice with the interface, play nice with the FFI.  And it's annoying but like Rust makes it kind of easy to do.  So I'm really happy with that.  And on this, so the build system is the biggest issue.  The unsafe APIs, you can work around them and then get a really nice code to use when  you do FFI.  The parser is the yes part and it does not really need to be fast because like video decoding  is so much slower than parsing that you really don't care about that.  And remember that you have to be a nice citizen.  You're not on your land.  You're using someone else's data so you have to be nice with the way the C handles it.  So from now, if you're fixated to do, I have to see how to seek in a file like move back  and forth and everything.  I have to put it back in the VLC repository.  I can build it now inside VLC so it's quite cool.  Predominant dependency will be something that's pretty interesting because with VLC we have  this big archive with all of the libraries we need.  And so for Rust, I'd like to have that as well.  And then to complete the imports, I have to play again a bit with bind gen and we'll be  able to replace some plugins, some VLC plugins with Rust.  And like you can do that with any C project.  It's easy to replace a C project file by file.  Well, easy.  For different reasons, different definitions of easy, but it's doable.  It's really doable.  So a few people that helped me, Guillaume Gomez, maybe you talked a bit about documentation,  Luca Barbato, who works at LibEvi on the LibEvi project and they helped me making this.  So thank you.  That's everything you need.  If you want to make your own module, here's the library extracted from the project and  you want to see the modules there.  Play with Rust, play with C.  Deploy C in production, it's cool.  Do not deploy C in production, please.  Because then I have to break it.  Okay.  Thank you.  Thank you.
Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_0_1559.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_1560_4691.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_4692_4739.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_4740_6071.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_6072_7991.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_7992_11987.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_11988_16931.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_16932_17015.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_17016_17507.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_17508_17555.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_17556_37559.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_37560_37691.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_37692_37823.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_37824_38363.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_38364_38507.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_38508_38639.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_38640_39827.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_39828_39899.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_39900_40019.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_40020_40067.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_40068_44519.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_44520_44591.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_44592_44639.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_44640_44735.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_44736_44867.png Integrating_some_Rust_in_VLC_media_player___Geoffr_egu_0reiq7s_44868_44882.png