QEMU internal APIs
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Habkost, Eduardo
QEMU is an open source machine emulator and virtualizer written in C. Overtime it has evolved multiple interfaces to interact with the outside world,and multiple internal APIs and abstractions to model and keep track of data.The talk will be a review of some of the challenges and trade-offs involved inmaking those abstractions work together. Over time, QEMU has evolved multiple interfaces to interact with the outsideworld (command-line interface, configuration files, its monitor protocol), andmultiple internal APIs and abstractions to model and keep track of data(including configuration, device hierarchy, device state, etc). Understanding the goals and limitations of each of those abstractions andfinding their way through the long list of acronyms (qdev, QOM, QAPI, QMP,VMState, QemuOpts, QObject, etc) can be intimidating to developers notfamiliar to QEMU. Sometimes each of them have conflicting world views,different goals and constraints, and interaction between them generateinteresting challenges to developers working on QEMU. The talk will be areview of some of the challenges and trade-offs involved in making thoseabstractions work together. The target audience are developers that want to get more familiar with QEMUinternal APIs, or see what are the lessons learned (or not learned) from them.
 I am pleased to introduce our next speaker, Eduardo Hapskost, Capkost from Red Hat, who will be speaking about TMIO. Hello. Can I hand it? So, hello. Thank you for coming. My name is Eduardo. I work mostly on QMU, the virtualization team on Red Hat. I'm going to talk about internal abstractions and APIs inside QMU. These are the contents of this talk. I first try to explain what QMU does and has to do. I won't try to explain everything about it, but just as an introduction. Then I'll give you an overview about some of the internal APIs inside QMU to perform its job. And after that, I'll try to talk about how those abstractions work or don't work together. Now, that's what's not included on this talk. I'll try to not present solutions or try to say what's the right way to do stuff. I'll just try to describe how things work. It's like an introduction talk, an introductory talk. I believe it can generate interesting discussions later. And also, keep in mind that this is not going to cover everything, every single API inside QMU, but just the ones that I think is most relevant. So let's see some context first. What QMU does, what it needs to do to do its job. So how many of you know what QMU is and what it does? How many of you have worked on QMU code? Okay, that's good. So I'll try to make this part quick. So what QMU does, it's described as its website as a generic open source machine emulator and a virtualizer. And it's also a very important component when you're using KVM and virtual machines. And let's see what QMU, and QMU has multiple interface to interact with outside world. The main ones are the command line, config files. It has the human monitor protocol, that's a command line interface to control QMU after it has started. And QMP, that's a machine friendly protocol to communicate with QMU based on JSON. This is an external interface it has. And so after this very quick overview of what QMU does from the outside, let's see what it has to do on the inside. Basically, on this slide we see an incomplete list of what QMU needs to handle internally. Among other things, it has to keep track of configuration options, handle monitor commands, keep track of device configuration, device state and backend configuration and other stuff. But each of the APIs we talk about will be used to solve one or more of those problems. Let's go inside QMU then after I try to make the introduction very quick. The first API we talk about is QMU ops. QMU ops is a very old API, introduced in 2009 to handle QMU command line options and config files. It has very few basic data types and it has basically a flat data model. QMU ops is basically used to parse many of the command line options inside QMU. Or if we look at what I think is the most relevant one, it's to handle the most of them. And also even when a command line option is not parsed using QMU ops, it uses it as a style system for command line options. So we could use the same infrastructure to handle config files and command line options at the same time. So let's see how it looks like. This is one example, a simple example of QMU ops in action. We can see here QMU command line and the declarations in the code that make it possible. What's most important here is the desk fashion. I forgot to say if something is unreadable here, you can see the slides on the Fawzen website, these slides are published there. So it describes the options accepted by the dash memory option. It's very simple. The next API I'll talk about is Qdev. Qdev is the bus and advisory hierarchy system inside QMU. It allows us to provide a generic interface, external and internal interface to create configure and plug devices in QMU. Instead of having different APIs or different command line options to each device type. And it has a property system that allows configuration and introspection of devices. One thing that's important to keep in mind is that Qdev was introduced in 2009, but when we enter this form that I'm going to talk about later, the Qdev abstractions were basically rebuilt on top of QMU, but I'll talk about that later. So Qdev is quite successful inside QMU. It's used internally to create and configure basically every device inside QMU. And in addition to its external usage, it provides generic command line and monitor interface to handle devices, plug devices. And its hierarchy and property system also allows users to introspect and pick what's inside a virtual machine, for example, using the InfoQtree device. So let's see. Qdev in action. Qdev does lots of stuff, so I won't explain every detail here because the focus of this talk is to see how stuff works together. So this slide is just a small sample of the property system in QMU. Oops. So we can see here how the properties of a device type like E1000 are declared inside the code. You have boilerplate code to define device types, define how they do it. It's just a sample of how it is used, how things are declared in the code. The relevant parts here are the Mac option is defined inside an array. It's declared here. It has many data types and many helper macros depending on the data types of properties. This is something also provided by Qdev. So Qdev allows you to see the whole device tree. InfoQtree is going to show all the buses and devices inside QMU. And this is how handled by Qdev. Next API, QAPI. So QAPI is a system for defining QMU external interfaces. It uses a JSON-like language to define data structures and interfaces like QMP comments. That's the monitor protocol I talked about. It provides a visitor API to implement data input, output, conversion, and so on. And it's based on a QAPI schema. It generates lots of codes for declaring C types corresponding to those data structures. Civilization, visitor functions, QMP comments dispatching, interfaces, and documentation. QAPI is very successfully used inside QMU to define and dispatch all QMP comments and to define and parse a few command line options. So every single data structure used in QMP is heavily documented on the QAPI schema. So it's a great system. And we have been trying to use QAPI more inside QMU when defining external interfaces. So let's see QAPI in action. This is a QMP comment. Let's try that at. It has a few arguments. One argument is ID and another one is backend. These are the actual QMP messages being exchanged. The response is empty because there are no extra data being returned. This is how it's described in the QAPI schema. Basically I removed all the documentation comments. But basically data structures are all defined. You have declarations to declare how QMP comments look like, what they get in an argument, what they return. You can, this comment uses a feature that's called a union so you can have different attributes depending on the type of option you are using. And this is the function that's going to implement the comment. So the QAPI system is going to take care of serializing everything, calling the right function using a simple C script and then returning the data later. So let's but not least, we have Qoom. Qoom is the QMU object model. Don't confuse that with Q object. That's something else I'm not going to cover in this talk. So Qoom started as a generalization of Qdev. It has lots of interesting features. I won't try to explain everything about it. It would cover a whole talk. So basically it has a type hierarchy of classes. It has a property system. And basically all the distractions in Qdev will be built on top of Qoom. So many of the Qdev code are just wrappers around Qoom. And let's see, in action, these are just a few examples. So I won't try to explain how Qoom works inside, but I'll just see where it is used. It's used when you're using Qdev. When you're plugging a device, Qdev is built on top of Qoom. It's used when you are creating a backend object file, backend object that can represent some entity that's used to implement VM. It's used by the machine type system in Qoom. It's used by the accelerator system. It's used by the CPU model configuration system. And it is also used internally. For example, we have data types for IRQs, memory regions. So it's used at lots of places. We benefit from the property system when we are dealing with options on the common line, when we are dealing with CPU options, or Qdev options. And we have lots of other places where Qoom is used that I'm not even aware of. So it is everywhere. And I see that. Now that we have seen an introduction to each of those abstractions individually, let's see how they work together. That's where things get interesting. First example we have inside Qoom is the dash-numer-common-line option. It's one case where we mix Qoom-Opts and QAPI together. Let's see how it does that. This is how the dash-numer-common-line option is defined on the code. Note the empty desk field here. That's where the accepted options for a dash-numer would be defined, and it's simply empty. Instead of defining the options here, you use the QAPI schema to define that. So all the options on dash-numer, let's go back here. These options here are defined on the QAPI schema. So basically you have a declaration saying that a social name and numeral-numer-numer-numer options can have those fields, and those fields can be specified on the common line. And we have some blue code to convert the Qoom-Opts data to QAPI data so they can be used elsewhere on the code. That's used by this trick here. So we have the visitor API provided by QAPI, and it can be used to convert Qoom-Options to QAPI data soupsions and validate them. So let's summarize this case. Numer is fortunately one case where things are working as they should because all the options are specified on the QAPI schema, and there's no duplication. There's no options on the Qoom-Opts declarations and the QAPI schema, and this is made possible by the ops visitor helper. Other common line options that are very similar are dash-net, dash-net-net, dash-ACPATable, and dash-machine. So this is basically an example where things are working. We can make sure that Qoom-Opts and QAPI work together. Let's see another example. Let's see one case where QAPI and common are used together. This is the object add, QMP command. This is used to create a backend object, basically. It's a genetic system to create objects inside Qoom-O, and there is a QMP command to do that. It has to take a few arguments. It takes the type of the object and the additional properties for the object. And the properties accepted by the object add are defined, basically, by the object itself, the object class itself. So the fireman arguments on the common here are declared not on the QAPI schema, but declared at the Qoom code by the RNG random class, because we are using the Qoom property system to declare which properties the backend object assets. And the QAPI schema looks like this. So we have a common called object add. It has those two arguments, and also an additional argument that can take any data type. So basically, it means that the QAPI schema says you can put anything here. What happens here? So in case of object add, the QAPI schema is incomplete. You don't have the Qoom data appearing on the QAPI schema. So you don't benefit from it. You don't have the type checks provided by QAPI. You don't have the generated code that you could use to implement the interface. And we have a very similar problem with device add. It's even a bit more complex on device add because of the way the properties are defined on the schema, but I won't get into that. So in this case, the abstractions don't work perfectly together. We have some limitations here. Let's see another example that's based on Qoom. That's the CPU. So the CPU model system inside Qoom uses Qoom internally because basically CPUs are devices. Devices are QDAF devices, and QDAF is implemented using Qoom. And we have common line options that are going to control the CPU devices basically. So let's see how it works. This is an example of the options accepted by the CPU or how they are specified. In the case of CPUs, each architecture registered their own Qoom properties the way they need. So in the case of X86, we have a few static properties using the QDAF property system. And we also have some properties declared registered at runtime because we have lots of CPU features configured by X86. I don't know about the details on other architectures. And this is the group code we have. The group code is going to take the common line options. It's not using Qoom ops. It's not using it's unparsed, but that's something we want to change. And using the QDAF global property system, that's something provided by QDAF that ensures that every device of a given type is going to have a few properties set. So basically when you use dash CPU, the CPU features declared on the common line are going to be declared as QDAF options. So the summary for dash CPU is we don't even have a QAPI scheme of dash CPU. It's simply not very well documented. So you don't have any information required to know which common line options are supported by dash CPU. If you want to know that, there's no interface to do that. And it has an additional problem that it does not even use Qoom ops to do that. But that's something, as I said, that we need to change. So this is one example where we see the friction between those interfaces. That Qoom-based stuff being used from the common line is not very well integrated. Let's see another example, even more complex, related to dash CPU options. That's very CPU model expansion, QMP common. So this is one case where we have a Qoom-based implementation and a QAPI-based interface. I don't know how to explain what the common does here, but basically you get a CPU model name and you are going to get information about it on the proper props to build, about all the features inside the CPU, all the profits for a CPU model. And this is the QAPI schema for that common. Back on the object ad case, we have to use this hack here that basically says the props field can have anything. So the semantics of this props field is documented on the documentation for the common, but the schema doesn't tell us which properties can be there, what's the data type, what you can expect there. You need to read the documentation to do that. So summarizing this case, in the case of Qoom-CPU model expansion, the QAPI schema is also incomplete. It doesn't have all the information we want. And it has an additional problem that the build code that will make sure the common implementation can report QAPI-based data is specific for each architecture of CPUs. So we need to clean up that to make sure we use common code to see how we can get common infrastructure to that. There's a pattern here that can be summarized here is that when they're using Qoom, you will normally have trouble when using something QAPI-based because Qoom classes and properties are registered at runtime, especially from properties. They are not a static file in the code. They are added at runtime when you're initializing Qoom. But the QAPI schema is a static file. So you simply can't represent any runtime information on the QAPI schema. So that's why we have these issues with form being used with QAPI. So I think I was faster than I expected. So concluding that, one, my first recommendation here is that lots of practices inside Qoom are not documented anywhere or not very well documented. So this talk is an attempt to write down some of these stuff that we have learned in practice, but you have any question, don't hesitate to ask on the QM of the male list. Because sometimes we are still discovering where we want to go, what practices we want to follow. So it's good to have these things explicitly discussed on the mailing list. I think I'll try to, just to show a summary as we have a few times. So this is something I had as bonus material if we had time. Just to summarize how the data types represented on each API looks like. So most of the APIs will help them involve some kind of data representation. And another issue we have is that they don't support the same data model. QAPI is the most complex, complete one. It supports all the range of data types I enumerated here. But Qoom ops and QBED are more limited. Qoom is also a bit more limited, but it has some limited support to more complex types. So this is one source of the friction we have when we are trying to make one abstraction talk to another one. Something else I tried to summarize. Okay, this is very hard to hit with. But basically, on all those abstractions here, we have some equivalent data structures. We have often some type or option type, some option type defined. We normally have a property system say which attributes can be set for each option type, device type, or schema data structure. On most of these cases, we can have mechanisms to define defaults for them. And then we have the runtime data structures. They are basically the options themselves, the device themselves, or the actual C-strat runtime data. And one difference on this case that makes it difficult that I tried to explain before is that some of the data that is static, for example, for QAPI, or that is static for QBEDF, is actually not static for the other systems. So, C-oom properties can be right so that one time, and you can easily translate to the static data on QBEDF. The defaults for Qoom ops and C-oom are also defined at one time, so you can't just convert to the other data structures later. So this is another source of friction. So questions? Comments? You can ask. I just wanted to ask, is there any plan or effort to reduce? Thank you very much. Okay, so Eduardo, is there any plan to get rid of some of these layers to do something about all this? Because there's a lot of duplication, and it's also causing bugs, and it causes a lot of effort during development. You don't know what you should do. Sometimes the overlap causes problems as well. That's a good question because the different part of this question one is that sometimes we are not sure or we are not in agreement of where to go, but I can say what I know or my impressions. My impressions are that instead of removing those layers, we can try to make them thinner. For example, QM ops today have a way to define those options, and we have a way to define common line options using Qoom or API. If we could move all the common line options to be defined using QAPI or Qoom, then we could remove the option definitions from QM ops and have only the parcel. On the other case of QAPI and Qoom, I think they are going to leave there. We just need to make them work together. I think we want to get rid of Qoom. We want to get rid of the runtime registration of properties, so we just need to make sure they work together. There is some work to describe common line options in a language like the QAPI schema, but I don't know the details of how it's going to be done, but we are trying to do that. I don't think on the case of the abstractions I showed, I don't think any of them is going to be removed in the near term, but we can make them be rebuilt on top of the others and make some of the code inside them less necessary. Qdev is one example. Qdev was basically rebuilt on top of Qoom, but it still has some abstraction that's not inside Qoom, but we can work to make Qdev reuse Qoom code instead of duplicating. I don't think we can remove the layers, but we can reduce the duplication between them by reusing the same infrastructure. One example are the parsers. When looking at the code, I found out that we have at least three different parsers, three different parsers for integral numbers, for strings, so you have a number on the string, you need to parse it. There are three different implementations, each one with different bugs and with different limitations. I think we can reduce duplication not by removing layers because it's going to be a huge effort, but by making them benefit from each other and reusing code from each other. Anybody else? Thank you. Qdev, do you have any recommendations on how other projects could avoid this mess on the long term? You've got multiple APIs with other lab in functionality and what not. Do you have any insights for other developers working on other projects on how to avoid and not with that? I don't have any advice, but I can say what I think went wrong. For example, I think some of the new abstractions were added without a clear plan to replace other ones. So when we introduced Chrome and tried to convert all the existing devices to use Chrome or Qdev, there was no clear plan to say, okay, we are going to convert these devices and then these ones and we are going to keep these and these abstractions and remove that one. There was no clear plan to do that. When we started using QAPI for some of the common line options, there was no clear plan to say, okay, this is the process we are going to follow. We want to convert the old ones to the new APIs. So I think that's what didn't work on QML. There was no clear plan. Yes? Adding another answer is having test cases ready from the start makes any conversions absolutely easy. That was one of the issues that we were facing. Anybody else? Okay, thank you. So the slides are available there and if you don't want to write down the link, you can just get the slides and follow them. There's a blog post I wrote about with an introduction to QM APIs with links to other ones. I have been trying to write a summary of which ones so if you want to follow it. Thank you.
 I am pleased to introduce our next speaker, Eduardo Hapskost, Capkost from Red Hat, who  will be speaking about TMIO.  Hello.  Can I hand it?  So, hello.  Thank you for coming.  My name is Eduardo.  I work mostly on QMU, the virtualization team on Red Hat.  I'm going to talk about internal abstractions and APIs inside QMU.  These are the contents of this talk.  I first try to explain what QMU does and has to do.  I won't try to explain everything about it, but just as an introduction.  Then I'll give you an overview about some of the internal APIs inside QMU to perform  its job.  And after that, I'll try to talk about how those abstractions work or don't work together.  Now, that's what's not included on this talk.  I'll try to not present solutions or try to say what's the right way to do stuff.  I'll just try to describe how things work.  It's like an introduction talk, an introductory talk.  I believe it can generate interesting discussions later.  And also, keep in mind that this is not going to cover everything, every single API inside  QMU, but just the ones that I think is most relevant.  So let's see some context first.  What QMU does, what it needs to do to do its job.  So how many of you know what QMU is and what it does?  How many of you have worked on QMU code?  Okay, that's good.  So I'll try to make this part quick.  So what QMU does, it's described as its website as a generic open source machine emulator  and a virtualizer.  And it's also a very important component when you're using KVM and virtual machines.  And let's see what QMU, and QMU has multiple interface to interact with outside world.  The main ones are the command line, config files.  It has the human monitor protocol, that's a command line interface to control QMU after  it has started.  And QMP, that's a machine friendly protocol to communicate with QMU based on JSON.  This is an external interface it has.  And so after this very quick overview of what QMU does from the outside, let's see what  it has to do on the inside.  Basically, on this slide we see an incomplete list of what QMU needs to handle internally.  Among other things, it has to keep track of configuration options, handle monitor commands,  keep track of device configuration, device state and backend configuration and other  stuff.  But each of the APIs we talk about will be used to solve one or more of those problems.  Let's go inside QMU then after I try to make the introduction very quick.  The first API we talk about is QMU ops.  QMU ops is a very old API, introduced in 2009 to handle QMU command line options and  config files.  It has very few basic data types and it has basically a flat data model.  QMU ops is basically used to parse many of the command line options inside QMU.  Or if we look at what I think is the most relevant one, it's to handle the most of them.  And also even when a command line option is not parsed using QMU ops, it uses it as a  style system for command line options.  So we could use the same infrastructure to handle config files and command line options  at the same time.  So let's see how it looks like.  This is one example, a simple example of QMU ops in action.  We can see here QMU command line and the declarations in the code that make it possible.  What's most important here is the desk fashion.  I forgot to say if something is unreadable here, you can see the slides on the Fawzen  website, these slides are published there.  So it describes the options accepted by the dash memory option.  It's very simple.  The next API I'll talk about is Qdev.  Qdev is the bus and advisory hierarchy system inside QMU.  It allows us to provide a generic interface, external and internal interface to create  configure and plug devices in QMU.  Instead of having different APIs or different command line options to each device type.  And it has a property system that allows configuration and introspection of devices.  One thing that's important to keep in mind is that Qdev was introduced in 2009, but when  we enter this form that I'm going to talk about later, the Qdev abstractions were basically  rebuilt on top of QMU, but I'll talk about that later.  So Qdev is quite successful inside QMU.  It's used internally to create and configure basically every device inside QMU.  And in addition to its external usage, it provides generic command line and monitor interface  to handle devices, plug devices.  And its hierarchy and property system also allows users to introspect and pick what's  inside a virtual machine, for example, using the InfoQtree device.  So let's see.  Qdev in action.  Qdev does lots of stuff, so I won't explain every detail here because the focus of this  talk is to see how stuff works together.  So this slide is just a small sample of the property system in QMU.  Oops.  So we can see here how the properties of a device type like E1000 are declared inside  the code.  You have boilerplate code to define device types, define how they do it.  It's just a sample of how it is used, how things are declared in the code.  The relevant parts here are the Mac option is defined inside an array.  It's declared here.  It has many data types and many helper macros depending on the data types of properties.  This is something also provided by Qdev.  So Qdev allows you to see the whole device tree.  InfoQtree is going to show all the buses and devices inside QMU.  And this is how handled by Qdev.  Next API, QAPI.  So QAPI is a system for defining QMU external interfaces.  It uses a JSON-like language to define data structures and interfaces like QMP comments.  That's the monitor protocol I talked about.  It provides a visitor API to implement data input, output, conversion, and so on.  And it's based on a QAPI schema.  It generates lots of codes for declaring C types corresponding to those data structures.  Civilization, visitor functions, QMP comments dispatching, interfaces, and documentation.  QAPI is very successfully used inside QMU to define and dispatch all QMP comments and  to define and parse a few command line options.  So every single data structure used in QMP is heavily documented on the QAPI schema.  So it's a great system.  And we have been trying to use QAPI more inside QMU when defining external interfaces.  So let's see QAPI in action.  This is a QMP comment.  Let's try that at.  It has a few arguments.  One argument is ID and another one is backend.  These are the actual QMP messages being exchanged.  The response is empty because there are no extra data being returned.  This is how it's described in the QAPI schema.  Basically I removed all the documentation comments.  But basically data structures are all defined.  You have declarations to declare how QMP comments look like, what they get in an argument, what  they return.  You can, this comment uses a feature that's called a union so you can have different attributes  depending on the type of option you are using.  And this is the function that's going to implement the comment.  So the QAPI system is going to take care of serializing everything, calling the right  function using a simple C script and then returning the data later.  So let's but not least, we have Qoom.  Qoom is the QMU object model.  Don't confuse that with Q object.  That's something else I'm not going to cover in this talk.  So Qoom started as a generalization of Qdev.  It has lots of interesting features.  I won't try to explain everything about it.  It would cover a whole talk.  So basically it has a type hierarchy of classes.  It has a property system.  And basically all the distractions in Qdev will be built on top of Qoom.  So many of the Qdev code are just wrappers around Qoom.  And let's see, in action, these are just a few examples.  So I won't try to explain how Qoom works inside, but I'll just see where it is used.  It's used when you're using Qdev.  When you're plugging a device, Qdev is built on top of Qoom.  It's used when you are creating a backend object file, backend object that can represent  some entity that's used to implement VM.  It's used by the machine type system in Qoom.  It's used by the accelerator system.  It's used by the CPU model configuration system.  And it is also used internally.  For example, we have data types for IRQs, memory regions.  So it's used at lots of places.  We benefit from the property system when we are dealing with options on the common line,  when we are dealing with CPU options, or Qdev options.  And we have lots of other places where Qoom is used that I'm not even aware of.  So it is everywhere.  And I see that.  Now that we have seen an introduction to each of those abstractions individually, let's  see how they work together.  That's where things get interesting.  First example we have inside Qoom is the dash-numer-common-line option.  It's one case where we mix Qoom-Opts and QAPI together.  Let's see how it does that.  This is how the dash-numer-common-line option is defined on the code.  Note the empty desk field here.  That's where the accepted options for a dash-numer would be defined, and it's simply empty.  Instead of defining the options here, you use the QAPI schema to define that.  So all the options on dash-numer, let's go back here.  These options here are defined on the QAPI schema.  So basically you have a declaration saying that a social name and numeral-numer-numer-numer  options can have those fields, and those fields can be specified on the common line.  And we have some blue code to convert the Qoom-Opts data to QAPI data so they can be  used elsewhere on the code.  That's used by this trick here.  So we have the visitor API provided by QAPI, and it can be used to convert Qoom-Options  to QAPI data soupsions and validate them.  So let's summarize this case.  Numer is fortunately one case where things are working as they should because all the  options are specified on the QAPI schema, and there's no duplication.  There's no options on the Qoom-Opts declarations and the QAPI schema, and this is made possible  by the ops visitor helper.  Other common line options that are very similar are dash-net, dash-net-net, dash-ACPATable,  and dash-machine.  So this is basically an example where things are working.  We can make sure that Qoom-Opts and QAPI work together.  Let's see another example.  Let's see one case where QAPI and common are used together.  This is the object add, QMP command.  This is used to create a backend object, basically.  It's a genetic system to create objects inside Qoom-O, and there is a QMP command to do that.  It has to take a few arguments.  It takes the type of the object and the additional properties for the object.  And the properties accepted by the object add are defined, basically, by the object itself,  the object class itself.  So the fireman arguments on the common here are declared not on the QAPI schema, but declared  at the Qoom code by the RNG random class, because we are using the Qoom property system  to declare which properties the backend object assets.  And the QAPI schema looks like this.  So we have a common called object add.  It has those two arguments, and also an additional argument that can take any data type.  So basically, it means that the QAPI schema says you can put anything here.  What happens here?  So in case of object add, the QAPI schema is incomplete.  You don't have the Qoom data appearing on the QAPI schema.  So you don't benefit from it.  You don't have the type checks provided by QAPI.  You don't have the generated code that you could use to implement the interface.  And we have a very similar problem with device add.  It's even a bit more complex on device add because of the way the properties are defined  on the schema, but I won't get into that.  So in this case, the abstractions don't work perfectly together.  We have some limitations here.  Let's see another example that's based on Qoom.  That's the CPU.  So the CPU model system inside Qoom uses Qoom internally because basically CPUs are devices.  Devices are QDAF devices, and QDAF is implemented using Qoom.  And we have common line options that are going to control the CPU devices basically.  So let's see how it works.  This is an example of the options accepted by the CPU or how they are specified.  In the case of CPUs, each architecture registered their own Qoom properties the way they need.  So in the case of X86, we have a few static properties using the QDAF property system.  And we also have some properties declared registered at runtime because we have lots  of CPU features configured by X86.  I don't know about the details on other architectures.  And this is the group code we have.  The group code is going to take the common line options.  It's not using Qoom ops.  It's not using it's unparsed, but that's something we want to change.  And using the QDAF global property system, that's something provided by QDAF that ensures  that every device of a given type is going to have a few properties set.  So basically when you use dash CPU, the CPU features declared on the common line are going  to be declared as QDAF options.  So the summary for dash CPU is we don't even have a QAPI scheme of dash CPU.  It's simply not very well documented.  So you don't have any information required to know which common line options are supported  by dash CPU.  If you want to know that, there's no interface to do that.  And it has an additional problem that it does not even use Qoom ops to do that.  But that's something, as I said, that we need to change.  So this is one example where we see the friction between those interfaces.  That Qoom-based stuff being used from the common line is not very well integrated.  Let's see another example, even more complex, related to dash CPU options.  That's very CPU model expansion, QMP common.  So this is one case where we have a Qoom-based implementation and a QAPI-based interface.  I don't know how to explain what the common does here, but basically you get a CPU model  name and you are going to get information about it on the proper props to build, about  all the features inside the CPU, all the profits for a CPU model.  And this is the QAPI schema for that common.  Back on the object ad case, we have to use this hack here that basically says the props  field can have anything.  So the semantics of this props field is documented on the documentation for the common, but the  schema doesn't tell us which properties can be there, what's the data type, what you can  expect there.  You need to read the documentation to do that.  So summarizing this case, in the case of Qoom-CPU model expansion, the QAPI schema is also incomplete.  It doesn't have all the information we want.  And it has an additional problem that the build code that will make sure the common implementation  can report QAPI-based data is specific for each architecture of CPUs.  So we need to clean up that to make sure we use common code to see how we can get common  infrastructure to that.  There's a pattern here that can be summarized here is that when they're using Qoom, you  will normally have trouble when using something QAPI-based because Qoom classes and properties  are registered at runtime, especially from properties.  They are not a static file in the code.  They are added at runtime when you're initializing Qoom.  But the QAPI schema is a static file.  So you simply can't represent any runtime information on the QAPI schema.  So that's why we have these issues with form being used with QAPI.  So I think I was faster than I expected.  So concluding that, one, my first recommendation here is that lots of practices inside Qoom  are not documented anywhere or not very well documented.  So this talk is an attempt to write down some of these stuff that we have learned in practice,  but you have any question, don't hesitate to ask on the QM of the male list.  Because sometimes we are still discovering where we want to go, what practices we want  to follow.  So it's good to have these things explicitly discussed on the mailing list.  I think I'll try to, just to show a summary as we have a few times.  So this is something I had as bonus material if we had time.  Just to summarize how the data types represented on each API looks like.  So most of the APIs will help them involve some kind of data representation.  And another issue we have is that they don't support the same data model.  QAPI is the most complex, complete one.  It supports all the range of data types I enumerated here.  But Qoom ops and QBED are more limited.  Qoom is also a bit more limited, but it has some limited support to more complex types.  So this is one source of the friction we have when we are trying to make one abstraction  talk to another one.  Something else I tried to summarize.  Okay, this is very hard to hit with.  But basically, on all those abstractions here, we have some equivalent data structures.  We have often some type or option type, some option type defined.  We normally have a property system say which attributes can be set for each option type,  device type, or schema data structure.  On most of these cases, we can have mechanisms to define defaults for them.  And then we have the runtime data structures.  They are basically the options themselves, the device themselves, or the actual C-strat  runtime data.  And one difference on this case that makes it difficult that I tried to explain before  is that some of the data that is static, for example, for QAPI, or that is static for QBEDF,  is actually not static for the other systems.  So, C-oom properties can be right so that one time, and you can easily translate to the  static data on QBEDF.  The defaults for Qoom ops and C-oom are also defined at one time, so you can't just convert  to the other data structures later.  So this is another source of friction.  So questions?  Comments?  You can ask.  I just wanted to ask, is there any plan or effort to reduce?  Thank you very much.  Okay, so Eduardo, is there any plan to get rid of some of these layers to do something  about all this?  Because there's a lot of duplication, and it's also causing bugs, and it causes a lot  of effort during development.  You don't know what you should do.  Sometimes the overlap causes problems as well.  That's a good question because the different part of this question one is that sometimes  we are not sure or we are not in agreement of where to go, but I can say what I know  or my impressions.  My impressions are that instead of removing those layers, we can try to make them thinner.  For example, QM ops today have a way to define those options, and we have a way to define  common line options using Qoom or API.  If we could move all the common line options to be defined using QAPI or Qoom, then we  could remove the option definitions from QM ops and have only the parcel.  On the other case of QAPI and Qoom, I think they are going to leave there.  We just need to make them work together.  I think we want to get rid of Qoom.  We want to get rid of the runtime registration of properties, so we just need to make sure  they work together.  There is some work to describe common line options in a language like the QAPI schema,  but I don't know the details of how it's going to be done, but we are trying to do that.  I don't think on the case of the abstractions I showed, I don't think any of them is going  to be removed in the near term, but we can make them be rebuilt on top of the others  and make some of the code inside them less necessary.  Qdev is one example.  Qdev was basically rebuilt on top of Qoom, but it still has some abstraction that's  not inside Qoom, but we can work to make Qdev reuse Qoom code instead of duplicating.  I don't think we can remove the layers, but we can reduce the duplication between them  by reusing the same infrastructure.  One example are the parsers.  When looking at the code, I found out that we have at least three different parsers,  three different parsers for integral numbers, for strings, so you have a number on the string,  you need to parse it.  There are three different implementations, each one with different bugs and with different  limitations.  I think we can reduce duplication not by removing layers because it's going to be a huge effort,  but by making them benefit from each other and reusing code from each other.  Anybody else?  Thank you.  Qdev, do you have any recommendations on how other projects could avoid this mess on  the long term?  You've got multiple APIs with other lab in functionality and what not.  Do you have any insights for other developers working on other projects on how to avoid  and not with that?  I don't have any advice, but I can say what I think went wrong.  For example, I think some of the new abstractions were added without a clear plan to replace  other ones.  So when we introduced Chrome and tried to convert all the existing devices to use Chrome  or Qdev, there was no clear plan to say, okay, we are going to convert these devices and  then these ones and we are going to keep these and these abstractions and remove that one.  There was no clear plan to do that.  When we started using QAPI for some of the common line options, there was no clear plan  to say, okay, this is the process we are going to follow.  We want to convert the old ones to the new APIs.  So I think that's what didn't work on QML.  There was no clear plan.  Yes?  Adding another answer is having test cases ready from the start makes any conversions  absolutely easy.  That was one of the issues that we were facing.  Anybody else?  Okay, thank you.  So the slides are available there and if you don't want to write down the link, you can  just get the slides and follow them.  There's a blog post I wrote about with an introduction to QM APIs with links to other  ones.  I have been trying to write a summary of which ones so if you want to follow it.  Thank you.
QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_0_47.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_48_107.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_108_923.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_924_4211.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_4212_7139.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_7140_8051.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_8052_10427.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_10428_10823.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_10824_11135.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_11136_14591.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_14592_15071.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_15072_17711.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_17712_19787.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_19788_22667.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_22668_23627.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_23628_23795.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_23796_24791.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_24792_25031.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_25032_25091.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_25092_28247.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_28248_32075.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_32076_38219.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_38220_39299.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_39300_40523.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_40524_46163.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_46164_48107.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_48108_49607.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_49608_49667.png QEMU_internal_APIs_and_conflicting_world_views_How_abstractions_inside_QEMU_49668_49723.png