urn:lj:livejournal.com:atom1:libv LIBV Intentionally Breaks Videodrivers What about Lovely, Intelligent and Beautiful Visionary? Luc Verhaegen 2014-03-05T15:32:27Z urn:lj:livejournal.com:atom1:libv:26434 The Raspberry Pi, en route to becoming a proper open platform. 2014-03-05T15:32:27Z 2014-03-05T15:32:27Z Fatboy Slim - We've come a long way baby - Praise you Like most, I was taken by surprise last weekend, I never expected this to happen, at all.<br /><br />Given that I was the one to poke a rather big hole in the Raspberry Pi image a year and a half ago, and since i have been playing a bit of a role in freeing ARM GPUs these last few years, I thought I'd say a few words on how I see things, and why I said what I said back then, and why I didn't say other things.<br /><br />It has become a bit of a long read, but this it's a eventful story that spans about two years.<h2>Getting acquainted.</h2>I talked about the lima driver at linuxtag in may 2012. As my contract work wasn't going any where real, I spent two weeks of codethink time (well, it's not as if I ever did spent just 8h per day working on lima) on getting the next step working with lima, so I had something nice to demo at Linuxtag. During that bringup, I got asked to form a plan of attack for freeing the Raspberry Pi GPU, for a big potential customer. Just after my lima talk at the conference, I downloaded some code and images through the linuxtag wifi, and set about finding the shader compiler on the train on the way back.<br /><br />I am not one for big micro-managed teams where constant communication is required. I am very big on structure and on finding natural boundaries (which is why you have the term modesetting and the structured approach to developing display drivers today). I tend to split up a massive task along those natural boundaries, and then have clued folks take full control and responsibility of those big tough subtasks. This is why I split the massive task of REing a GPU between the command stream side and the shader compiler. This is the plan of attack that I had formed in the first half of 2011 for Mali (which we are still sticking to today), and I was of course looking to repeat that for the RPi.<br /><br />On the train, I disassembled glCompileShader from the VideoCore userspace binary, and found the code to be doing surprisingly little. It quickly went down to some routines which were used by pretty much all gl and egl calls. And those lower routines were basically taking in some ids and some data, and then passing it to the kernel. A look at the kernel code then told me that there was nothing but passing data through happening at that side...<br /><br />There was nothing real going on, as all the nifty bits were running somewhere else.<br /><br />In the next few days, I talked to some advanced RPi users on IRC, and we figured out that even the bootloader runs on the Videocore and that that brings up the ARM core. A look at the 3 different binary blobs, for 3 different memory divisions between the ARM and the VideoCore, revealed that they were raw blobs, running directly on the videocore. Strings were very clearly visible, and what was immediately clear is that it was mostly differences in addresses between the 3 blobs. Apart from the addresses, nothing was immediately apparent, apart from the fact that even display code was done by the Videocore. I confirmed with Ben Brewer, who was then working on the shaders for Lima, and who has a processor design background.<br /><br />The news therefor wasn't good. The Raspberry Pi is a completely closed system with a massive black box pulling all the strings. But at least it is consistently closed, and there is only limited scope for making userspace and 'The Blob' getting out of sync, unlike the PowerVR, where userspace, kernel and microcode are almost impossible to keep in sync.<br /><br />So I wrote up the proposal for the potential customer (you know who you are), basically a few manmonths of busywork (my optimistic estimate doubled, and then doubled again, as I never do manage to keep my own optimistic timelines) to free the userspace shim, and a potential month for Ben for poking at the videocore blobs. Of course, I never heard back from codethink sales, which was expected with news that was this bad.<br /><br />I decided to stay silent about the Raspberry Pi being such a closed system, at least for a few weeks. For once, this customer was trying to do The Right Thing by talking to the right people, instead of just making big statements, no matter what damage they cause or what morals they breach. I ended up being quite busy in the next few months and I kind of forgot about making some noise. This came back to haunt me later on.<br /><br /><h2>The noise.</h2>So on oktober 24th of 2012, <a href="http://www.raspberrypi.org/archives/2221" rel="nofollow">the big announcement</a> came. Here is the excerpt that I find striking:<pre>"[...] the Raspberry Pi is the first ARM-based multimedia SoC with fully-functional, vendor-provided (as opposed to partial, reverse engineered) fully open-source drivers, and that Broadcom is the first vendor to open their mobile GPU drivers up in this way. We at the Raspberry Pi Foundation hope to see others follow."</pre>While the text of the announcement judiciously talks about "userland" and "VideoCore driver code which runs on the ARM", it all goes massively downhill with the above excerpt. Together wit the massive cheering coming from the seemingly very religious RPi community, the fact that the RPis SoC was a massively closed system was completely drowned out.<br /><br />Sure, this release was a very good step in the right direction. It allowed people to port and properly maintain the RPi userspace drivers, and took away a lot of the overhead for integration. But the above statement very deliberately brushed the truth about the BCM2835 SoC under the carpet, and then went on to brashly call for other vendors to follow suit, even though those others had chosen very different paths in their system design, and did not depend on a massive all-controlling black box.<br /><br />I was appalled. Why was this message so deliberately crafted in what should be a very open project? How could people be so shortsighted and not see the truth, even with the sourcecode available?<br /><br />What was even worse was the secondary message here. To me, it stated "we are perfectly happy with the big binary blob pulling the strings in the background". And with their call to other vendors, they pretty much told them: "keep the real work closed, but create an artificial shim to aid porting!". Not good.<br /><br />So I went and talked to the other guys on the #lima channel on irc. The feeling amongst the folks in our channel was pretty uniform one of disgust. At first glance, this post looked as a clear and loud message to support open drivers, it only looked like that on the absolute surface. Anyone who would dig slightly deeper would soon see the truth, and the message would turn into "Keep the blob guys!".<br /><br />It was clear that this message from the RPi foundation was not helping the cause of freeing ARM GPUs at all. So I decided to react to this announcement.<h2>The shouting</h2>So I decided to go post in the broadcom forums, and <a href="http://www.raspberrypi.org/archives/2221#comment-34981" rel="nofollow">this is what I wrote</a>:<pre>Erm… Isn’t all the magic in the videocore blob? Isn’t the userland code you just made available the simple shim that passes messages and data back and forth to the huge blob running on the highly proprietary videocore dsp? – the developer of the lima driver.</pre>Given the specific way in which the original RPI foundation announcement was written, i had expected all from the RPI foundation itself to know quite clearly that this was indeed just the shim driver. The reminder of me being the lima driver developer, should've taken away any doubt about me actually knowing what i was talking about. But sadly, these assumptions were proven quite wrong when Liz Upton <a href="http://www.raspberrypi.org/archives/2221#comment-34985" rel="nofollow">replied</a>:<pre>No. There’s some microcode in the Videocore – not something you should confuse with an ARM-side blob, which could actually prevent you from understanding or modifying anything that your computer does. That microcode is effectively firmware.</pre>She really had no idea to what extent the videocore was running the show, and then happily went and argued with someone whom most people would label an industry expert. Quite astounding.<br /><br />Things of course went <a href="http://www.raspberrypi.org/archives/2221#comment-35048" rel="nofollow">further</a> <a href="http://www.raspberrypi.org/archives/2221#comment-35049" rel="nofollow">downhill</a> <a href="http://www.raspberrypi.org/archives/2221#comment-35054" rel="nofollow">from there</a>, with several RPI users showing themselves from their best side. Although I did like <a href="http://www.raspberrypi.org/archives/2221#comment-35089" rel="nofollow">the comment about ATI and ATOMBIOS</a>, some people at least hadn't forgotten that nasty story.<br /><br />All-in-all, this was truly astounding stuff, and it didn't help my feeling that the RPI foundation was not really intending to do The Right Thing, but instead was more about making noise, under the guise of marketing. None of this was helping ARM GPUs or ARM userspace blobs in any way.<h2>More shouting</h2>What happened next was also bad. Instead of sticking to the facts, Dave Airlie went and <a href="http://airlied.livejournal.com/76383.html" rel="nofollow">posted this blog entry</a>, reducing himself to the level of Miss Upton.<br /><br />A few excerpts:<pre>"Why is this not like other firmware (AMD/NVIDIA etc)? The firmware we ship on AMD and nvidia via nouveau isn't directly controlling the GPU shader cores. It mostly does ancillary tasks like power management and CPU offloading. There are some firmwares for video decoding that would start to fall into the same category as this stuff. So if you want to add a new 3D feature to the AMD gpu driver you can just write code to do it, not so with the rPI driver stack."</pre><pre>"Will this mean the broadcom kernel driver will get merged? No. This is like Ethernet cards with TCP offload, where the full TCP/IP stack is run on the Ethernet card firmware. These cards seem like a good plan until you find bugs in their firmware stack or find out their TCP/IP implementation isn't actually any good. The same problem will occur with this. I would take bets the GLES implementation sucks, because they all do, but the whole point of open sourcing something is to allow other to improve it something that can't be done in this case."</pre>Amazing stuff.<br /><br />First of all, with the videocore being responsible for booting the bcm2835, and the bcm2835 already being a supported platform in the linux kernel (as of version 3.7), all the damage had been done already. Refusing yet another tiny message passing driver, which has a massive user-base and which will see a lot of testing, then is rather short-sighted. Especially since this would have made it much easier for the massive RPI userbase to use the upstream kernel.<br /><br />Then there is the fact that this supposed kernel driver would not be a drm driver. It's simply not up to the self-appointed drm maintainer to either accept or refuse this.<br /><br />And to finish off, a few years ago, Dave forked RadeonHD driver code to create a driver that did things the ATI way (read: much less free, much more dependent on bios bytecode, and with many other corners being implemented the broken ATI way), wasted loads of AMD money, stopped register documentation from coming out, and generally made sure that fglrx still rules supreme today. And all of this for just "marketing reasons", namely the ability to make trump SuSE with noise. With a history like that, you just cannot go and claim the moral high ground on anything anymore.<br /><br />Like I've stated before, open source software is about making noise, and Dave his blog entry was nothing more and nothing less. Noise breeds more noise in our parts. This is why I decided to not provide a full and proper analysis of the situation back then, as we had lowered ourselves to the level that the RPI foundation had chosen, and nothing I could have said would've improved on that.<h2>Redemption</h2>I received an email from Eben Upton last friday, where he pointed out the big news. I replied that I would have to take a thorough look first, but then also stated that I didn't expect Eben to email me if this hadn't been the real thing. I then congratulated him extensively.<br /><br />And congratulations really are in order here.<br /><br />The Raspberry PI is a global sales phenomenon. It created a whole new market, a space where devices like the Cubieboard and Olimex boards also are in now. It made ARM development boards accessible and affordable, and put a linux on ARM in the hands of 2.5 million people. It is so big that the original goal of the Raspberry Pi, namely that of education, is kind of drowned out. But just the fact that so many of these devices were sold already, will have a bigger educational impact than any direct action towards that goal. This gives the RPI foundation a massive amount of power, so much so, that even the most closed SoC around is being opened now.<br /><br />But in Oktober of 2012, my understanding was that the RPI foundation wasn't interested in wielding that power to reach those goals which I and many others hold dear. And I was really surprised when it now was revealed that they did, as I had personally given up on this ever happening, and I had assumed that the RPI foundation was not intending to go all the way.<h2>What's there?</h2>After the email from Eben, I joined the #raspberrypi-internals channel on freenode to ask the guys who are REing the VideoCore, to get an expert opinion from those actually working on the hardware.<br /><br />So what has appeared:<br />* A user manual for the 3D core.<br />* Some sample code for a different SoC, but code which which runs on the ARM core and not on the videocore.<br />* This sample code includes some interesting headers which helps the guys who are REing the Videocore.<br />* There's a big promise of releasing more about the Videocore and at least providing a properly free bootloader.<br /><br />These GPU docs and code not only means that Broadcom has beaten ARM, Qualcomm, Nvidia, Vivante and Imagination technologies to the post, it also makes broadcom second only to Intel. (yes, in front of AMD, or should I say ATI, as ATI has run most of the show since 2008).<br /><br />Then there is the extra info on the videocore and the promise of a fully free bootloader. Yes, there still is the massive dependance on the VideoCore for doing all sorts of important SoC things, but releasing this information takes time, especially since this was not part of the original SoC release process and it has to be made up for afterwards. The future is looking bright for this SoC though, and we really might get to a point where the heavily VideoCore based BCM2835 SoC becomes one of the most open SoCs around.<br /><br />Today though, the BCM2835 still doesn't hold a candle to the likes of the Allwinner A10 with regards to openness. But the mid to long term difference is that most of Allwinners openness so far was accidental (they are just now starting to directly cooperate with the linux-sunxi community), and Allwinner doesn't own all the IP used in their SoC (ARM owns the GPU for instance, and it is being a big old embedded dinosaur about it too). The RPI SoC not only has very active backing from the SoC vendor, that SoC vendor also happens to be the full owner of the IP at the same time.<br /><br />The RPI future looks really bright!<h2>Whining.</h2>I do not like the 10k bounty on getting Q3A going with the freshly released code. I do not think this is the best way to get a solid gpu driver out, as it mostly encourages a single person to make a quick and dirty port of the freshly released code. I would've preferred to have seen someone from the RPI foundation create a kanban with clearly defined goals, and then let the community work off the different work items, after which those work-items are graded and rewarded. But then, it's the RPI foundations money, and this bounty is a positive action overall.<br /><br />I am more worried about something else. With the bounty and with the promise of a free bootloader, the crazy guys who are REing the VideoCore get demotivated twice. This 10k bounty rules them out, as it calls for someone to do a quick and dirty hack, and not for long hard labour towards The Right Thing. The promise of a free bootloader might make their current work superfluous, and has the potential to halt REing work altogether. These guys are working on something which is or was at least an order of magnitude harder than what the normal GPU REing projects have or had to deal with. I truly hope that they get some direct encouragement and support from the RPI foundation, and that both parties work together on getting the rest of the VideoCore free.<br /><br />All in all, this Broadcom release is the best news I have heard since I started on the open ARM GPU path, it actually is the best news I've had since AMD accepted our proposal to free the ATI Radeon (back in june 2007). This release fully makes up for the bad communication of Oktober 2012, and has opened the door to making the BCM2835 the most free SoC out there.<br /><br />So to Eben and the rest of the guys from the RPI Foundation: Well done! You made us all very happy and proud. urn:lj:livejournal.com:atom1:libv:26292 FOSDEM14 lima driver talk. 2014-02-19T12:00:07Z 2014-02-19T12:00:07Z Fatboy Slim - We've come a long way baby - Right here, right now The FOSDEM organizers have captured more than 16h of video for 22 DevRooms and they are still working hard on getting all of these videos cut and transcoded and put on their site. This is a mammoth task, and some <a href="http://video.fosdem.org/2014/H1301_Cornil/" rel="nofollow">talks in the graphics devroom</a> are still up in the air because of this.<br /><br />Luckily, some people from the <a href="http://linux-sunxi.org/Main_Page" rel="nofollow">sunxi community</a> took the livestream of sunxi related talks, and produced some quick dumps of it. I have now copied <a href="http://people.freedesktop.org/~libv/FOSDEM2014_lima_driver.webm" rel="nofollow">the Mali driver talk</a> over to my fd.o account so that the sunxi server doesn't become too overloaded. <a href="http://people.freedesktop.org/~libv/FOSDEM2014_lima_driver.pdf" rel="nofollow">Slides are also available</a>.<br /><br />This talk mentions where i am at with my mesa driver, why i am taking the route i am taking, the importance of SoC communities, and how the lack thereof hampers driver development, and also talks about the company ARM and their stance towards the open driver.<br /><br />Enjoy! urn:lj:livejournal.com:atom1:libv:25862 FOSDEM, and the SuSE bus. 2014-02-06T16:16:09Z 2014-02-06T16:16:09Z The Prodigy - The fat of the land - No good When i was still at SuSE, <a href="http://news.opensuse.org/2007/10/12/people-of-opensuse-martin-lasarsch/" rel="nofollow">Localhorst</a> would rent a Ford S-Max, and stuff it to the brim with openSuSE kit and swag and drive to FOSDEM. I usually was tolerated on board as well, with <a href="https://scontent-a-fra.xx.fbcdn.net/hphotos-prn2/t1/1621703_720766164624137_698881002_n.jpg" rel="nofollow">my sportsbag full of DevRoom kit</a>, provided i sang along to the radio, and didn't mention burger-king. Everyone at SuSE either made their own arrangements, or everyone was stuck on a flight from Nuremberg to Brussels (which was quickly dubbed "The SuSE-Bomber").<br /><br />After the massive and rather counterproductive layoffs after FOSDEM 2009, SuSE tended to organize a bus for its own employees. And from what i heard, it was a pretty good solution. Imagine a load of happy geeks, from a place in the world with the best beers, stuck on a bus. It made the whole event seem like a school trip, but one where some beers were actually allowed. And, since there usually was tons of extra space on the bus, a load of ex-SuSE guys got to hitch a ride as well. The result was that a lot of SuSE employees visited FOSDEM, and got to catch up on things with some ex-SuSE guys and generally start doing what conferences are for from the second the bus left Nuremberg. Since busses are cheap, this really was a perfect solution, and everyone was happy.<br /><br />I never took the bus. For FOSDEM, I want to arrive in brussels around suppertime on friday, and leave monday around midday (when the alcohol of the previous night has worn off a bit). The bus tended to arrive around suppertime as well, but would leave again around 19:00 on sunday. Also, i tend to run a devroom and have at least one talk. I need those 5 hours of peace on the train to prepare my talk. But I heard good things about the bus, and that it all was great fun and a bit of a community event before (and after) the big community event.<br /><br />This year, however, things were different. For a long time, apparently, nobody from the OpenSuSE team was really bothered with FOSDEM. This i find truly amazing, and a really bad sign with respect of where SuSE and openSuSE is apparently heading. From what i have heard, there was always a bit of a plan to get a bus, but it was unclear where the budget would come from, and no-one took any action. I also heard that two weeks before the event, SuSE employees were asked whether they wanted to go to FOSDEM. Now if you do this 2 weeks before the event, with people who often have a wife and kids these days, most will already have made other plans. Then, if you also state that those people who might be interested, also need to have some travel budget left over, and need to get approval in a few days time, you of course only get a handful of people who end up going to the biggest open source event of the planet. I heard the number 8.<br /><br />8 people from SuSE went to FOSDEM. An absolute disgrace for what once was the leading european linux distribution.<br /><br />Here is an idea: why not make the bus a community service from the start? Why doesn't OpenSuSE sponsor a bus, one which starts at nuremberg, and perhaps stops in Frankfurt-Flughafen (so some people can grab a smoke and empty their beer-filled bladders), and then continues on to Brussels? Give the SuSE employees 4 weeks early notice to get their seats reserved (which gives them an incentive to think about FOSDEM early on), and then make seat reservation open to anyone who wants to go visit FOSDEM and lives near Nuremberg or Frankfurt? You can even hand the community members a bag of SuSE swag and a frankonian beer.<br /><br />SuSE would not only do something good for their own employees and makes it easier on them to visit FOSDEM. They would actually sponsor FOSDEM and help boost their openSuSE community.<br /><br />I am actually surprised that this has to be said, and that idea hasn't been spawned from within the OpenSuSE team itself. But here you go. Now make it happen for next year. urn:lj:livejournal.com:atom1:libv:25725 FOSDEM, the best conference... In the world. 2014-02-05T14:51:52Z 2014-02-05T14:53:00Z Underworld - Beaucoup Fish - Jumbo Now that my body is well on its road to recovery, it's time to talk about the great success that FOSDEM was, once again.<br /><br />We had a really nice devroom and pretty good crowds, but the most amazing thing was the recordings and the livestream. The FOSDEM organizers really outdid themselves there.<br /><br />After the initial announcement of the graphics devroom went out, Martin Peres contacted me and we talked about getting proper recordings from the DevRoom, and briefly looked into what this would end up costing if we were to buy the equipment for it. Our words were barely cold when the FOSDEM organizers announced what can only be seen as absolutely insane: they would try to provide for full recording of everything.<br /><br />In the end, FOSDEM provided recording in 22 rooms. They had to get 22 mixing desks, 22 sets of microphones, 22 cameras, 44 laptops... This was apparently mostly rented, and apparently, surprise, surprise, there aren't many rental companies which have such an insane amount of kit available.<br /><br />Apart from some minor issues (like a broken firewire cable in the wine devroom) things worked out amazingly. Only the FOSDEM guys could pull something this insane off. We had all our talks in the Graphics DevRoom streamed out live, with no issues at all.<br /><br />I would like to thank all the speakers in the graphics devroom, but i particularly would like to thank Martin Peres, who took full control and responsibility of the video equipment. Then there's Marcus Meissner and Thiery Redding who willingly sat in the back of the devroom and handled the recordings themselves, directing the streams, for only meagre rations of water and cookies. Without people stepping up and doing their bit, a devroom like this would not be possible. And the same goes for the whole of FOSDEM.<br /><br />At the end of the final talk, after i had talked about sunxi_kms, i tried to thank the FOSDEM organizers, and get the remaining audience to clap for them. But i mostly stood there and babbled, at a loss for words, because what the FOSDEM organizers had achieved with this insane goal is simply amazing. And thinking about it now still, i still get quite emotional...<br /><br /><b>How on earth did they manage to pull this off, on top of organizing the rest of FOSDEM, a FOSDEM which caters for something like 8000 people as it is... It's just not possible!</b><br /><br />It's not as if these guys get paid for what they are doing, FOSDEM is a low budget organization, purely based on volunteers. The absolute core of the organization is just a handful of people who have very busy jobs. And yet, they have succeeded where any other organization would have failed. There's no politics or powerplay, there is no money or commerce. There is just the absolute drive to make FOSDEM the best event on the planet, by making small changes every year...<br /><br />This was my 7th DevRoom this year, and if i can help it there will be another one next year. I am really proud that i am allowed to do my, comparatively little, part as well. Every sunday evening after FOSDEM, after we sit down in the restaurant with the remainders of the graphics devroom, i am physically broken, but i am also one of the happiest people on the planet...<br /><br />Each year, no matter what happened in the year before, no matter what nasty open source politics or corporate nonsense took place over that year... Each year, the FOSDEM organizers prove that something amazing can happen if only people do their bit, if only people work towards the same selfless goal. Each year, FOSDEM reminds me of why i do what i do, and why i need to keep on doing it.<br /><br />Thank you. urn:lj:livejournal.com:atom1:libv:25551 Graphics DevRoom at FOSDEM2014. 2013-10-12T21:35:30Z 2013-10-12T21:36:19Z Chris Clark - Vengeance drools Yes, there is going to be another exciting DevRoom about graphics on <a href="https://fosdem.org/2014/" rel="nofollow">the upcoming FOSDEM</a>.<br /><br />It's not called the X.org DevRoom this time round, but a hopefully more general Graphics DevRoom. As was the case with the X.org DevRooms before, anything related to graphics drivers and windowing systems goes. While the new name should make it clearer that this DevRoom is about more than just X, it also doesn't fully cover the load either, as this explicitly does include input drivers as well.<br /><br />Some people have already started wondering why I haven't been whining at them before. Well, my trusted system of blackmailing people into holding talks early on failed this year. The FOSDEM deadline was too early and XDC was too late, so I decided to take a chance, and request a devroom again, in the hope that enough people will make it over to the fantastic madness that is FOSDEM.<br /><br />After endless begging and grovelling the FOSDEM organizers got so fed up that they gave us two full days again. This means that we will be able to better group things, and avoid a scheduling clash like with the ARM talks last year (where ARM system guys were talking in one room exactly when ARM graphics guys were talking in another). All of this doesn't mean that First Come, First Serve doesn't apply, and if you do not want to hold a talk with a hangover in an empty DevRoom, you better move quickly :)<br /><br />The FOSDEM organizers have a system called pentabarf. This is where everything is tracked and the schedules are created, and, almost magically, at the other end, all sorts of interesting things fall out, like the unbelievably busy but clear website that you see every year. This year though, it is expected that speakers themselves manage their own details, and that the DevRoom organizers oversee this, so we will no longer use the trusted wiki pages we used before. While i am not 100% certain yet, i think it is best that people who have spoken at the DevRoom (most of whom i will be poking personally anyway) in the past few years first talk to me first before working with pentabarf, as otherwise there will be duplicate accounts which will mean more overhead for everyone. More on that in the actual call for speakers email which will hit the relevant mailing lists soon.<br /><h2>FOSDEM futures for ARM</h2>Connor Abbott and I both have had chromebooks for a long long time. Connor bought his when it first came out, which was even before the last FOSDEM. I bought mine at a time where I thought that Samsung was never going to sell it in germany, and the .uk version arrived on my doorstep 3 days before the announcement for Europe went out. These things have been burning great big holes in our souls ever since, as i stated that we would first get the older Mali models supported properly with our Lima driver, and deliver a solid graphics driver before we lose ourselves again in the next big thing. So while both of us had this hardware for quite a while, we really couldn't touch these nice toys with an interesting GPU at all.<br /><br />Now, naturally, this sort of thing is a bit tough to impose on teenagers, as they are hormonally programmed to break rules. So when Connor got bored during the summer (as teenagers do), he of course went and broke the rules. He did the unspeakable, and grabbed ARMs standalone shader compiler and <a href="http://limadriver.org/T6xx+ISA/" rel="nofollow">started REing the Mali Midgard ISA.</a> When his father is at FOSDEM this year, the two of us will have a bit of 'A Talk' about Connors wild behaviour, and Connor will be punished. Probably by forcing him to finish the beers he ordered :)<br /><br />Luckily, adults are much better at obeying the rules. Much, much better.<br /><br />Adults, for instance, would never go off and write a command stream tracer for this out of bounds future RE project. They would never ever dare to replay captured command streams on the chromebook. And they definitely would not spend days sifting through a binary to expose the shader compiler of the Mali Midgard. Such a thing would show weakness in character and would just undermine authority, and I would never stoop so low.<br /><br />If I had done such an awful thing, then I would definitely not be talking about how much harder capture and replay were, err, would be, on this Mali, and that the lessons learned on the Mali Utgard will be really useful... In future? I would also not be mentioning how nice it would be to work on a proper linux from the get-go. I would also never be boasting at how much faster Connor and I will be at on turning our RE work on T6xx into a useful driver.<br /><br />It looks like Connor and I will have some very interesting things to own up to at FOSDEM :) urn:lj:livejournal.com:atom1:libv:25325 Intel & Mir: The point-of-view of a graphics driver developing bystander. 2013-09-09T23:22:34Z 2013-09-09T23:56:16Z Amon Tobin - Sordid Only a few days ago did I write about <a href="http://libv.livejournal.com/24870.html" rel="nofollow">how open source software is not about "code or design or doing The Right Thing". "Open source software is about power, politics, corporate affiliation, and loads and loads of noise."</a> I would like to thank Intel for so <a href="http://cgit.freedesktop.org/xorg/driver/xf86-video-intel/commit/?id=58a7611cc" rel="nofollow">succinctly underlining that now with their current action</a>.<br /><br />Before I go any further, this seems to not be Chris Wilsons decision or his preferred solution. Chris wrote the patch which he was told, by unnamed party or parties in Intel, to back out. Also, I personally do not condone the actions taken by Canonical, but, as a graphics driver developer, I find Intels actions far worse. I rather doubt that Intel thought this one through properly.<br /><br /><h2>What's the problem?</h2>As a graphics driver developer, I fail to see the big problem with Mir.<br /><br />So what if Canonical has decided to reinvent Wayland? Apart from the weird contribution agreement (which will only limit contributions), Mir is fully free software isn't it? Who are they hurting apart from their own resources and their own users? It's not that I am applauding Canonical for their decision, but I really don't see the massive problem here.<br /><br />Why is Canonical not allowed to do this?<br /><h2>Reinvention galore</h2>I personally really hate things being reinvented all the time. It is the disease that plagues open source software, and that what makes sure that we don't have a growing linux market.<br /><br />How often have we heard that something is outdated and broken and doesn't fit modern demands anymore? We are then invariably being told that something new is being built afresh, from the lessons learned of what was done "wrong" before, and that in a few months time, everything is going to be fantastic. Sadly, such timeframes never pan out, and while the known "errors" are fixed, everything else gets broken, which then has to be reinvented or ported as well (or which simply remains broken). And then several years down the line, things are still not perfect, and then someone else (or sometimes even the same person) goes off and implements the next great thing from scratch, again.<br /><br />We never have something that just works, we just go from broken state to broken state. And nobody learns from this, nobody apparently ever states "Hang on, isn't that pretty much the same story we heard 3 years ago?"<br /><br />To me, as a stupid shortsighted driver developer, Wayland seems like X reinvented. A server/client display architecture with the new lessons learned implemented, but with everything else broken. We've been waiting for getting all those little niggles worked out ever since 2009, and at one point, networking was added to Wayland making it even more of an X replacement.<br /><br />So then <a href="http://www.phoronix.com/scan.php?page=news_item&amp;px=MTMxNzI" rel="nofollow">Mir was announced...</a> And suddenly the world was ablaze. Huge flamewars broke out everywhere and effigies of Mark Shuttleworth were getting burned in the forums. I found the Mir move quite ironic, at first, and thought that the outrage was quite out of proportion, but then <a href="http://www.phoronix.com/scan.php?page=news_item&amp;px=MTMxNzY" rel="nofollow">I read this article</a>. It is a who's who of reinventers, complaining about Canonical reinventing Wayland. I was appalled.<br /><br />What exactly gives these people the sole monopoly on reinvention?<br /><h2>What is Intel afraid of?</h2>How could Mir possibly threaten Wayland?<br /><br />Intel is a pretty big company, and it probably has the largest contingent of open source developers devoted to graphics. It employs some of the brightest and most influential people in the business. On top of that, Wayland was there first, has had more time to mature, has had more applications and toolkits ported, and has a much larger mindshare. Most people would think that Waylands future is pretty secure.<br /><br />So what could possibly be so much better about Mir that makes Mir such a big threat to Wayland that Intels graphics driver developers have to be told not to support XMir at all? Honestly, in the above constellation, how vastly more superior technically does Mir have to be to justify such an action? If Intel really feels that it has to react like this, well, then it might as well just throw in the towel and go Mir immediately, as Wayland clearly must be completely useless.<br /><br />What a way to oust your own insecurity.<br /><h2>Software Fascism</h2>Intel finds it necessary to play games with their X.org graphics driver, instead of having Wayland battle it out directly with Mir.<br /><br />This kind of powerplay is quite insidious, and far more damaging than most people would expect. It completely skews the ability of software to compete on a fair and equal grounds, and hurts us all as it is mostly applied by those who are not able to compete properly, or those who feel as if they shouldn't need to bother to compete properly. It tends to favour the least technically advanced and the least morally acceptable.<br /><br />The best example which I have come across so far is the RadeonHD versus Radeon battle. RadeonHD beat ATI by actually <a href="http://www.phoronix.com/scan.php?page=article&amp;item=842" rel="nofollow">providing a solid open source driver in September 2007</a>, and we at SuSE had <a href="http://www.phoronix.com/scan.php?page=article&amp;item=amd_radeonhd_four&amp;num=6" rel="nofollow">a stated goal of being able to ship a solid open source driver on enterprise desktop rollouts</a>. 3 months later, Radeon came around with support for the same hardware. It was technically inferior, and <a href="http://lists.x.org/archives/xorg-commit/2007-November/013806.html" rel="nofollow">"borrowed" much of the hard work of radeonHD</a> plus some noise added on top. What was worse was how the so-called X.org community used software fascism to artificially boost the Radeon driver. This started out with <a href="https://bugs.freedesktop.org/show_bug.cgi?id=45232#c3" rel="nofollow">the refusal of a mailing list at the usual place</a>, hit a low point with <a href="http://lists.x.org/archives/xorg/2008-October/039550.html" rel="nofollow">RadeonHD being dropped from the build script for the xserver</a>, and sank to whole new levels when, 2 years after the obvious death of the RadeonHD driver, <a href="http://lists.x.org/pipermail/xorg/2010-November/051666.html" rel="nofollow">the RadeonHD repository got vandalized</a> (and <a href="http://lists.x.org/pipermail/xorg/2010-November/051704.html" rel="nofollow">the whistleblower got tarred and feathered</a> while <a href="http://lists.x.org/pipermail/xorg/2010-November/051729.html" rel="nofollow">the perpetrators were commended for their "quick" confession</a>).<br /><br />So who won?<br /><br />Well, it definitely was not RadeonHD, as <a href="http://www.h-online.com/open/features/Wash-me-but-do-not-make-me-wet-746941.html" rel="nofollow">that died early 2009 with Novell laying off a large portion of SuSE developers in Nuremberg</a>. As luck had it, at the same time, <a href="http://www.nytimes.com/2008/10/07/technology/07chip.html?bl&amp;ex=1223611200&amp;en=6c2c0d7539595be6&amp;ei=5087%0A&amp;_r=0" rel="nofollow">AMD experienced serious financial difficulties</a> and did not continue the RadeonHD project with SuSE. But although Radeon did survive, it did not win either. ATI won, AMD (which wanted a proper open source driver, whereas ATI seriously didn't) lost, and we all lost with it. Fglrx still rules supreme today, but now it does not get as much flack as it did before, as the figleaf driver provides some sort of an alternative for those who are unhappy with fglrx. But it goes beyond that, the radeon driver consistently applies or applied the solutions ATI fglrx developers recommended, instead of <a href="http://cgit.freedesktop.org/xorg/driver/xf86-video-radeonhd/tree/src/rhd_pll.c#n1402" rel="nofollow">the empirical solutions we at RadeonHD usually chose</a>, and the radeon driver is not as good as it could be.<br /><br />Software fascism goes further than just badly skewing competition, and it always is a negative influence on software. Who knows what other bad decisions will make their way into the Intel driver now?<br /><h2>The responsibility of a graphics driver</h2>The main responsibility of a graphics driver is to support the users of your graphics hardware. If you are actually employed by the vendor, your users are those who bought your hardware and who will buy your hardware again if they are satisfied. This is the business case for providing optimal support for your hardware for a given operating system or infrastructure. On top of that, in open source software, the users are more than just the customers, they are also the testers.<br /><br />Canonicals plan and marketing seem to have worked out quite well over the years, to the extent that half the planet thinks that linux equals ubuntu, and ubuntu probably has the larger part of the linux desktop market. This means that ubuntu users are a sizable portion of Intels userbase, and as a hardware vendor (and only secondarily a maker of display servers), Intel simply cannot afford to refuse to support or even alienate these users. Canonical has decided that Mir will be the primary display server on future Ubuntu releases, and this in turn means that Intel has an obligation to support Mir.<br /><br />The Xmir patch to the Intel graphics driver seems rather minimal and not very invasive. There also seems or seemed, as the case may be now, direct communication between Intels graphics driver developers and Ubuntus developers. As Mir will ship on the next Ubuntu versions, there will be a large amount of users which will test the Xmir code in the Intel graphics driver. There is no chance that the Xmir code will bitrot for the foreseeable future, and Intels own investment in this code will be minimal.<br /><br />The real art of writing good drivers is to provide for quick and painless debugging. Graphics hardware is complex, the drivers for this hardware are also complex, and neither is ever perfect, so one has to work hard to maximize the chance for bug resolution. This means easy communication with users, and giving the user an easy route to test changes so that proper feedback can be provided quickly. If you fail to make it easy enough for users, you will simply not get your bugs fixed, and the higher the resolution threshold becomes, the worse your driver will become.<br /><br />By not carrying this patch, Intel forces Ubuntu users to only report bugs to Ubuntu, which then means that only few bug reports will filter through to the actual driver developers. At the same time, Ubuntu users cannot simply test upstream code which contains extra debugging or potential fixes. Even worse, if this madness continues, you can imagine Intel stating to its customers that they refuse to fix bugs which only appear under Mir, even though there is a very very high chance of these bugs being real driver bugs which are just exposed by Mir.<br /><br />The reality of the matter is, Intel is hurting its own graphics driver more than it could potentially hurt Mir or Canonical.<br /><h2>The andriodization of linux</h2>The biggest installed base of Linux is android, and it is bigger by many orders of magnitude. Sadly the linux which we call android is little more than the linux kernel and some new-ish (mostly) open source infrastructure on top. While this, to some extent, is quite the boon for open source software, it also holds a major threat. If we are not careful, we get fully locked hardware. We are only sporadically able to enforce the GPL on the kernel, and we have no chance at all to get open source userspace drivers. This limits the usefulness of the now ubiquitous linux hardware out there, and with the way the desktop and mobile are evolving, this will soon limit the availability of hardware for which more-or-less complete open source is available. On top of that, all those electronics companies that are churning out hardware at an amazing rate, they are either unable to see the advantages of actively contributing to open source, or they are having a very hard time in learning how to do so.<br /><br />This is exactly why I created <a href="http://limadriver.org/" rel="nofollow">the lima driver</a>, and why some other brave souls created their respective GPU reverse engineering projects. We recognized this danger, and are sacrificing a large portion of our lives trying to prevent catastrophe. And even though things are not going as fast or as smooth as we expected, we have come a very long way.<br /><br />Things took a wrong turn a while back though. In an effort to create a stopgap solution, <a href="http://en.wikipedia.org/wiki/Hybris_%28software%29" rel="nofollow">Jolla developer Munk created libhybris</a>, a wrapper library which allows the usage of android drivers on top of glibc, and thus on a normal linux installation. I find this hack pretty dangerous, as it makes all vendors complacent, and it cements the android way of working and the it makes binary drivers the default. Our biggest open source hopes for mobile; Sailfish, Firefox-OS and Ubuntu-Phone Mir readily embraced this way of working.<br /><br />I have, so far, not seen anything from either Jolla, The Mozilla Foundation or Canonical, along the lines of active support of the route we have chosen with open ARM GPU drivers, and we've been at it for quite some time now. Those companies are more dependent on open source software than your average android vendor, and know how to do things the open source way, but they have fully embraced the binary drivers built for android only, with no signs of them wanting to change this.<br /><br />The only reason why I favour Wayland over Mir is that <a href="http://unity.ubuntu.com/mir/android_technical_details.html" rel="nofollow">Canonical immediately chose the libhybris route with Mir</a>. Wayland currently has patches for libhybris, so soon Wayland sadly will have sunk to Mirs level as well, from a graphics driver point of view.<br /><br />Intel employs a small army for their open source software, and specifically for their open source graphics driver. But Intel also has other teams working on graphics drivers, and while I am not certain, I do think that Intel ships binary only drivers on their android devices.<br /><br />Canonical is happy with using libhybris, but currently would prefer to use a proper graphics driver for their future products. This preference now got significantly reduced. Intel now potentially has driven one of the last big users of open source graphics drivers to exclusively using android binaries as well, seriously reducing the relevance of its own OSTC driver developer team in the process.<br /><h2>The low road</h2>Up until now, intel had the moral high ground in the Wayland versus Mir situation. With the simple decision to revert the Xmir patch, this situation now got reversed.<br /><br />Well done. urn:lj:livejournal.com:atom1:libv:24870 The lima mesa driver runs es2gears. 2013-09-03T22:57:34Z 2013-09-03T22:59:48Z Lemon Jelly - Ramblin' man <h2>es2gears!</h2>Progress is slow but steady on the lima mesa driver, mostly because I am not giving lima as much time as I should. I now have working attributes, uniforms, vertex buffers and even some state is being set correctly. Enough to run es2gears. Here is <a href="http://www.youtube.com/watch?v=4WOILEYAxWE" rel="nofollow">the video on youtube</a>, there is <a href="http://www.youtube.com/watch?v=3L-cYo6IPFc" rel="nofollow">an older capture with a rotating smoothed cube as well</a>.<br /><br />This lima mesa driver uses the old-school (but contrary to popular belief, definitely not deprecated) mesa infrastructure, which, with surprisingly little work, allows me to run the mali binary drivers compiler, and allows me to build the lima driver externally. Using the binary compiler would not be possible from gallium, and with some improvements to the (intel developed, and thus intel focused) mesa glsl compiler, it seems that we might have a potent compiler for the mali ISAs as well. This way, the task of bringing up mesa on mali is nicely split, and we will be able to debug the command stream work separately from the shader compiler. I believe that we did lose the ability to run gles1 programs until <a href="https://gitorious.org/open-gpu-tools/cwabbotts-open-gpu-tools" rel="nofollow">Connors open-gpu-tools</a> is hooked up to the mesa glsl compiler. This is just a small price to pay, in comparison to the size of the hurdle to take when doing everything in one go.<br /><br />We are running es2gears at 130fps on the A10, and 310fps on the Exynos for a 300x300 render. Resizing is currently broken, or rather not implemented. I will need to split up the PLBU command stream to be able to do that properly under DRI2 (where resizes happen behind the mesa drivers back). The way DRI2, mesa and <a href="https://github.com/ssvb/xf86-video-sunxifb" rel="nofollow">the sunxifb X driver</a> now work together also means that i have to wait for jobs to finish (and then usleep for good measure), so there is a lot of potential for speed-up as well. I am not sure yet how things will have to be hacked to keep X from copying the region before rendering is absolutely done, i guess that we will have to hack something into UMP and the sunxifb driver. But a solution will be found, and we should see around a 50% increase in framerate from that, and even much much more if we manage to use overlays. Since we are in control of all the code now, we should theoretically be able to squeeze every last bit of performance out of the GPU, a luxury not offered to the users of the binary X11 driver.<br /><br />I will now continue implementing textures, so that i can run all the limare egl tests. After that i will clean up the code and push it out. This will include patches to common mesa versions (and mesa packages) to allow building lima against them. Resizing and job interleaving will have to wait until after that, so keep your eyes peeled on this space :)<br /><br /><h2>10 years in.</h2>A few weeks ago, it was the 10th anniversary of <a href="http://comments.gmane.org/gmane.comp.xfree86.devel/3028&quot;&quot;" rel="nofollow">my first contribution to X</a> (a small display fix to the via driver). I cannot state that this anniversary was a happy event.<br /><br />Looking back, I cannot believe that i once thought that in open source software, code, design and doing <i>The Right Thing</i>, both technically and morally, were paramount. Open source software is about power, politics, corporate affiliation, and loads and loads of noise. Noise and misinformation always wins over code, no matter how good this code is or how hard you work at it. I have had to learn this several times over.<br /><br />This is especially true in the case of forks. Not the git clone variation, but the loud, aggressive and very detrimental community kind. While often technical reasons are claimed to be the cause, this is never ever the case. It always is about politics and power. And code always suffers as a result, and this suffering is never a short term thing, especially on big forks. See, a fork always means a big stink, a lot of noise, and a power vacuum, and this attracts a certain kind of individual. These personalities form the basis of the new community together with those who instigated or accelerated the fork. Good code simply has no chance in such an environment. Bad code and bad design tends to hang around for a long long time, and tends to influence (read, limit) the thinking of any new blood that turns up. On top of that, the bad mentality also tends to linger for many many years. Politics and noise continues to take precedence over code and design, for the foreseeable future.<br /><br />The only thing to do in case of a fork, is to go play somewhere else, somewhere where a major fork hasn't taken place. If you don't, or if you do not go far enough, you will see your work impacted, especially if you are not willing to let yourself be limited by the existing mentality and powerbalance.<br /><br />Now if only i wasn't such a stubborn bastard ;) urn:lj:livejournal.com:atom1:libv:24735 Old and new limare code, and management overhead... 2013-06-04T01:16:13Z 2013-06-04T01:17:02Z Hole - Live through this 04 - Asking for it I just pushed <a href="https://gitorious.org/lima/lima/commits/master" rel="nofollow">updated limare code</a> and <a href="https://github.com/libv/ioquake3" rel="nofollow">a fix to ioquake3</a>.<br /><br />In almost 160 patches, loads of things change:<ul><br /><li>clean FOSDEM code supporting Q3A timedemo on a limare ioquake3.</li><br /><li>support for r3p2 kernel and binary userspace as found on the odroid-x series.</li><br /><li>multiple PP support, allowing for the full power of the mali 400MP4 to be used.</li><br /><li>fully threaded job handling, so new frames can be set up while the first is getting rendered.</li><br /><li>multiple textures, in rgb888, rgba8888 and rgb565, with mipmapping.</li><br /><li>multiple programs.</li><br /><li>attribute and elements buffer support.</li><br /><li>loads of gl state is now also handled limare style.</li><br /><li>memory access optimized scan pattern (hilbert) for PP (fragment shader).</li><br /><li>direct MBS (mali binary shader) loading for pre-compiled shaders (and <a href="https://gitorious.org/~cwabbott/open-gpu-tools/cwabbotts-open-gpu-tools" rel="nofollow">OGT</a> shaders!!!).</li><br /><li>support for UMP (arm's in kernel external memory handler).</li><br /><li>Properly centered companion cube (now it is finally spinning in place :))</li><br /><li>X11 egl support for tests.</li><br /><li>...</li><br /></ul>Some of this code was already published to allow the immediate use of <a href="http://libv.livejournal.com/24402.html" rel="nofollow">the OGT enabled ioquake3</a>. But that branch is now going to be removed, as <a href="https://gitorious.org/lima/lima/commits/master" rel="nofollow">the new code</a> replaces it fully.<br /><br />As for performance, this is no better or worse than <a href="http://libv.livejournal.com/23886.html" rel="nofollow">the FOSDEM code</a>. 47fps on timedemo on the Allwinner A10 at 1024x600. But now on the Exynos 4, there are some new numbers... With the CPU clocked to 2GHz and the Mali clocked to 800MHz (!!!) we hit 145fps at 720p and 127fps at 1080p. But more on that a bit further in this post.<br /><br /><h2>Upcoming: Userspace memory management.</h2><br />Shortly after FOSDEM, <a href="http://libv.livejournal.com/23886.html" rel="nofollow">i blogged about the 2% performance advantage</a> over the binary driver when running Q3A.<br /><br />As you might remember, we are using ARMs kernel driver, and despite all the pain that this is causing us due to shifting IOCTL numbers (whoever at ARM decided that IOCTL numbers should be defined as enums should be laid off immediately) I still think that this is a useful strategy. This allows us to immediately throw in the binary driver, and immediately compare Lima to the binary, and either help hard reverse engineering, or just make performance comparisons. Rewriting this kernel driver, or turning this into a fully fledged DRM driver is currently more than just a waste of time, it is actually counterproductive right now.<br /><br />But now, while bringing up a basic mesa driver, it became clear that I needed to work on some form of memory management. Usually, you have the DRM driver handling all of that, (even for small allocations i think - not that i have checked). We do not have a DRM driver, and I do not intend to write one in the very near future either, and all I have is the big block mapping that the mali kernel driver offers (which is not bad in itself).<br /><br />So in the train on the way back from linuxtag this year, I wrote up a small binary allocator to divide up the 2GB of address space that the Mali MMU gives us. On top of that, I now have 2 types of memory, sequential and persistent (next to UMP and external, for mapping the destination buffer into Mali memory), and limare can now allocate and map blocks of either at will.<br /><br />The sequential memory is meant for per-frame data, holding things like draws and varyings and such, stuff that gets thrown away after the frame has been rendered. This simply tracks the amount of memory used, adds the newly requested memory at the end, and returns an address and a pointer. No tracking whatsoever. Very lightweight.<br /><br />The persistent memory is the standard linked list type, with the overhead that that incurs. But this is ok, as this memory is meant for shaders, textures and attribute and element buffers. You do not create these _every_ draw, and you tend to reuse them, so it's acceptable if their management is a bit less optimized.<br /><br />Normally, more management makes things worse, but this memory tracking allowed me to sanitize away some frame specific state tracking. Suddenly, Q3A at 720p which originally ran at 145fps on the exynos, ran at 176fps. A full 21% faster. Quite some difference.<br /><br />I now have a board with a Samsung Exynos 4412 prime. This device has the quad A9s clocked at 1.7GHz, 2GB LP-DDR2 memory at 880MHz, and a quad PP Mali-400MP4 at 440MHz. This is quite the powerhouse compared to the 1GHz single A8 and single PP Mali-400 at 320MHz. Then, this Exynos chip I got actually clocks the A9s to 2GHz and the mali to a whopping 800MHz (81% faster than the base clock). Simply insane.<br /><br />The trouble with the exynos device, though, is that there are only X11 binaries. This involves a copy of the rendered buffer to the framebuffer which totally kills performance. I cannot properly compare these X11 binaries with my limare code. So I did take my new memory management code to the A10 again, and at 1024x600 it ran the timedemo at 49.5fps. About a 6% margin over the binary framebuffer driver, or tripling my 2% lead at FOSDEM. Not too bad for increased management, right?<br /><br />Anyway, with the overclocking headroom of the exynos, it was time for a proper round of benchmarking with limare on exynos.<br /><br /><h2>Benchmark, with a pretty picture!</h2><br /><img src="http://people.freedesktop.org/~libv/limare_q3a_timedemo_exynos4412.png" alt="Limare Q3A benchmark results on exynos4412" width="512"><br /><br />The above picture, which I quickly threw together manually, maps it out nicely.<br /><br />Remember, this is an Exynos 4412 prime, with 4 A9s clocked from 1.7-2.0GHz, 2GB LP-DDR2 at 880MHz, and a Mali-400MP4 which clocks from 440MHz to an insane 800MHz. The test is the quake 3 arena timedemo, running on top of limare. Quake 3 Arena is single threaded, so apart from the limare job handling, the other 3 A9 cores simply sit idle. It's sadly the only good test I have, if someone wants to finish the work to port Doom3 to gles, I am sure that many people will really appreciate it.<br /><br />At 720p, we are fully CPU limited. At some points in the timedemo (as not all scenes put the same load on cpu and/or gpu), the difference in mali clock makes us slightly faster if the cpu can keep up, but this levels out slightly above 533MHz. Everything else is simply scaling linearly with the cpu clock. Every change in cpu clock is a 80% change in framerate. We end up hitting 176.4fps.<br /><br />At 1080p, it is a different story. 1080p is 2.25 times the amount of screen real estate of 720p (if that number rings a bell, 2.25MB equals two banks of Tseng ET6x00 MDRAM :p). 2.25 times the amount of pixels that need to pushed out. Here clearly the CPU is not the limiting factor. Scaling linearly from the original 91fps at 440MHz is a bit pointless, as the Q3A benchmark is not always stressing CPU and GPU equally over the whole run. I've drawn the continuation of the 440-533MHz increase, and that would lead to 150fps, but instead we run into 135.1fps. I think that we might be stressing the memory subsystem too much. At 135fps, we are pushing over 1GBps out to the framebuffer, this while the display is refreshing at 60fps, so reading in half a gigabyte. And all of this before doing a single texture lookup (of which we have loads).<br /><br />It is interesting to see the CPU become measurably relevant towards 800MHz. There must be a few frames where the GPU load is such that the faster CPU is making a distinguishable difference. Maybe there is more going on than just memory overload... Maybe in future i will get bored enough to properly implement the mali profiling support of the kernel, so that we can get some actual GP and PP usage information, and not just the time we spent waiting for the kernel job to return.<br /><br /><h2>ARM Management and the Lima driver </h2><br />I have recently learned, from a very reliable source, that ARM management seriously dislikes the Lima driver project.<br /><br />To put it nicely, they see no advantage in an open source driver for the Mali, and believe that the Lima driver is already revealing way too much of the internals of the Mali hardware. Plus, their stance is that if they really wanted an open source driver, they could simply open up their own codebase, and be done.<br /><br />Really?<br /><br />We can debate endlessly about not seeing an advantage to an open source driver for the Mali. In the end ARMs direct customers will decide on that one. I believe that there is already 'a slight bit of' traction for the general concept of open source software, I actually think that a large part of ARMs high margin products depend on that concept right now, and this situation is not going to get any better with ARMv8. Silicon vendors and device makers are also becoming more and more aware of the pain of having to deal with badly integrated code and binary blobs. As Lima becomes more complete, ARMs customers will more and more demand support for the Lima driver from ARM, and ARM gets to repeat that mantra: "We simply do not see the advantage"...<br /><br />About revealing the internals of the Mali, why would this be an issue? Or, let me rephrase that, what is ARM afraid of?<br /><br />If they are afraid of IP issues, then the damage was done the second the Mali was poured into silicon and sold. Then the simple fact that ARM is that apprehensive should get IP trolls' mouths watering. <i>Hey IP Trolls! ARM management believes that there are IP issues with the Mali! <a href="https://gitorious.org/lima/lima/commits/master" rel="nofollow">Here is the rainbow!</a> Start searching for your pot of gold now!</i><br /><br />Maybe they are afraid that what is being revealed by the Lima driver is going to help the competition. If that is the case, then it shows that ARM today has very little confidence in the strength of their Mali product or in their own market position. And even if Nvidia or Qualcomm could learn something today, they will only be able to make use of that two years or even further down the line. How exactly is that going to hurt the Mali in the market it is in, where 2 years is an eternity?<br /><br />If ARM really believes in their Mali product, both in the Mali's competitivity and in the originality of its implementation, then they have no tangible reason to be afraid of revealing anything about its internals.<br /><br />Then there is the view that ARM could just open source their own driver. Perhaps they could, it really could be that they have had very strict agreements with their partners, and that ARM is free to do what they want with the current Mali codebases. I personally think it is rather unlikely that everything is as watertight as ARM management imagines. And even then, given that they are afraid of IP issues... How certain are ARMs lawyers that nothing contentious slipped into the code over the years? How long will it take ARMs legal department to fully review this code and assess that risk?<br /><br />The only really feasible solution tends to be a freshly written driver, with a full development history available publically. And if ARM wants to occupy their legal department, then they could try to match intel (AMD started so well, but ATI threw in the towel so quickly, but luckily the AMD GPGPU guys continued part of it), and provide the Technical Reference Manual and other documents to the Mali. That would be much more productive, especially as that will already be more legal overhead than ARM management would be willing to spare, when they do finally end up seeing the light.<br /><br />So. ARM management hates us. But guess what. Apart from telling us to change our name (there was apparently the "fear" of a trademark issue with us using Remali, so we ended up calling it Lima instead), there was nothing that they could do to stop us a year and a half ago. And there is even less that ARM can do to stop us today :) <br /><br /><b>A full 6.0%...</b> urn:lj:livejournal.com:atom1:libv:24402 Q3A with open source generated shaders! 2013-03-18T18:28:48Z 2013-03-18T18:28:48Z Cornershop - Brimful of asha (norman cook remix) The combination of <a href="https://gitorious.org/lima/lima" rel="nofollow">limare</a> and <a href="https://gitorious.org/~cwabbott/open-gpu-tools/cwabbotts-open-gpu-tools" rel="nofollow">open-gpu-tools</a> can now run Quake 3 Arena timedemo without depending on the binary driver for the shader compiler!<br /><br /><a href="https://fosdem.org/2013/schedule/speaker/connor_abbott/" rel="nofollow">Connor Abbott</a> has been being his amazing (16y old!) self again in the weeks after <a href="https://fosdem.org/2013/schedule/event/maliisa/" rel="nofollow">his talk at FOSDEM</a>, and he pushed his compiler work in his <a href="https://gitorious.org/~cwabbott/open-gpu-tools/cwabbotts-open-gpu-tools" rel="nofollow">open-gpu-tools tree</a> to be able to handle basic vertex shaders. Remember that our vertex shader is a rather insane one, where the compiler has to work real hard on getting scheduling absolutely right. This is why an assembler for our vertex shader was not too useful and the most part of a compiler had to be written for it to generate useful results. A mammoth task, and Connor his vertex shader code is now larger than the code I have in my limare library.<br /><br />So it was high time that we brought limare and OGT together to see what they were capable of with some basic shaders. Luckily, the Q3A GLES1 emulation has basic shaders, what a nice coincidence :)<br /><br />So Connor turned my simple vertex shader essl into the high level language used by the OGT vertex shader compiler, and through steps described at <a href="http://limadriver.org/Compiling_Q3A_Shaders/" rel="nofollow">this wiki page</a>, turned them into MBS files (Mali Binary Shader - the file type output by the standalone compiler, and also by newer binary driver integrated compilers). Limare can then load and parse those MBS files, and run the shaders. No need to involve the ARM binary anymore when we have OGT generated MBS files :)<br /><br />The result was quite impressive. We had a few issues where the limare driver (which has mostly taken its cues from the output of the binary driver) and OGT disagreed over symbol layout, but apart from that, bringing up the shaders connor produced was pretty painless. Amazingly effortless, for such a big step.<br /><br />Connor then spent another day playing with the fragment shader assembler, fixed some bugs, and produced 3 fragment shaders for us. One for the clear shader used by limare directly, and 2 for Q3A. After some more symbol layout issues, these also just worked! We even seem to be error-margin faster with the MBS files (due to texture coordinate varyings being laid out differently).<br /><br />So this is a really big milestone for <a href="http://limadriver.org/" rel="nofollow">the lima driver project</a>. Even with our insane pre-optimized architecture, we now are able to run Quake 3 Arena without any external dependencies, and we are beating the ARM binary while doing so.<br /><br />For generating your own shader MBS files, check out <a href="https://gitorious.org/~cwabbott/open-gpu-tools/cwabbotts-open-gpu-tools" rel="nofollow">Connors OGT</a>, and then you can head straight to <a href="http://limadriver.org/Compiling_Q3A_Shaders/" rel="nofollow">Connors wiki page</a>. My <a href="https://github.com/libv/ioquake3" rel="nofollow">Q3A tree</a> now has the MBS code included directly. And i pushed <a href="https://gitorious.org/lima/lima/commits/fosdem2013_pile" rel="nofollow">a dirty version of my FOSDEM limare code</a>.<br /><br />As for this new limare code, <a href="https://gitorious.org/lima/lima/commits/fosdem2013_pile" rel="nofollow">this fosdem_2013_pile branch</a> will vanish soon, as i need to properly pry things apart still. This is run-for-the-price code, and often includes many unrelated fixes in the same commit. It's better to do archeology on it now, than 3y from now, so this needs to be split. But in the meantime, you all can go and give Q3A on a fully free driver stack on Mali hw a go :)<br /><br />I will not post a video, as there really is nothing new to see. It is the exact same timedemo, running some promille faster. Build things, and then run it yourself on your sunxi hardware (i am still working on porting it to the new kernel of a more powerful platform). That's the best proof there is!<br /><br />For building limare, check out <a href="https://gitorious.org/lima/lima/commits/fosdem2013_pile" rel="nofollow">the fosdem2013_pile branch</a> and then just run make/make install.<br /><br />For building Q3A all you need to do is run:<br /><pre>make ARCH=arm USE_LIMARE=1</pre> And, when you have the full quake installed in ~ioquake3/baseq3, you can create a file called ~ioquake3/baseq3/demofour.cfg with the following content:<br /><pre>cg_drawfps 1 timedemo 1 set demodone "quit" set demoloop1 "demo four; set nextdemo vstr demodone" vstr demoloop1</pre>You can then run the ioquake3 binary with "+exec demofour.cfg" added to the command line, and you will have the demo running on top of fully free software!<br /><br />Now we really have covered all the basics, time to find out how Mesa will play with our plans :) urn:lj:livejournal.com:atom1:libv:24092 Hey ARM! 2013-02-13T04:13:43Z 2013-02-13T04:13:43Z Pixies - Surfer Rosa - River Euphrates <h2>Quake 3 Arena code.</h2><br />I pushed out <a href="https://github.com/libv/ioquake3-mali_fb/tree/limare" rel="nofollow">the Quake 3 Arena code</a> used for demoing limare and for benchmarking.<br /><br />You can build it on your linux-sunxi with<br /><pre> make ARCH=arm</pre><br />or, for the limare version (which will not build without the matching limare driver, which i haven't pushed out yet :))<br /><pre> make ARCH=arm USE_LIMARE=1</pre><br />for the GLESv2 version (with the broken lamps due to missing alphatest):<br /><pre> make ARCH=arm USE_GLES2=1</pre><br />Get a full Quake 3 Arena version first though, and stick all the paks in ~/ioquake3/baseq3. Add this to demofour.cfg in the same directory:<br /><pre> cg_drawfps 1 timedemo 1 set demodone "quit" set demoloop1 "demo four; set nextdemo vstr demodone" vstr demoloop1</pre><br />To run the timedemo then run the quake binary with +exec demofour.cfg<br /><br />For your own reverse engineering purposes, to build the GLESv1 version with logging included, edit code/egl/egl_glimp.c, and remove the // before:<br /><pre> //#define QGL_LOG_GL_CALLS 1</pre><br />But be aware, you are not free to spread that dumped data. That is ID Software data, albeit in a raw form.<br /><br />I'd be much obliged if anyone hacks up input support, or re-adds sound. Or even adds the missing GLES2 shaders (color as a uniform for instance). That would make this code playable from the console, and should make it easier for me to provide playable limare code.<br /><br />As you all can see, we have nothing to hide. The relevant differences between GLES1 and limare is in the GL implementation layer. I did shortcut vertex counting, this to ease logging, but this only has limited effect on CPU usage. The lesser CPU usage of limare is not significant or interesting as we do less checking than a full driver anyway. In simpler tests (rotating cubes), our scheduling results in a much higher CPU usage though (like 50% more, from 10% to 15% :)), even if we are not significantly faster. As said in my previous, i am not sure yet whether to keep this, or to find some improvements. Further tests, on much more powerful hardware, will tell.<br /><h2>Connors Compiler Work.</h2><br />Connor had a massive motivation boost from FOSDEM (and did not suffer from the bug that was going round which so many of us in the last week). Earl from <a href="http://zareason.com/shop/home.php" rel="nofollow">zaReason</a> is sending him an A13 tablet, which should spur him on even further.<br /><br />He has been coding like a madman, and he is now close to being able to compile the relatively simple shaders used in the Quake 3 Arena demo. He still has to manually convert the ESSL of the vertex shader to his own GP_IR first though, but that's already massive progress which gets us very close to our goals.<br /><br />I am going to add MBS loading (Mali Binary Shader format) to limare to be able to forgo the binary compiler and load pre-compiled shaders into our programs. Since MBS is also spit out by <a href="https://gitorious.org/~cwabbott/open-gpu-tools/cwabbotts-open-gpu-tools" rel="nofollow">the open-gpu-tools</a>, we can then distribute our own compiled MBS files directly, and provide a fully open source Q3A implementation on mali.<br /><br />How cool is that!<br /><h2>The very near future.</h2><br />With <a href="http://libv.livejournal.com/23886.html" rel="nofollow">my post purely about Q3A and its numbers</a>, the reactions were rather strange. Seems like a lot of people were hung up exclusively on us being only 2% faster or because we were using this "ancient" game. The blog entry itself explained fully why this ancient game was actually a very good choice, yet only very few read it. Very few realized what a massive milestone an almost pixel-perfect Quake 3 Arena is for a reverse engineered driver.<br /><br />As for performance... When i started investigating the mali, i had postulated that we would be happy to have only 75% of the performance of the binary driver. I assumed that, even with performance per watt being mighty important in the mobile space, 75% was the threshold at which the advantages of an open source driver would outweight the loss of performance. This would then lead to only ARMs big partners would end up shipping ARMs own binaries. And for projects like CyanogenMod and proper linux distributions there would be no question about what to ship.<br /><br />With Q3A, and with the various rotating cubes, we now have proven that we can have a 100% match in performance. Sometimes we can even beat performance. All of this is general, no Q3A specific tricks here!<br /><br />This is absolutely unique, and is beyond even the wildest dreams of a developer of any reverse engineered driver.<br /><br /><b><u>Absolutely nothing</u></b> stops us now from delivering an open source driver that broadly matches the binary driver in performance! And this is exactly what we will be doing next!<br /><br /><h2>Hey ARM!</h2><br />We are not going away, we are here to stay. We cannot be silenced or stopped anymore, and we are becoming harder and harder to ignore.<br /><br />It is only a matter of time before we produce an open source graphics driver stack which rivals your binary in performance. And that time is measured in weeks and months now. The requests from your own customers, for support for this open source stack, will only grow louder and louder.<br /><br />So please, stop fighting us. Embrace us. Work with us. Your customers and shareholders will love you for it.<br /><br />-- libv. urn:lj:livejournal.com:atom1:libv:23886 Quake 3 Arena timedemo on top of the lima driver! 2013-02-06T01:08:47Z 2013-02-06T13:15:48Z Cinematic Orchestra - The revolution At <a href="https://fosdem.org/2013/" rel="nofollow">FOSDEM</a>, I had a mainline talk about <a href="https://fosdem.org/2013/schedule/event/operating_systems_open_arm_gpu/" rel="nofollow">"Open ARM GPU Drivers"</a>, going over all the projects and GPUs, talking about the developers doing the hard reverse engineering work and the progress that they have made so far. I will write up a blog entry summarizing this talk soon, but for now i will just talk about the Lima demo i showed at the end of the talk.<br /><br />Let me get straight to the main point before delving into details: We now have a limare (our proto/research driver) port of <a href="http://en.wikipedia.org/wiki/Quake_3_Arena" rel="nofollow">Quake 3 Arena</a> which is running the q3a timedemo <b>2% faster</b> than the binary driver. With <b>3% less cpu</b> overhead than the binary driver to boot!<br /><br /><a href="http://www.youtube.com/watch?v=l92S3wg2iSU" rel="nofollow">Here is the timedemo video up on youtube.</a> It is almost pixel-perfect, with just a few rounding errors introduced due to us being forced to use a slightly different vertex shader (ESSL, pulled through the binary compiler instead of a hand coded shader). We have the exact same tearing as the binary drivers, which are also not synced to display on the linux-sunxi kernel (but ever so slightly more tearing than the original ;)).<br /><br />This Q3A port is not playable for a few reasons. One is, i threw out the touchscreen input support, but never hacked in the standard SDL based input, so we have no input today. It should be easy to add though. Secondly, i only include the shaders that are needed for running the timedemo. The full game (especially its cut scenes) requires a few more shaders, which are even simpler than the ones currently included. I also need to implement the equivalent of glTexSubImage2d, as that is used by the cut-scenes. So, yes, it is not playable today, but it should be easy to change that :)<br /><br />We are also not fully open source yet, as we are still using the binary shader compiler. Even after begging extensively, Connor was not willing to "waste time" on hand coding the few shaders needed. He has the necessary knowledge to do so though. So who knows, maybe when i push the code out (the q3a tree is a breeze to clean, but the lima code is a mess, again), he might still give us the few shaders that we need, and we might even gain a few promille performance points still :)<br /><br />I will first be pushing out the q3a code, so that others can use the dumping code from it for their own GPU reverse engineering projects. The limare code is another hackish mess again (but <a href="https://gitorious.org/lima/lima/commits/master" rel="nofollow">not as bad as last time round</a>), so cleaning that up will take a bit longer than cleaning up q3a.<br /><h2>Why frag like it is 1999?</h2><br />Until now, i was mostly grabbing, replaying, and then porting, EGL/GLES2 programs that were specifically written for reverse engineering the mali. These were written by an absolute openGL/openGLES newbie, someone called <a href="http://libv.livejournal.com/" rel="nofollow">libv</a>. These tests ended up targetting very specific but far too limited things, and had very little in common with real world usage of the GPU. As most of the basic things were known for mali, it was high time to step up things a level.<br /><br />So what real world OpenGL(ES) application does one pick then?<br /><br />Quake 3 Arena of course. The demo four timedemo was the perfect next step for reverse engineering our GPU.<br /><br />This 1999 first person shooter was very kindly open sourced by <a href="http://en.wikipedia.org/wiki/Id_Software" rel="nofollow">ID Software</a> in 2005. <a href="http://omcfadde.blogspot.de/" rel="nofollow">Oliver McFadden</a> later provided <a href="http://omcfadde.blogspot.de/2010/01/ioquake3-on-n900.html" rel="nofollow"> an openGLES1 port of ioquake3 for the Nokia N900</a>. With the Mali binary providing an OpenGLES1 emulation library, it was relatively easy to get a version going which runs on the Mali binary drivers. Thank you Oliver, you will be missed.<br /><br />The Q3A engine was written for fixed 3D pipelines and this has some very profound consequences. First, it limits the dependency on the shader compiler and allowed me to focus almost purely on the command stream. This completely fits with the main strategy of our reverse engineering project, namely it being 2 almost completely separate projects in one (command stream versus shader compilers). Secondly, and this was a nice surprise when i started looking at captures, the mali OpenGLES1 implementation had some very hardware specific optimizations that one could never expose with OpenGLES2 directly. Q3A ended up being vastly more educational than I had expected it to be.<br /><br />With Q3A we also have a good benchmark, allowing us to get a better insight into performance for the first time. And on top of all of that, we get a good visual experience and it is a dead-certain crowdpleaser (and it was, thanks for the cheers guys :))<br /><br />The only downside is that the data needed to run demo four is not available with the q3a demo release and therefor not freely downloadable. Luckily you can still find Q3A CDs on ebay, and i have heard that steam users can easily download it from there.<br /><h2>The long story</h2><br />After linuxtag, where i <a href="http://www.linuxtag.de/2012/de/program/program/vortragsdetails-talkid481.html" rel="nofollow">demoed the rotating companion cube</a>, I assumed that my knowledge about the mali was advanced enough that bringing up Q3A would take only a given number of weeks. But as these things usually go, and with work an real life getting in the way, it never pans out like that. January 17th is when i had q3a first work correctly, time enough to worry about some optimization still before FOSDEM, but only just enough.<br /><br />I started with an android device and the kwaak3 "app", which is just Olivers port with some androidiness added. I captured some frames to find out what i still missed with limare. When i finally had some time available, i first spent it cleaning up the linuxtag code, which i pushed out early december. I had already brought up Q3A on linux-sunxi with the mali binary drivers, which can be seen from <a href="http://www.youtube.com/watch?v=k4LbQGdwslA" rel="nofollow">the video i then published on youtube.</a><br /><br />One thing about the youtube video though... Oliver had a tiny error in his code, one that possibly never did show up on the N900. In his version of the texture loading code, the lightmaps original format would end up being RGB whereas the destination format is RGBA. This difference in format, and in-driver conversion, is not supported by the openGLES standard. This made the mali driver refuse to load the texture, which later on had the driver use only the primary texture, even though a second set of texture coordinates were attached to the command stream. The vertex shader did not reflect this, and in my openGL newbieness i assumed that Ben and Connor had a bug in their vertex shader disassembler. You can clearly see the flat walls in the video i posted. Once i fixed the bug though, q3a suddenly looked a lot more appealing.<br /><br />I then started with turning the openGLES1 support code in Quake's GLimp layer into a dumper of openGLES1 commands and data in a way that made it easy to replay individual frames. Then i chose some interesting frames, and replayed them, turned them into a GLES2 equivalent (which is not always fully possible, alphaFunc comes to mind), and then improved limare until it ran the given frames nicely through (the mali has hw alphaFunc, so limare is able to do this directly too). Rince and repeat, over several interesting frames.<br /><br />By the evening of January the 16th, i felt that i knew enough to attempt to write a GLimp for limare. This is exactly when my father decided to give me a call. Some have met him at Le Paon last Friday, when he, to my surprise, joined us for a beer after work as his office is not far away. He remarked that i seemed "a bit on edge" when he called on the 16th. Yes, i indeed was, and how could i be anything else at a time like this :) I hacked all night, as at the time i was living purely at night anyway, and minutes before my girlfriend woke up i gave it my first shot. Crash, a stupid bug in my code. I told my girlfriend that i wouldn't join her for "breakfast" before i went to bed, as i was simply way too close. By the time she left for work, i was able to run until the first few in-game frames, when the rendering would hang, with the mali only coming back several seconds later. After a bit of trying around, i gave the GP (vertex shader) a bit more space for its tile heap. This time it ran for about 800 frames before the same thing happened. I doubled the tile-heap again, and it ran all the way through!<br /><br />The evening before i had hoped that i would get about 20fps out of this hardware. This already was a pretty cocky and arrogant guess, as the binary driver ran this demo at about 47.3fps, but i felt confident that the hardware had little to hide. And then the demo ran through, and produced a number.<br /><br /><b>30.5fps</b><br /><br />Way beyond my wildest dreams. Almost <b>65%</b> of the performance of the binary driver. Un-be-liev-ab-le. And this was with plain sequential job handling. Start a GP job, wait for it to finish, then start the PP job, wait for it to finish, then flip. 30.5fps still! Madness!<br /><br />I had two weeks left for FOSDEM, so i had a choice, either add input support and invite someone from the public to come and play before the audience, or, optimize until we beat the binary driver. The framerate of the first pass decided that, optimization it was. I had a good benchmark, and only a third of the performance needed to be found, and most of the corners for that extra performance were known.<br /><br />My first optimization was to tackle the PP polygon list block access pattern. During my previous talk at FOSDEM, i explained that this was the only bit I found that might be IP encumbered. In the meantime, over the weekly beers with Michael Matz, the SuSE Labs toolchain lead, i had learned that there is thing called the <a href="http://en.wikipedia.org/wiki/Hilbert_curve" rel="nofollow">"hilbert space filling curve"</a>. Thanks Matz, that was worth about <b>~2.2fps</b>. I benchmarked another few patterns: two level hilbert (inside plb block, and out), and the non-rotated hilbert pattern that is used for the textures. None would give us the same performance as the hilbert curve.<br /><br />Building with -O3 then gave us another <b>1.5fps</b>. Passing vec2s between the shaders gave us <b>0.3fps</b>. It was time to put in proper interleaved job handling. With the help of Marcus Meissner (the SuSE Security lead), an ioctl struct sizing issue was found for the job wait thread. This fixed the reliability issues with threading on the r3p0 kernel of linux-sunxi. (ARM! Stable kernel interfaces now!) But thanks Marcus, as proper threading and interleaved job handling put me at <b>40.7 fps!</b><br /><br />And then i got stuck. I only had 40.7fps and knew nothing that could account for such a big gap in performance. I tried a few things left and right, but nothing... I then decided to port q3a to GLES2 (with the loss of alphafunc and buggered up lamps as a result) to see whether our issue was with the compiled versus hand-coded shader. But I quickly ran into an issue with multi-texture program state tracking, which was curious, as the lima code was logically the same. Once this was fixed the GLES2 port ran at about 47.6fps, slightly faster than GLES1, which i think might be because of the lack of alphafunc.<br /><br />Immediately after that i ported the multi-texture state tracking fix to the limare GLimp, but i sadly got no change in framerate out of it. Strangely, it seemed like there was no multitexturing going as my debugging printfs were not being triggered. I then noticed the flag for telling Q3A that our GL implementation supports multitexturing. Bang. <b>46.7fps</b>. I simply couldn't believe how stupid that was. If that had been correct on the first run, i would've hit <b>above 75%</b> of the framerate, how insane would that have been :)<br /><br />For the final 1.5fps, which put us at <b>48.2fps</b>, i added a third frame, this while only rendering out to two framebuffers. Job done!<br /><br />Adding a fourth frame did not improve numbers, and i left some minute cpu usage and memory usage optimizations untouched. We are faster than the binary driver, while employing no tricks. We know what we need to know about this chip and there is nothing left to prove with Q3A performance.<br /><h2>The numbers.</h2><br />The fact that we are slightly faster is actually normal. We do not have to adhere to the OpenGLES standard, we can do without a lot of the checking that a proper driver normally needs to do. This is why the goal was not to match the binary driver's performance, but to beat it, which is exactly what we achieved. From some less PP and CPU bound programs, like the spinning cubes, it does seem that we are more aggressive with scheduling though.<br /><br />Now let's look at some numbers. Here is the end of the timedemo log for the binary driver, on an Allwinner A10 (single cortex a8, at 1GHz), with a Mali-400MP1 at 320MHz, rendering to a 1024x600 LCD, with framerate printing enabled:<br /><pre> THEINDIGO^7 hit the fraglimit. marty^7 was melted by THEINDIGO^7's plasmagun 1260 frames 27.3 seconds 46.2 fps 10.0/21.6/50.0/5.6 ms ----- CL_Shutdown ----- RE_Shutdown( 1 ) -----------------------</pre><br />And here is the end of the timedemo log for the limare port:<br /><pre> THEINDIGO^7 hit the fraglimit. marty^7 was melted by THEINDIGO^7's plasmagun ]64f in 1.313632s: 48.719887 fps (1280 at 39.473158 fps) 1260 frames 26.7 seconds 47.2 fps 9.0/21.2/74.0/5.6 ms ----- CL_Shutdown ----- RE_Shutdown( 1 ) ]Max frame memory used: 2731/4096kB Auxiliary memory used: 13846/16384kB Total jobs time: 32.723190 seconds GP job time: 2.075425 seconds PP job time: 39.921429 seconds -----------------------</pre><br />Looking at the numbers from the limare driver, my two render threads are seriously overcommitted on the fragment shader (PP). We really are fully fragment shader bound, which is not surprising, as we only have a single fragment shader. Our GP is sitting idle most of the time.<br /><br />It does seem promising for a quad core mali though. I will now get myself a quad-core A9 SoC, and put that one through its paces. My feeling is that there we will either hit a wall with memory bandwidth or with the CPU, as q3a is single threaded. Since limare does not yet support multiple fragment shaders the last remaining big unknown will get solved too.<br /><br />Another interesting number is the maximum frame time. 50.0ms for the binary driver, versus 74.0ms for limare. My theory there is that i am scheduling differently than the original driver and that we get hit by us overcommitting the fragment shader. Wait and see whether this difference in scheduling will improve or worsen the numbers on the potentially 4 times faster SoC. We will not be context switching anymore with our render threads, and we will no longer be limited by the fragment shader. This should then decide whether another scheme should be picked or not.<br /><br />Once we fix up the Allwinner A10 display engine, and can reliably sync to refresh rate, this difference in job scheduling should become mostly irrelevant.<br /><h2>The star: the mali by falanx.</h2><br />In the previous section i was mostly talking about the strategy of scheduling GP and PP jobs, of which one tends to have 1 of each per frame. Performance optimization is a very high level problem on the mali, which is a luxury. On mali we do not need to bother with highly specific command queue patterns which most optimally use the available resources, which then ends up being SoC and board specific. We are as fast as the original driver without any trickery, and this has absolutely nothing to do with my supposed ability as a hacker. The credit fully goes to the design of the mali. There is simply no random madness with the mali. This chip makes sense.<br /><br />The mali is the correct mix of the sane and the insane. All the real optimization is baked into the hardware design. The vertex shader is that insane for a reason. There is none of that "We can fix it in software" bullshit going on. The mali just is this fast. And after 20 months of throwing things at the mali, i still have not succeeded in getting the mali to hard or soft lockup the machine. Absolutely amazing.<br /><br />When i was pretty much the only open source graphics developer who was pushing display support and modesetting forwards, I often had to hear that modesetting was easy, and that 3d is insane. The mali proves this absolutely wrong. Modesetting is a very complex problem to solve correctly, with an almost endless set of combinations that requires very good insight and the ability to properly structure things. If you fail to structure correctly, you have absolutely no chance of satisfying 99.9% of your users, you'll be lucky if you satisfy 60%. Compared to modesetting, 3D is clearly delineated, and it is a vastly more overseeable and managable problem... Provided that your hardware is sane.<br /><br />The end of the 90s was an absolute bloodbath for graphics hardware vendors with just a few, suddenly big, companies surviving. That's exactly when a few Norwegian demo-sceners, at the Trondheim University, decided that they would do 3D vastly better than those survivors and they formed a company to do so, called <a href="http://en.wikipedia.org/wiki/Falanx" rel="nofollow">Falanx</a>. It must've seemed like suicide, and I am very certain that pretty much everybody declared them absolutely insane (<a href="http://www.engadget.com/2005/02/03/falanxs-malis-console-quality-3d-on-cellphones/" rel="nofollow">like engadget did</a>). Now, 12 years later, seeing what came out of that, I must say that I have to agree. Falanx was indeed insane, but it was that special kind of insanity that we call pure genius.<br /><br />You crazy demo-sceners. You rock, and let this Q3a port be my salute to you. urn:lj:livejournal.com:atom1:libv:23584 FOSDEM Aftermath. 2012-02-10T12:56:44Z 2012-02-10T12:56:44Z QOTSA - QOTSA - Regular John FOSDEM was awesome this year. We had an overbooked schedule for our DevRoom, we inaugurated the beautiful and fantastic K building, and i got to present the lima driver.<br /><br />First off, i would like to thank the FOSDEM organizers and the ULB. The already unique event that is FOSDEM just keeps getting better and better. Pascal & friends: congratulations, like every year, you've outdone yourselves.<br /><br />Secondly, i would like to thank all the speakers in my devroom. It is clear by now why the first-come-first-serve algorithm has to be used, and it is also clear that it is working. But thank you all for making this a successful event (even Chris, who couldn't make it due to a train derailment). I hope you guys had a lot of fun too, both during your talk and with the rest of FOSDEM.<br /><br />Lastly, to all those who attended my talk (and those who couldn't get in anymore as well): Thank you all for your very positive feedback. No matter what happens with lima in future, this talk will be the most memorable moment. (oh, and a big thanks to Will Stephenson, from SuSE and KDE, for getting a webcam up that quickly). To whoever shouted something along the lines of "we don't see that, it looks like a perfect cube to us" when the caching went off in the rotating cube hack: this is the open source spirit in its most tangible form. Thank you very much.<br /><br />To end this post, let me plug <a href="http://limadriver.org/" rel="nofollow">the lima website</a> again. We also have <a href="http://vlists.pepperfish.net/cgi-bin/mailman/listinfo/lima-limadriver.org" rel="nofollow">a mailinglist</a> and the #lima channel on freenode. <a href="https://gitorious.org/lima/lima" rel="nofollow">The limare code</a> has been available since yesterday night. <a href="http://www.heise.de/newsticker/meldung/Open-Source-Treiber-fuer-ARM-Grafik-1432120.html" rel="nofollow">Heise</a> and <a href="https://lwn.net/Articles/480457/" rel="nofollow">lwn</a> posted the story already, and the videos from the FOSDEM talk should soon hit <a href="http://www.phoronix.com/scan.php?page=home" rel="nofollow">phoronix</a> as well. urn:lj:livejournal.com:atom1:libv:23462 XDC 2012: Nuremberg! 2011-12-09T04:32:10Z 2011-12-09T04:32:10Z Downtown rush echoing... Yes! The board has decided! XDC comes to Nuremberg!<br /><br />For 2012 we (Egbert Eich, Professor Hopf, and I) will be hosting the annual X conference in Nuremberg!<br /><br />Egbert will try to get the main SuSE conference room, or, failing that, Matthias will try to get us a university aula, so the venue itself will work itself out beautifully in one way or another. Then... Nuremberg is one of those places which is perfect for large crowds who need food and some liquids in the evening (frankonian/bavarian beergarten culture), so it is the perfect (and highly affordable) conference area from that point of view. And, the best part, even though Nuremberg is not the international hub that Frankfurt is, or the european hub that Munich is, it is halfway between the two, and travel is relatively easy from either of those points, either you take the plane, or you take a much more comfortable train from either airport, and get to Nuremberg in pretty much the same time. You can really make a big save comparing those two airports when flying inside european aerospace, and this for no time difference. One insider tip though: you get to ride the ICE at full speed (300+km/h!) when traveling from Munich (you do have to endure the rather pedestrian S-bahn for 45 minutes though).<br /><br />Anyway, the main action item now is that Egbert can start to poke SuSE to see when their main conference room is available for 3 days in september 2012 (working network and enough power sockets are a given then!). I doubt that we will get an answer still in the three remaining weeks of this year.<br /><br />The actual proposal e-mail sent to the board is sadly only available to X.org foundation members, but a wiki page will soon be created which recreates most of that information. But rest assured, we will get close to the wonderful experiences of XDC Toulouse (thank you Matthieu!) and XDC chicago (thank you Michael!) indeed!<br /><br />(oh, and btw, we have <a href="http://wiki.x.org/wiki/fosdem2012" rel="nofollow">a FOSDEM DevRoom</a> this year, which is rapidly getting its schedule filled! If you are coming, get your talk in right now: first come, first serve!) urn:lj:livejournal.com:atom1:libv:23285 Wheelbuilding... 2011-08-12T22:46:01Z 2011-08-12T22:46:01Z Portishead - Glorybox (Bicycle-) Wheel building is an art. An art perfectly suited for a geek; it requires technical insight, knowledge, feeling and some experience. For those interested, here are some tips and pointers from my own experience.<br /><br />1) Buy <a href="http://www.wheelpro.co.uk/wheelbuilding/book.php" rel="nofollow">"Professional guide to Wheel Building" by Roger Musson</a>, it is going to be the best 9GBP you have spent in a long long while. [HINT1]<br />2) Read it, twice!<br />3) Buy the rims and hubs before you buy the spokes (and get the necessary tools too if you haven't already).<br />4) Measure the ERD (Effective Rim Diameter) using the old-spokes with glued-on nipple method that Roger describes [HINT2].<br />5) Buy the spokes that spocalc.xls then calculates for you.<br />5) Lace your wheel like Roger describes, to the letter.<br />6) Tension your wheel like Roger describes, to the letter [HINT3].<br /><br />HINT1: Do not read any other sources, Gerd Schraner's book is just pure nostalgia and does not help you much. Especially his explanation for tensioning your spokes should be ignored: while it might get you a straight wheel, your spokes might have wildly varying tension, and are therefor likely to either break due to fatigue or have the wheel go out of true quickly.<br /><br />HINT2: For creating the cut-off spokes for measuring the ERD as Musson describes; screw your nipples onto your spokes so that your spoke only _just_ comes out of the nipple into the groove for the nipple-driver. This is the measuring length you should use. If you use the absolute top of the nipple for measuring the length, then you will have no room for error, and you will very likely use up all of the thread on the spoke while bringing the wheel up to full tension (this is the experience bit right here). If it is still inside the nipple, then you most likely will end up with too short a spokes, with thread still showing, this too is a nightmare for wheel-building (your nipple-driver will not disengage). Once you bring your wheel up to its final tension, the spoke (especially double butted spokes) will come slightly further out of the nipple as with the measurement-spokes.<br /><br />HINT3: For the final stage of tensioning, where the spokes tend to turn with the spoke-key, I marked the rim-sides of the spokes with different colour alcohol markers. This gave me the ability to view the turning of the spokes, and to undo it, close to the rim and nipple, without hampering the spoke-key. Since this is an alcohol based marker on stainless steel, you can rub it off afterwards, or you could just take some alcohol to wipe it off. I just kept it on now, knowing full well that most of it will disappear soon enough in the rain and mud.<br /><br />I am using Extreme Airline 3s, which i got from <a href="http://www.roseversand.de/" rel="nofollow">Rose</a>. These are rather deep rims that are very stable and sturdy, and they have a wear-indicator still. The joint is not done well, and you will always have a third or so of a mm difference in diameter there, but for trekking or mtb tires, this is no issue, it is just annoying when working on the wheels in the stand. Because these rims are so sturdy, the Schraner method becomes quite unreliable, you can much more easily get away with differently tensioned spokes, as the rim is much more likely to even differences out for you instead of showing where the differences are. You actually need to pluck the spokes instead, like Musson describes, early on in the tensioning process, to get rid of the differences in tone and therefor tension.<br /><br />I ordered a pre-built set of Airline 3s (28" with LX hub and 3N72 dynamo) from Rose more than a year ago, and they seem quite sturdy and have served me well so far. But, sadly, these pre-built wheels were not up to tension, which I could hear on steep climbs as the spokes were rubbing against eachother with heavy and changing load. They were subsequently very hard to tension further, my guess is because of badly oiled nipples before assembly.<br /><br />Recently I built a first set according to the Schraner method, and while this went well, and the wheels feel good, I am not sure how good they really are as I haven't used them yet. It could be that they go out of true quickly, especially after pumping quite a bit of heat in them going down some slope in the Fränkische Schweiss. The spoke lengths I used for the 28" Extreme Airline 3 rims, triple crossed (of course!) and with 12mm nipples are 276mm for the front, 281/283 for the back.<br /><br />For the 26" version of the same rim, with the same hubs, same lacing, same nipples, I used 246mm for the front, and 251/253 for the back. This was calculated with spocalc after measuring the rim, according to Musson, and the ERD is 523mm (after correcting for my mistake). These wheels are for a <a href="http://velotraum.de" rel="nofollow">velotraum cross crmo frame</a> that I am just now building up, so there are no kms on them either, but I have a very very good feeling about them, as I did use Mussons book for them, and the wheels came together as good or even better than described. So while my own handiwork is still untested in real-life conditions, at least I can tell the difference between Schraner and Musson, and the spoke lengths are (now) correct too ;)<br /><br />In any case, if you are into cycling, have done all other jobs around the bicycle, and mastered them, already, then try your hand at wheel-building to complete the skill-set. It is not black magic, it is actually highly logical, but you should not use sentences like "How hard can it be?" or "Right, I'll get my hammer!" when doing so. Read the right book, get the right tools and the correct spokes, and then take your time; it is really very rewarding. urn:lj:livejournal.com:atom1:libv:22968 This way, the free software desktop is never going to make it. 2011-01-15T15:52:40Z 2011-01-15T15:52:40Z In order to get easier access to Nokia things, and to boost security (as in, encrypt stuff, for a change), I've been reinstalling my trusted hp 6715b. Most nokians use ubuntu, so i went for 10.4LTS. I already severely disliked the way in which you have no installation options to chose from. You get the grandmother version every time, no "i have a clue, let me decide what i want to do, myself" button anywhere.<br /><br />I was lucky, in 10.4 my now 3y old graphics card was still working out of the box. But, of course, i want to have my big virtual screen back. This, of course got dropped with randr 1.2 and the Virtual keyword was reused for something else. Matthias Hopf then re-added it in 1.3; mostly to appease me, and the handful of other weirdos out there. But, try finding this option in the xorg.conf manpage. Nothing! Try googling for it, and the first 50 hits either only explain the commandline version or the old style Virtual (which got broken). Apparently, you need to add 'Option "Panning" "${H}x${V}".<br /><br />Easy, pico /etc/X11/xorg.<tab>. Damn. Nothing. head /var/log/Xorg.0.log says xorg.conf.d. Type man xorg.conf.d. Damn. Nothing: "No manual entry for xorg.conf.d" Suuuper. Apparently people are supposed to _know_ that this is part of the xorg.conf manpage.<br /><br />So, create a new screen, device and monitor section in 01-screen in xorg.conf.d, and press ctrl-alt-backspace, like any experienced driver developer is used to. Damn. Nothing. Head into gnome preferences stuff, enable key combination. Try again. Drop into the console. Wait for the display manager to try again. And wait. And wait. Damn. Nothing again. Ok, the DM might have died, and i don't trust this new gnome stuff, so it might be better to reboot. So Ctrl-alt-del, which worked first time round. At least something one can depend on.<br /><br />Next time i look back, ubuntu is showing its plymouth style loading, but the panel is gradually turning white. Something is not driving the panel and the driver died, for whatever reason. WTF? Try some key combinations to get a console. Damn, nothing! Pinging the box still worked, but of course, no sshd was installed. Attempting a reboot didn't bring anything either, it just runs into the exact same issue. Nothing is checking whether a previous boot got one to a working console or a working X.<br /><br />So, insert the ubuntu installation cd, choose live system, mount the fs, chroot to it, apt-get install ssh, and less /var/log/Xorg.0.log to reveal:<br /><br />&gt; (==) Using config directory: "/etc/X11/xorg.conf.d"<br />&gt; Parse error on line 3 of section Monitor in file /usr/lib/X11/xorg.conf.d/01-screen.conf<br />&gt; The Option keyword requires 1 or 2 quoted strings to follow it.<br />&gt; Parse error on line 3 of section Monitor in file /usr/lib/X11/xorg.conf.d/01-screen.conf<br />&gt; "2560x1920" is not a valid keyword in this section.<br />&gt; (EE) Problem parsing the config file<br />&gt; (EE) Error parsing the config file<br />&gt; <br />&gt; Fatal server error:<br />&gt; no screens found<br /><br />I forgot to put apostrophes around "Panning", and i got greeted with a bleeding panel, with no option to easily get around it. What on earth are we thinking here?<br /><br />This is Ubuntu LTS, with radeon, KMS, plymouth and xorg.conf.d. 5 nails in the free software desktops coffin. urn:lj:livejournal.com:atom1:libv:22693 OpenSuSE 11.1 and a recent scanner 2011-01-05T14:24:39Z 2011-01-05T14:24:39Z Wax Tailor - Que sera As some might know, I am switching ("intermediate") employers, and i am going to do home-office from now on. Home-office probably has tons of advantages, but one disadvantage is that you need to own your own office hardware, like a printer and a scanner. Such beasts were sitting around in Belgium, but in the 3.5 years that i have been in Nuernberg, i have either depended on the office i was respectively working for, or i ran to the copyshop around the corner. The latter is extremely unpractical and becomes rather expensive.<br /><br />So, today, a Canon Canoscan LiDE 110 arrived from amazon (plus a basic samsung laser), and i have just succeeded in getting it to work with openSuSE 11.1, albeit in a very unscientific way. Here is how.<br /><br />Sane is divided in front and backends. openSuSE 11.1 requires just an updated backend.<br /><br />For the LiDE 110, only very recent sane (git from halfway december 2010) supports the LiDE 110 and 210, so grab <a href="http://git.debian.org/?p=sane/sane-backends.git" rel="nofollow">the git repo</a>.<br /><br />Then, grab a recent openSuSE sane-backends package, for instance from <a href="https://build.opensuse.org/package/show?package=sane-backends&amp;project=home%3Ajimfunk" rel="nofollow">here</a>. Get yourself the src.rpm, and install it.<br /><br />A crude way of getting something our specfile can work with is to tar -jc up the sane-backends git repo, and to move that to /usr/src/packages/SOURCES<br /><br />Then edit the specfile, make sure to bump the "Version" and/or "Release" directives. Then have the "Source0" directive is pointing to the correct tarball, and make sure that the line with "%setup" is pointing to the right %{name}-... directory.<br /><br />If you are as lucky as i was today, the existing patches, which are mostly about integration, will apply rather cleanly, and rpmbuild will succeed.<br /><br />Install the created files (you probably won't need -devel), and you should now be able configure your scanner using yast. If yast complains about hal, then run rchal restart.<br /><br />Now scanimage -L should be happy, and then you're all set.<br /><br />Happy scanning! urn:lj:livejournal.com:atom1:libv:22502 The linux desktop is dead! 2010-09-17T13:36:10Z 2010-09-17T13:36:10Z Or so it will be, soon, if <a href="http://lists.x.org/archives/xorg-devel/2010-September/012807.html" rel="nofollow">these guys</a> get their way.<br /><br />Apparently, and this has been the hot new idea for the last year or two; for Xserver 1.10 people want to get rid of one of the greatest things that XFree86 brought us, and one of the better changes that happened after the X.org fork: modular graphics drivers.<br /><br />While the current proposal is simply to undo the modularization work of the mid-naughties (thanks <a href="http://en.wikipedia.org/wiki/Jeremy_Clarkson" rel="nofollow">jezza!</a>), it immediately sparked the imagination of others to go even further (to which <a href="http://lists.x.org/archives/xorg-devel/2010-September/012941.html" rel="nofollow">Alanc answered rather strikingly</a>). But merging drivers back is in itself already a very damaging move.<br /><br /><b>So what is the goal behind merging drivers?</b><br /><br />The official reason for this is "cleaning up the API", but I fail to see any logical link between being able to clean up APIs and mashing everything together.<br /><br />There is simply nothing that stops APIs from being improved when drivers are not a full and whole part of the xserver build tree.<br /><br />A mashed-together tree has no more advantage than a buildsystem like <a href="http://tinderbox.freedesktop.org/" rel="nofollow">our tinderbox</a>.<br /><br />And having modular drivers does not mean that one has to have a fully static API and ABI, you just need to have dependable ABI bumping, and, for the sake of overhead, sane and forward-looking API changes. Free software drivers are of course best able to keep in sync with API changes, but this is no different whether they are external or internal to the server build tree.<br /><br />However, there is a difference in how one approaches API cleanups in a modular world, as one needs to think a bit more about how to do such API changes. This often leads to a cleaner design, a better structure, and it often means that people spend time trying to understand existing code, and how to best adjust it to fit the new needs, without throwing out the baby with the bathwater. By moving the drivers into the xserver tree, and outlawing the API, we will only open the door for having a libpciaccess type breakage every month.<br /><br />So maybe this is the real wish behind wanting to merge back drivers: being able to pull crazy stunts with halfarsedly designed, badly structured and untested code, without implications, without accountability.<br /><br />Apart from APIs degrading further, there are other more fundamental issues with this, with actually far reaching consequences.<br /><br />When tying in the graphics drivers with the X server, the only way one could get driver updates, to get bugfixes, new features or new hardware support, is by installing a new Xserver.<br /><br />This is probably going to be claimed as a benefit, <a href="http://lists.x.org/archives/xorg-devel/2010-April/007020.html" rel="nofollow">as people want more testing of upstream code</a>, but a slight increase in usage of upstream code, will mean a much bigger decrease in userbase on released code, and people will be even more afraid of updating anything in their system than today.<br /><br /><b>But this is how the kernel does it!</b><br /><br />We've all heard this from our mothers: "If some other kid jumps off a cliff, is that a reason to jump off that cliff as well?"<br /><br />Basically, while it might be a good idea for the often much simpler devices that have rather complete drivers (at least compared to graphics drivers :)) in the kernel to be a full and whole part of the kernel, it does not and will not work well for graphics drivers.<br /><br />The complexity and the amount of movement in graphics drivers, especially with the many parts staying in userspace and the very unstable interfaces to them, makes this rather messy. And the only way that this is feasible is when those drivers are rather stable, and they definitely need to have a very stable ABI to userspace.<br /><br />No-one will be able to maintain such a level of stability for graphics drivers, and i am sure that no-one will stand up to defend going that route, if this requirement is mixed into the discussion.<br /><br /><b>How to sneak in a 1 to 1 version dependency between xserver, mesa and the linux kernel... Pt. 1.</b><br /><br />In January this year, in the run-up to xserver 1.8, there was a commit to the xserver, labelled <a href="http://cgit.freedesktop.org/xorg/xserver/commit/?id=b68f0204" rel="nofollow">"xserver: require libdri 7.8.0 to build"</a>, where an autoconf rule was added to depend on this version of "libdri". I believe that this was mainly because of DRI2 changes.<br /><br />When I say depend here, there is not a complete dependency on a given version of libdri. One can always build the xserver without any DRI support whatsoever. But who, on the desktop, really wants that today?<br /><br />So while this all-or-nothing decision is in itself questionable, there is another question to be asked here: what is this libdri?<br /><br />There is a dri.pc on most systems today, and there is a libdri.so on most systems today. The former is a package config file coming from the mesa tree, the latter, is an xserver internal convenience library (hence the lack of so versioning). Smells fishy, doesn't it?<br /><br />Now, while you might want to spend time looking high and low for the libdri from mesa, you will not find it. Mesa comes with 10 or more different libdris, one for each driver it supports, with the whole of the mesa linked in statically, in the form of driver_dri.so...<br /><br />Urgh, how broken is that?<br /><br />So, the xserver now depends on 10 or more different, driver specific, enormous binaries, all because its dri support now depends on a given version of the dri protocol. Or, re-stating that, the xserver depends on a very specific version of the monolithic, 80s style, mesa tree.<br /><br />Expanding the logic for the xserver and the drivers: why not just mash the mesa and xserver trees together then? :)<br /><br /><b>More parts come into play... (or dependency Pt. 2)</b><br /><br />The xserver depends on the standard drm infrastructure, and this is compatible up to a 4+ year old release of libdrm, namely version 2.3.0, as the basic libdrm code has barely changed since.<br /><br />Mesa, however, is a different story altogether. It depends, hard, on the latest version of libdrm, and this has been so since Oktober 2008, when intel introduced libdrm_intel in libdrm 2.4.0.<br /><br />In essence, this libdrm_intel is nothing more than a driver-stack internal convenience library. It only contains code that is specific for intel hardware and the only dependencies are parts of the intel driver stack (if those parts were living separately already). There are no direct dependencies from anything else.<br /><br />But, ever since Oktober 2008, both the intel x driver and the intel mesa driver depend on the latest libdrm version, and since then, both radeon and nouveau joined in the frenzy.<br /><br />So, while there might be some backwards compatibility between dri drivers and libdrm drivers, the reality is that intel, radeon and nouveau are today playing hopscotch. Because mesa is monolithic, and at least one of its drivers is going to depend on the latest libdrm version, the whole of monolithic mesa simply depends on the latest libdrm version.<br /><br />Since mesa has been depending on the latest libdrm for a few years now, and the xserver has been depending on the latest mesa version since the start of 2010, in turn, the xserver now depends on the latest libdrm version.<br /><br />Nice!<br /><br /><b>How does this tie in the kernel? (dependency Pt. 3).</b><br /><br />Well, since libdrm has the driver specific sublibraries, those of course call drm driver specific ioctls, and of course, these ioctls change all the time. While some people claim that they try to abstract at this layer (and that this strategy is good enough for everyone...), and claim to try to keep the kernel to userspace interface stable, this of course is only true for a very limited range of kernel and userspace parts. Now, we have intel, radeon _and_ nouveau playing at this level, dividing whatever median compatibility range there is, by three.<br /><br />The result is that libdrm can pretty much only be backwards compatible to the kernel by accident.<br /><br />So, continuing our logic from earlier, the latest xserver depends on the latest mesa, the latest libdrm and the latest kernel.<br /><br />Smashing lads! Well done! And all of this on a set of connections and foundations that make a house of cards look like a block of granite.<br /><br /><b>The root of the problem.</b><br /><br />Graphics hardware is horribly complex. Several years ago, a single graphics card already broke the terraflop boundary, managing what a huge IBM supercomputer only managed a good decade earlier. Single graphics cards come with many hundreds of shader cores, running at frequencies above 1Ghz, have multiple gigabytes of ram, eat 200+ Watts, and can drive up to 6 displays today. There is no other single piece of hardware which is this complex.<br /><br />And this complexity is of course also there in software.<br /><br />You cannot count the different parts of a modern graphics driver stack on free software on one hand anymore. There is the kernel drm part, the firmware, the libdrm part, the X driver, a pair of mesa drivers, an xvmc and possibly another media acceleration driver. A graphics driver stack, can be made up of up to 8 parts today.<br /><br />All of those parts are scattered over the system. There is 2 parts shipped with the kernel, 1 part shipped with libdrm, 2 drivers shipped with mesa, and the remainder can be found in an xf86-video tree.<br /><br />Naturally, in order to work most optimally, these different parts have a very direct and acute dependency on each other. Bugs, new features and new hardware support usually incur changes to interfaces between those different parts all the time.<br /><br />The way that those different parts are spread all over the place today make it almost impossible to have an optimal setup. Most of the time one is glad if it works at all. What's more, this spread is the core reason for the de-facto 1-1 version tie between kernel, libdrm, xserver and mesa.<br /><br /><b>The consequences of a 1-1 version tie between kernel, xserver and mesa.</b><br /><br />With graphics hardware and graphics drivers being this complex, there is simply no way to have them in a bugfree or a constant "useful" state.<br /><br />We just _have_ to live with the fact that graphics drivers will be buggy, and we should try to handle this as gracefully as possible.<br /><br />This means that we should be able to replace all or parts of the graphics driver stack at any time, without negatively affecting other parts of the system.<br /><br />This is what our audience, our customers as it were, expect from us.<br /><br />But, by having kernel, libdrm, xserver and mesa tied together, and the different parts of the driver stack spread over them, it is impossible to exchange 1 part of the graphics driver stack, or to exchange just the graphics driver stack, without changing the whole.<br /><br />By forcing our users to update all this infrastructure each, we will usually trigger a cascade of updates that reach far up the whole software stack, to the extent where trying to fix some small issue in the graphics driver, might mess up openOffice or another program that your average linux desktop user depends on.<br /><br />Also, what is the chance of getting both wireless, suspend/resume and your graphics driver working to an acceptable level at the same time? This becomes very very small, and when it does work, you better not run into issues somewhere else, as an update might ruin that very precarious balance.<br /><br /><b>Killing the desktop for everyone.</b><br /><br />No normal person can then run a free software desktop system, and expect to use it, because an arbitrary mix of hardware cannot possibly work together acceptably, at least not for a measurable amount of time.<br /><br />What will be left over is preloads and embedded system.<br /><br />Preloads is when some OEM, either itself, or through a linux distributor, spends many many man-years on making all parts work together properly. In the end, images will be produced which install on a very specific system and cannot be updated or maintained, except by a specialised team of people. Embedded systems basically work the same way: one combination of hardware, one image, no updates for average users except those provided by the manufacturer or their partners.<br /><br />So while people might buy a free software based system in a shop around the corner, and be somewhat happy with it for a while, normal desktop users will be left out in the cold.<br /><br />Looking further, by shutting out our own users, we will take away the breeding ground that free software is based on.<br /><br /><b>What solution is there?</b><br /><br />By now, that should be pretty obvious.<br /><br />Bring the different parts of the graphics driver stack together, and make its parts independent of the infrastructure they depend on.<br /><br />This allows driver developers to change internal structure and API at will, while at the same time providing the infrastructure compatibility that users, hardware and distribution vendors require.<br /><br />All it takes is a little care in designing infrastructure APIs, and a little care in keeping driver stacks compatible, even if that compatibility comes at the cost of disabling some features for some combinations of the infrastructure.<br /><br />This is not hard to do, and it is done in multiple places.<br /><br /><b>Why the Nvidia binary driver is that popular.</b><br /><br />In <a hreaf="http://www.phoronix.com/scan.php?page=article&amp;item=lgs_2010_xds&amp;num=2">a recent phoronix survey</a>, the amount of users using Nvidia hardware and drivers is larger than the users using any other combination.<br /><br /><br />This has a reason, and it has nothing to do with Nvidia being a completely closed source shop. Nvidia gives users the ability to install any graphics driver stack, and it should mostly be compatible with the environment it is installed in. This is simply what our users need.<br /><br />What is affected by Nvidia being binary only, is that Nvidia has to put in a lot of work on making things compatible. Free software drivers have a much much easier task, or at least they would, if they, and the infrastructure they depend on, was developed in a different fashion than is the case today.<br /><br /><b>An open proof of concept.</b><br /><br />My <a href="http://www.phoronix.com/scan.php?page=article&amp;item=fosdem_2010_unification&amp;num=1" rel="nofollow">talk at FOSDEM</a>, of course mentions my unichrome driver a lot, as it pretty much is my playground these days.<br /><br />Even though the featurelist of this driver is very limited, it is now integrating X, DRM and DRI drivers in one massively backwards compatible build-system, with autotools detecting all the API changes across all currently used versions of the necessary infrastructure. What one can see there is that, when some care is taking in structuring the driver, it is not that hard to achieve this: it basically just takes the will to do this.<br /><br />When I talked at FOSDEM, some people were stating that, while it might be possible for DRM and the Xserver, it would be totally impossible on Mesa/DRI, but for Mesa/gallium it should be easy.<br /><br />In the next month or so, I took all Mesa versions that were out in the wild, and split off the main libraries from the actual DRI drivers, created a set of headers as required by the drivers, created package config files, and then move the drivers out to their own git repositories. Basically, a DRI SDK was created, and the drivers were now building and running externally to this SDK. This across 3 years of DRI development.<br /><br />When I <a href="http://lists.freedesktop.org/archives/dri-devel/2010-April/000004.html" rel="nofollow">took that back to the Mesa community</a>, what I of course got was indifference, and, suddenly, claims that while this SDK might be possible for mesa/DRI it would definitely not be possible for Mesa/gallium!<br /><br /><b>The future?</b><br /><br />The proposed future direction for graphics drivers is to create graphics driver stacks. If not, we, the developers, might just as well stop working on free software graphics drivers altogether.<br /><br />And while the current situation currently is bad, it is not impossible to fix. The problems are known and clear, a path to the solution should by now also be clear, but the willingness to put in the bit of extra thought is simply lacking.<br /><br />So guys, if you really want to move into the wrong direction, please state the real reasons for doing so, state the consequences to your users; and know what the end result will be. urn:lj:livejournal.com:atom1:libv:22039 The DRI SDK and modular DRI drivers. 2010-03-17T00:29:22Z 2010-03-17T00:29:22Z Kinobe - Slip into something more comfortable. At <a href="http://www.fosdem.org/2010/" rel="nofollow">FOSDEM</a> I held a talk about <a href="http://www.fosdem.org/2010/schedule/events/xorg_stack" rel="nofollow">"The free software graphics driver stack"</a>, analyzing the structure and the distribution of the different parts of the graphics driver stack, and, based on clear requirements, proposing a re-organization to be able to bring the individual parts of the graphics driver stack together. <a href="http://people.freedesktop.org/~libv/graphics_driver_stack_(FOSDEM2010_-_slides).pdf" rel="nofollow">The slides for that talk are available here</a>, and <a href="http://video.fosdem.org/2010/devrooms/xorg/fosdem2010-xorg-luc.mp3" rel="nofollow">the audio</a> and <a href="http://video.fosdem.org/2010/devrooms/xorg/fosdem2010-xorg-luc.avi" rel="nofollow">video of this talk</a> have been made available too (Thanks <a href="http://www.phoronix.com/" rel="nofollow">Michael!</a>).<br /><br />Since I do not like to talk thin air, i also made <a href="http://cgit.freedesktop.org/~libv/xf86-video-unichrome/log/?h=unified" rel="nofollow">a fully unified unichrome graphics driver stack</a> available, proving that it is possible to unite all of Xorg, DRM and DRI drivers in one tree, and that it is possible to provide a reasonable amount of backwards compatibility (trivial for a stagnated driver) even in the Mesa/DRI driver.<br /><br />My slides have a TODO section, and the most significant of those TODOs was for Mesa. This listed providing shared libraries, pkgconfig files and the necessary headerfiles for Mesa, much like the SDK that X has had ever since xfree86 4.0.0. Of course, such a thing would not happen on its own, so a bit after FOSDEM I set off and now <a href="http://cgit.freedesktop.org/~libv/" rel="nofollow">I have implemented just that</a>.<br /><br />You can find <a href="http://cgit.freedesktop.org/~libv/mesa-dri-sdk/" rel="nofollow">the SDK enabled Mesa DRI tree here</a>, and the individual drivers can also be found in <a href="http://cgit.freedesktop.org/~libv/" rel="nofollow">my personal git repositories</a>. <a href="http://people.freedesktop.org/~libv/dri-sdk.txt" rel="nofollow">A README like document is available</a> too, to explain what one should do to build and install the SDK and to build and install the drivers.<br /><br />What this gets you, once the SDK is properly installed, is the ability to build and install just the DRI driver to be able to fix bugs, get some performance gains, or just provide the driver developers with good feedback (provided that someone updated the driver tree to the latest compatible changes that is ;)). This without having to replace your libGL, which is where all the dependencies come weighing in. Anyone who has had to update this so far, knows how painful such an update is, and that it is often easier to do a major distribution upgrade.<br /><br />So this brings us another step closer to make the free software desktop happen: the ability to easily update our graphics driver stack without touching anything else in our system! urn:lj:livejournal.com:atom1:libv:21942 VIA Technologies takes the blue pill. 2010-03-07T01:01:31Z 2010-03-07T01:01:31Z Queens Of The Stone Age - Queens Of The Stone Age - Regular John <a href="http://www.paranormal-entertainment.com/idr/blog/" rel="nofollow">@IanRomanick</a>, with <a href="http://www.paranormal-entertainment.com/idr/blog/posts/2010-03-06T23:33:19Z-Little_blue_GPU/" rel="nofollow">his post on a little cultural problem that VIA encountered</a>.<br /><br />I actually had some issues when unichrome.sf.net was first created in Q2 2004. Some commit emails would get caught by the sf.net spamfilter. There was an interesting struct in <a href="http://cgit.freedesktop.org/~libv/xf86-video-unichrome/tree/src/via_privioctl.h?id=cf134d7586" rel="nofollow">this headerfile</a>.<br /><br />A quick grep of VIAs current codedrops does not show much of an improvement. The same structs name is now preceded with "NEW". urn:lj:livejournal.com:atom1:libv:21700 Coreboot and Xorg DevRooms at FOSDEM... Aftermath. 2010-02-16T15:56:46Z 2010-02-16T15:56:46Z Frank Valli & the Four Seasons - Beggin' (Pilooski Re-Edit) FOSDEM this year was another huge success.<br /><br />We had a great room this year, with luxuries as windows, which massively helped to keep the temperature under control :)<br /><br />We had 100% coverage all the time, and during the first coreboot, and most of the announced Xorg talks, we had to limit the number of people getting in, as there was no standing space left (100 or so people in a 60 people devroom).<br /><br />Some people complained over the size of the room for a project like X.org. But given the amount of talks that X.org people offered to give this year, we should count ourselves lucky that we had a room at all. No matter how big a project thinks it is, if no-one steps up to talk, there is nothing that can be done.<br /><br />For those who got refused, don't worry. Now that Michael is back in the states, he will soon have the videos up on <a href="http://www.phoronix.com/" rel="nofollow">phoronix</a>.<br /><br />My talk about the graphics driver stack did seem to cause <a href="http://www.phoronix.com/forums/showpost.php?p=111565&amp;postcount=10" rel="nofollow">some crap-throwing</a>. Seems that I touched some nerves, strange how my usually very shrewd and correct insights always cause similar reactions in the same people.<br /><br />The FOSDEM event wouldn't be complete without being able to go out and enjoy some good meals with other open source people. This year was exceptionally nice, the Mirabelle had an exceptional Canard A L'Orange, and we found a really nice bar just a few streets away from the grand place, where beers are reasonable, the atmosphere is great and talking is actually possible!<br /><br />I would like to thank Egbert Eich, and whoever else who helped out to make this devroom run smoothly, and thanks also go to Michael for taping the talks and all the speakers for giving them and getting a mostly pleased and happy crowd in! And thanks to those visitors, who, at FOSDEM, are always very interested and very open minded. urn:lj:livejournal.com:atom1:libv:21335 Last Preparations for Coreboot/Xorg DevRooms at FOSDEM. 2010-02-04T09:20:13Z 2010-02-04T09:20:13Z The noise of a crappy high speed via mini-itx fan. In about 30h I will be on the ICE to Brussels to FOSDEM, to have 2 DevRooms there.<br /><br />The Sportsbag with kit has been packed. The meeting at the customary restaurant tomorrow evening has been pretty much set up. Both my talks should be ready, and just need some studying. And I am now making the posters with the schedules and the fliers (used for possible status changes and directions) for the two devrooms so that they can be printed out later today.<br /><br />The <a href="http://www.coreboot.org/FOSDEM_2010" rel="nofollow">schedule for the Coreboot DevRoom (on saturday)</a> is unchanged since my last post:<br /><br />* 13:00 : Peter Stuge - coreboot introduction.<br />* 14:00 : Peter Stuge - coreboot and PC technical details.<br />* 15:00 : Rudolf Marek - ACPI and Suspend/Resume under coreboot.<br />* 16:00 : Rudolf Marek - coreboot board porting.<br />* 17:00 : Carl-Daniel Hailfinger - Flashrom.<br />* 18:00 : Luc Verhaegen - Flash Enable BIOS Reverse Engineering.<br /><br />All seems well, and it seems that we will have everything filmed nicely too!<br /><br />The <a href="http://wiki.x.org/wiki/fosdem2010" rel="nofollow">schedule for the Xorg DevRoom (on sunday)</a> has seen one last minute change with the addition of Nicolai Haehnle's talk:<br /><br />* 12.00: Nicolai Hähnle : Towards GLSL in the r300 Gallium driver<br />* 13.00: Daniel Stone : Polishing X11 and making it shiny.<br />* 14.00: Luc Verhaegen : The free software desktop’s graphics driver stack.<br />* 15.00: Jerome Glisse : GPU Userspace - kernel interface & Radeon kernel modesetting status.<br />* 16.00: Mikhail Gusarov : X on e-Paper.<br /><br />This addition was very last minute, and did not make the FOSDEM folder. Nicolai was quite lucky as the FOSDEM organisers had already <a href="http://www.fosdem.org/2010/schedule/rooms/aw1.124" rel="nofollow">assigned the DevRoom to the openMoko project</a> in the morning, which luckily left a single slot free for Nicolai.<br /><br />I've finished my slides for my own talk in the X.org devRoom, and it promises to be an interesting one, albeit controversial for some.<br /><br />It examines some of what we can tell a few years down the road from the modular tree, and goes over the real needs. Then it explains the unification of graphics driver stacks, how to build it for the unichrome driver, and what infrastructural changes it could use.<br /><br />But before i go into details i will have a small in room survey (Which will probably become a more widespread survey online). One of the questions i will ask is: <i>"For those using, or those who have used, the nvidia or ati closed source drivers: how would you like it if nvidia or ATI told you that you needed the very latest upstream kernel, X, and mesa to be able to run the latest catalyst or nvidia driver, especially when you might need this version for its new hardware support or for bugfixes?"</i><br /><br />(and urgh... the msi fuzzy cn700t motherboard i was using as a mailserver just died on me, even the port80 card says that no bios code is being run. Now i stuck in another cn700 board, and it has one of those really noisy fans :() urn:lj:livejournal.com:atom1:libv:21100 So openSUSE needs a new community manager? 2010-01-27T18:26:20Z 2010-01-27T18:26:20Z So openSUSE needs a new community manager? Remember <a href="http://www.google.com/search?q=martin+lasarsch" rel="nofollow">this guy</a>?<br /><br />He knows a thing or two about SUSE and its community. He is used to travelling a lot, to give interviews, talking to the press, and to speaking in public, and on top, he knows how to organise events for a once big linux distribution. And thanks to a really fun group event at his previous company right after FOSDEM last year, he might actually be available for a new opportunity.<br /><br />Plus, he lives like 200 meters away from the SUSE Nuernberg office, how lucky is that?<br /><br />(oh, music is the stones, track 4 of 12x5, listed here as suse planet is horribly broken with respect to livejournal.) urn:lj:livejournal.com:atom1:libv:20873 FOSDEM: Coreboot and X.org Schedules. 2010-01-22T01:22:05Z 2010-01-22T01:22:05Z Fingathing - SuperHero Music - SuperHero Music <u><b><font size="3"><a href="http://www.coreboot.org/FOSDEM_2010" rel="nofollow">coreboot DevRoom</a></font></b></u><br /><br />This year, at FOSDEM, there is <a href="http://www.coreboot.org/FOSDEM_2010" rel="nofollow">a coreboot DevRoom</a>, and for the hardware-lover, there are some really interesting talks there. When putting together the final schedule, I actually had to spend some time on containing people's enthusiasm and limiting their ideas to something feasible for FOSDEM. A rather good sign, and i am sure that many people will enjoy the truly interesting and juicily in-depth talks there.<br /><br />Here is a quick copy of <a href="http://www.coreboot.org/FOSDEM_2010#Schedule" rel="nofollow">the saturday afternoon schedule</a>:<br /> * 13:00 : Peter Stuge - coreboot introduction.<br /> * 14:00 : Peter Stuge - coreboot and PC technical details.<br /> * 15:00 : Rudolf Marek - ACPI and Suspend/Resume under coreboot.<br /> * 16:00 : Rudolf Marek - coreboot board porting.<br /> * 17:00 : Carl-Daniel Hailfinger - Flashrom.<br /> * 18:00 : Luc Verhaegen - Flash Enable BIOS Reverse Engineering.<br /><br />So yeah, i have a talk at 18:00 which promises to be fun.<br /><br />Basically, for flashrom, we sometimes need to find out what write protection certain motherboards implement, so we can disable that from within flashrom. This usually means that someone (usually Michael Karcher or me) has to go and disassemble the BIOS to find out which GPIO line needs to be toggled.<br /><br />For a single board, this is usually not a hard task (and actually fun), but the overal amount of work in flashrom is sizable, and the two of us are not available 24/7 to do this for every user in need. So this talk is meant to alleviate that.<br /><br />I just wrote the slides for it, and while i might alter the notes and twiddle it a bit still in the next two weeks, it should be pretty much finished.<br /><br />The structure of the talk is the following:<br />* quick overview of how flashes are usually wired up on motherboards.<br />* quick overview of how to handle Award, ASUS, AMI and phoenix BIOSes.<br />* example of disassembly of a BIOS.<br /><br />The BIOS that is being disassembled is <a href="http://people.freedesktop.org/~libv/award_crafted.bin.bz2" rel="nofollow">an especially crafted one</a>, with an award style flash enable for a fictitious board. It's mostly filled with 0xFF, apart from the flash enable itself. And the talk itself is pretty accessible, you do not have to know much of assembly or hardware to be able to participate.<br /><br />If you are visiting the talk, and you want to join in, then you might already want to download <a href="http://people.freedesktop.org/~libv/award_crafted.bin.bz2" rel="nofollow">the image</a>, and make sure that you have hexdump and ndisasm installed (as there are about 4000 others trying to use the wireless at FOSDEM).<br /><br />It promises to be a blast!<br /><br /><u><b><font size="3"><a href="http://wiki.x.org/wiki/fosdem2010" rel="nofollow">X.org DevRoom</a></font></b></u><br /><br />On sunday there is of course the <a href="http://wiki.x.org/wiki/fosdem2010" rel="nofollow">X.org DevRoom</a>, for which the schedule is also online.<br /><br />It seems to be a tradition that I have to go and beg people to talk at truly great event like FOSDEM.<br /><br />Last year, we had 7 talks in the brochure, while there was room for 13, and we eventually had 9 talks. All these talks were hugely popular, with almost full capacity usage most of the time, but the actual talk coverage itself left much to be desired.<br /><br />This year I decided to split the usual two day X.org DevRoom into coreboot (saturday afternoon) and X.org (sunday), so that the X.org schedule could be tighter and more exciting.<br /><br />Here is <a href="http://wiki.x.org/wiki/fosdem2010#schedule" rel="nofollow">the result</a>;<br />* 10.00: ...<br />* 11.00: ...<br />* 12.00: ...<br />* 13.00: Daniel Stone : Polishing X11 and making it shiny.<br />* 14.00: Luc Verhaegen : The free software desktop’s graphics driver stack.<br />* 15.00: Jerome Glisse : GPU Userspace - kernel interface & Radeon kernel modesetting status.<br />* 16.00: Mikhail Gusarov : X on e-Paper.<br /><br />I, and a lot of fosdem visitors with me, thank Daniel, Jerome and Mikhail for their commitment. These talks seem very interesting and will no doubt draw in a lot of visitors and carry out the good name of X.org.<br /><br />It does make one think. There seems to be a direct and inverse relationship between the size and status of a free software project, and the willingness of its members to go and promote this project on the biggest free software event on that continent that holds the majority of its current developers and the largest potential of future contributors.<br /><br />In any case, I have been communicated that, as there are many very worthwhile and also very willing projects who get denied a DevRoom each year, a project with such a poor showing will not be given another DevRoom.<br /><br />While I start putting together the slides for my X.org talk, I can only concede. urn:lj:livejournal.com:atom1:libv:20707 Emergency BIOS update, the easy way. 2010-01-15T15:01:35Z 2010-01-15T15:01:35Z Bathtub-baritone singing Golden Brown. Jetway sells a cute VX800 mini-itx board which comes with a choice of VIA C7 and Nano cpu's: <a href="http://www.jetwaycomputer.com/NF76.html" rel="nofollow">the JNF76</a>. At the end of last year, i became the proud owner of one of the rare DVI modules for this board, and shortly thereafter there was an ebay auction which got me a brand new board at half the listed price. I now found the time to go and play with this board, to at least give the ebay seller the positive rating he deserves :)<br /><br />What I found is that this board booted a debian old-stable (etch) just fine, but that debian stable (lenny), opensuse 11.1 and 9.04 ubuntu, all locked up shortly after initialising the ide controller. Some <a href="http://osdir.com/ml/linux-kernel/2009-04/msg12014.html" rel="nofollow">digging around revealed</a> that this was an issue with the processor power state and recent kernels, and that this is only seen with an older bios version, and that version A03 and higher would fix this. Upon reboot, it was indeed confirmed that this was an old version, namely, A02.<br /><br />Since sadly, the network controller on this board is not supported by the old-stable kernel, i had to use a usb stick. So I stuffed <a href="http://www.jetwaycomputer.com/NF76.html" rel="nofollow">the latest bios</a> and <a href="http://www.flashrom.org/Downloads" rel="nofollow">a copy of flashrom master</a> on the stick, and stuck it in the jetway board. As i already had the libpci-dev package installed, a simple run of make got me a working flashrom binary. I then used flashrom -r jnf76.old.bios to dump the current bios, as a precaution. And then i ran flashrom -w on the new image.<br /><br />When running flashrom, there is a relatively high probability (50% is my current estimate, but will improve to about 30% soon) that it will not work. Motherboard makers like to provide an extra bit of protection for their bios images, and often use a gpio line from either the southbridge or the superio chip to set write protection on the flash chip.<br /><br />In flashrom, we keep a list of special board enables, which today mostly map which io line to set, and Michael Karcher and I spend quite a bit of our time digging out bioses for users to map this. So in this case, before i had tried to write to this chip, i had already done my 5 minute magic (<a href="http://www.coreboot.org/FOSDEM_2010" rel="nofollow">which i will explain at the coreboot devroom at FOSDEM</a>) and found out that there was no special protection present.<br /><br />Flashrom then erased the chip correctly, and then subsequently wrote and verified the image. And i was then able to reboot into any of the installed distributions painlessly.<br /><br />If there was protection present, (and we didn't handle it yet in flashrom) then this would've made the erase fail at the first byte that isn't 0xFF, and then nothing was lost. There is also another type of protection only found on LPC/FWH/Parallel chips, where the boot block can be locked, and this has the same effect, but only on the boot block part, where you just need to restore all other pages to be able to continue working. We will then happily help create the board enable with you on the flashrom mailing list, so that you still can get your bios updated correctly (now, and in future).<br /><br />In any case, this was one of the easiest emergency bios updates i had ever done. No messing around with dos images and what not. I had already identified a working linux kernel when trying to nail down the issue, and then just needed to run flashrom to live happily ever after :) urn:lj:livejournal.com:atom1:libv:20412 Coreboot and Xorg DevRooms at FOSDEM! 2009-12-02T03:20:19Z 2009-12-02T03:20:19Z Compuphonic & Kolombo - Antimatter (unreleased edit) Yeah, you read that correctly, there will be two close to the metal DevRooms at <a href="http://www.fosdem.org" rel="nofollow">FOSDEM</a> this year (or next year, Mr. Daenzer :))<br /><br />FOSDEM 2010 runs on the 6th and 7th of February and in <a href="http://archive.fosdem.org/2009/maps/campus" rel="nofollow">the AW building</a> in room AW.124 there will be some really interesting things happening :)<br /><br />On Saturday the 6th, the first <a href="http://www.coreboot.org" rel="nofollow">coreboot</a> DevRoom will be held there. It's still early days, but several key developers will be there, and we will be showing off coreboot, <a href="http://www.flashrom.org/" rel="nofollow">flashrom</a> and will be presenting some really cool stuff. You can track the talks list on <a href="http://www.coreboot.org/FOSDEM_2010" rel="nofollow">the coreboot wiki</a>.<br /><br />On Sunday, we will have the X.org DevRoom there, for the fifth time already. This DevRoom has really been one of the best filled and most interesting ones for the past few years, with highly in-depth and technical talks, and we of course will provide the same this year :) X.org tracking page is on <a href="http://wiki.x.org/wiki/fosdem2010" rel="nofollow">the X.org wiki.</a><br /><br />As per usual, we will try to get people together on friday evening and visit that one restaurant on the grand place that we tend to visit each year (whose name i can never remember :)). And if enough people give me a heads up, i will try to reserve a big table at the excellent Mirabelle again for saturday.<br /><br />As for people who are interested in speaking at one of the DevRooms, check the page of the individual projects for more details and do not hesitate to mail me or <a href="http://egbert-e.livejournal.com/" rel="nofollow">Egbert</a>.