Cross Collaboration Panel
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
DeMaio, Douglas (Moderation)
Gompa, Neal
Balla, Mariana
Et al.
Čermák, Dan
This open-source community panel will discuss cross collaboration between projects. It will offer a Q&A session.
 All right. So welcome back everyone. So we're going to have a cross collaboration panel and we might have a few people joining us and who might come a little late. But anyway, I want to introduce you to the panel that we have. So we have myself, Douglas, and I'm going to sort of moderate this. And then we have Adrian Schroeder. Adrian works on OBS. So Ben Cotton, Ben's the release manager for Fedora. We have Matthew Miller just coming in here. We also have Dan Sermak. I'm not sure if I'm saying that correct, Dan. Please feel free to. That depends on how you define that. But close enough. Yeah. And it looks like Neil Gampas joined us as well. So thank you. Thank you all for being a part of this panel. So if you wouldn't, actually, yeah, looks like Maria. Marie just joined us. Thank you. So Marie and I are kind of going to moderate this, but we're obviously going to have input as well. And so the format is kind of go, we're roughly going to be here for an hour. If we could go that long, we could go much further. But just to start off a few questions, could you each like tell me a little bit about yourself and what you work on? And we could start probably, we'll start with Ben. Hi. So I'm the Fedora program manager. I've been in this role for about three years. And basically my job is the chief cat herding officer. Basically managing Fedora release schedule and making sure we're actually sticking to it and gently encouraging people to come along and follow the processes. I've been a member of the Fedora community for over 10 years. And I sort of tangentially perspate in several other open source projects as well. I guess we'll go with Dan. Hi, I'm Dan. I'm a software developer at SUSE. I work in the developer engagement program that I've started out working on vagrant boxes, then build few Visual Studio Code extensions and sites that I also contribute to a few open source projects. I'm active in the open SUSE community as a package maintainer. And then I'm also quite active in the Fedora community. So I'm part of the I3 special interest group where we release the I3 spin of Fedora. I recently got elected in the FESCO and the Fedora engineering steering committee. So yeah, I have my feet also there. And Marie, if you want to like tell us a little bit about yourself and take over. My name is Marie Norden. I am Fedora's community action and impact coordinator. That is a community agreed upon name. It's very long. It means community manager, support person. So I help with a lot of the administrative type things behind Fedora. But I also work on community initiatives. And I work with a lot of the non coding teams, giving them guidance and support. And also work with mental projects. So I do a whole bunch of stuff with Fedora. I'm going to pick on Matt next. Okay, I'm Matthew Miller. I am the Fedora project leader. That means I throw cats at Ben. I think that's the main responsibility. In seriousness, Fedora is a community directed and community led project. So it means that as a leader, my job isn't to tell the community where to go, but to help the community find its collective direction and actually go there. Because I think the nature of communities is to go in every direction at once. And that's not always the most effective way to get somewhere. It's an interesting search technique, but not the most efficient one. So my job is to help us be more efficient. Really? Do I pick on the next person? Is that how we're doing this? Why not? Neil, go. All right, then. Lovely. So my name is Neil Gampa. I am someone who's all over open source stuff. Professionally, I am a senior DevOps engineer at Datto working on software delivery stuff and release engineering type things. But in the open source world, I am a long time member of both Fedora and OpenSUSA communities. I've been a member of VESCO for a year now. But also I am a member of the OpenSUSA board since January. And I do quite a lot in the KDE, SIG and Fedora, as well as the workstation working group, cloud working group, a number of other places. And I'm also in the OpenSUSA heroes team, helping build up infrastructure and setting up solutions to help support the OpenSUSA community to do great things. And you work so much with Adrian, you probably couldn't just pass it on to him. That's true. Hi, Adrian. Hi, Neil. I'm Adrian Schulter. I'm an OpenSUSA. I'm in first place responsible for the open build service. Yeah. And then someone gets new ideas how to build distributions like this with the jump concept. And I have to work on that. And apart from that, I take care about all the configurations from the non-SUSA distributions in the build service. So I try to help out people. And in the old times, I actually was a project manager in the OpenSUSA project when it started. And before that, I was doing the KDE desktop at SUSA. My daughter is messaging me that there's a large bug in the backyard and to come and save them. But I think this is one that she can probably handle herself. Sorry for the distraction. So it's not a paneled walking bug? Right. Well, so, you know, we kind of have some listed questions that we kind of came up with together. And I guess, you know, we've already talked a little bit about the background, but sort of one of the aspects of that was, you know, what sort of cross collaboration do you have within the project you're working with? Anyone wants to take that, feel free to just answer. So we certainly have some tooling level cross collaboration where a really big example is the OpenQA service that we use that came from SUSA. And it's really instrumental to Fedora's testing efforts at this point. And then we also have some key shared code like the library that DNF uses to actually do its work comes from SUSA. And then of course, we also have a lot of interested shared upstream projects which we work together. I think that's the highlight. We have Neil and Neil does all cross collaboration with all projects ever. And we've got Dan. So we've got actually some Venn diagram overlap of people as well, which is awesome. Yeah. And since you mentioned, since you mentioned lip solve, which is at the heart of lip DNF, if I understood it correctly nowadays, the micro opens with the micro s, which is using transactional updates that in turn is now using lip DNF. So it's interesting. Earlier this year, Richard Brown and I switched OpenSUSA micro as for desktops from using the weird shell wrapper thingy calling zipper to using a LibDNF plugin that implements the whole transactional updates mechanism so that package kit can use it. And so GNOME software can actually do upgrades this way, as well as Plasma Discover and any of those other things like that. We also have micro DNF now able to do transactional updates with Butterfast and things like that. And there's some further development down the pipeline to enable it for cockpit as well. And so there's some there's some fun stuff going on with that. And I'm hoping that we will have, you know, the DNF package manager with the transactional updates enablement actually used throughout micro os on all flavors desktop server, whatever, because it's actually worked out real nice and made for a much cleaner experience for integrating in the stack. Yeah, I think we also got some, we have some decent collaboration in the Rust ecosystem. As far as I know, the Fedora Rust special interest group was leveraging the open build service quite a lot. I guess Adrian flinches now because I think it cost quite some strain on on the build service in a few at a few times. But there's been some some call up in there as far as I mean, where I think they've also boxed some resources there, not read and taking endless but yeah, that's got salt. It gave Michael some heartburn when when we when we started doing the dynamic build dependencies and OBS because the processes will get hung. It will just kind of stay there. Keeping the VMs up even after the builds were done. Because every time the every time you ran the build process, it starts up a brand new virtual machine and runs the whole build process inside of there. But because it didn't know that the dynamic build dependencies cycle was complete, it keep the VM running, doing nothing. So I have a question about this at a higher level. Rust is an example, but basically, every every language, especially every new language has its own package manager, its own way of doing things. And as Fedora, we've struggled with what the best way to provide a good experience for users, both developers who are using those languages and then consumers of those developers work, like in our in our world. And our traditional answer has been, you know, to cram all of that through RPM with increasing automation, which is definitely cramming it all through RPM with one with automation is definitely one approach. There might be other approaches to do that. So I guess on the rust thing in specific, I think people see little value, it's hard for us to convince the world of the value of putting rust crates into RPM format. Arguably, there's value. But it's a it's an argument that we have to continually make if we want to say that. I think that there is almost zero value in both the Fedora community doing it and the open SUSE community doing it. So how could we make it so that it's just one community doing that packaging? We would have to sit down and essentially come up with a way how we want to do that. But there's also there's also a limit to how far we should go since we can't package the world anymore, at least with Rust, with Node.js. These these ecosystems, they have really small standard libraries, and they have huge dependency stacks. And at a certain point, you really come down to think, does packaging everything as RPMs, does that really provide additional value? It does if you can, if you can, for instance, so what there's been some recent development for the in the build service to provide to provide vending of the of the dependencies where you can replace certain parts of them with with RPMs that you ship and certain parts that you take from upstream. And there you could say, okay, we have these core libraries where we provide support as a distribution or as an enterprise. And then and the rest you can just grab from from npm or from from crates IO. But yeah, okay, three different problems from my point of view. I mean, you're right that I mean, the build part is one problem. And we could automate it or do it in RPMs or not RPMs. This is one discussion. But the other problem is if you do it button up or not in a unified format, then you would need to deal with the cost dependency between the RPM world and Ruby gems and Python pit and so on. So if you update my my SQL, it has an effect on all the other formats. And then you need to transfer this logic into into the update tooling. So DNF, super and so on would need to learn about it. I mean, I think there's no wrong or right answer here. I think we can should work in all directions and evaluate. We can't know can't limit people to use other formats. We can't force people to RPMs. But on the other side, there is the additional value I completely agree. But what I think comes to the origin of the question, when working together with packages, no matter if it's past or whatever, I think there is really the big amount of our packages we could really maintain together. And I think our tooling is is blocking us there at the moment because build services so much different than Koji and whatever copper whatever else you have. And that's definitely an area where we should. I mean, we don't need to consolidate the build tooling, but we should maybe consolidate how we maintain sources. So yeah, and I think what's also blocking us is convention. So we have really this really different conventions between open SUSE and Fedora in certain regards. So in Rust, it's relatively comparable since they have essentially the same heritage. But if you take a look at if you take a look at Python packaging, it works it works kind of differently. If you take a look at Koji, I think that's completely different. If you take a look at Ruby gems, it's also quite different. Although open SUSE uses a fork version of gem to RPM. So I think to really to really come to share our sources together, we'd have to all sit together at one table and decide we are going to do it now this way and reduce the reduced differences that we have. So we did this once before, about a decade ago, where we came together and reconciliated the way that our structures and guidelines in our packages were I think was 2011 or something like that, where basically open SUSE and Fedora communities after like, it was a desktop summit or something like that, where the guidelines were reforked from Fedora to start over the guidelines in open SUSE. And things were built off from there. What I think failed after that effort was there was no ongoing communication between the two projects to solve problems together. So like something that when the Fedora Rust was being assembled that Igor, Rates and I did together was we made sure that the stakeholders from the different distributions that started the project together stayed in contact throughout the time that the stuff evolved. And that's something that we probably should be doing if we want to like make that successful this time around. From a tooling perspective, I agree with Adrienne that like we don't necessarily need to be like, all right, Koji, OBS, Smush, done. That's probably one not going to happen and two very, very difficult. But we can do things like there's experiments going on with pulling the sources out of OBS's version control system into a diskit style system with code.openSUSE.org, which is a Pagger instance. Like we can start and the OBS team has actually started working on SCM integration with Forges and they're working on Pagger integration that could actually be used in this manner. So we could start leveraging some of the same conventions and blending a little bit of best practices to try to make this better and easier to share between the two projects. One place that would make sense to me is the source git thing that some people in Fedora and Red Hat have been working on, which is basically maintain the spec file stuff in an upstream repository rather than in diskit, an actual fork of the upstream source and then use automation to go to diskit and then to build from there. It seems like maybe it could be made so the same source git tree could then go to SUSE's git and build system from there, but using the same original source, that would be a huge win. Yeah, I just want to correct a bit the things about continuous integration support at the moment. This is not, I mean, this has a different purpose, but we are working also on an alternative asset stores. So the entire project even can be replaced with the git, so that an external git can be or other protocols can be used. So that would be the vehicle in our case. And yes, then we could exactly achieve this and it doesn't matter if it's git or if we agree on anything else. It's more important, I think, that we do the sources, the source hosting together, and what's especially important, I think, is that we agree how do we validate which sources we can trust? How do we trust upstream projects and so on, because I see there's the biggest value in our aggregation. The build is, yeah, that's also something we need to deal with, but the really, really important and valuable work we do is collecting the trust. Actually, that brings to mind another thing, which is license identification in packages. I have no idea what you folks do in SUSE with that. They're SPDX. Okay, which is excellent, because we are actually working on a project to possibly move Fedora to SPDX for our license tagging. There's a lot of work to go there, but that means that we can audit everything. Yeah, I think that would help collaboration there as well. Yeah, and especially we could also leverage projects like Cable, which is a license, that's a license checker module that runs in the open build service. It essentially scans all the sources that you throw into a package and tries to find out all kinds of licenses. It's pretty good. It finds all kinds of interesting things. About a year ago, I've packaged our studio for Open SUSE, and then through that it came into Fedora. It found a lot of licenses that upstream bundled and wasn't aware of. It also found a bunch of stuff that upstream still thought they had bundled and didn't anymore for five years. What is that tool called? Cable. C-A-V-I-L. I think it's known as license sticker. Oh yeah, license sticker was the old name. So something that might be interesting would be if we could get Cable deployed in Fedora infrastructure and then wire up a bot to do it on Pagger PRs. We could also replace the thing that's done in Fedora review with using Cable and see, start using it that way. We could also share the license corpus that we use. We actually discussed this in Tristan a little bit, Matthew. Yeah. So Red Hat has a tool called PELC, which is used internally, P-E-L-C, that is part of the REL process. It's an internal tool, not a public tool. We've had some talks about open sourcing this. I feel like SUSE does this and we could collaborate. Might be something we could move towards. I saw Neil's face. There's a tool that exists that I've never heard of before. Yes. What language is it written in out of curiosity? It's been around for a long time. So my guess is Python, but there's a small chance of it being Pearl, right? I don't actually know. I haven't actually looked. Cable is Pearl. We could merge all the pearls. Excellent. Yeah. If merge all the pearls is one thing, but then if it's Python, then we'll have to have a shootout and figure out which code base is less awful to work with. The one that's not in Pearl is the one that's less awful to work with. Come on. And I'm not even wearing my Pearl shirt today, but I have one that I wear all the time. I love Pearl. Matthew, I've tried to be diplomatic here. That's not too bad. It's really hard to transition though, because we have a lot of other things we want to hit on. Sorry. I'm going to go to Ben here. I'm going to read this because I couldn't break it up so easily. The messages really collaborating amongst the projects really resonates. I think we see it as we talk about it. Ben, do you have any opinion about why that is? You just started out with Ben. Ben, yes. Ben said he would do it. So you asked if I have an opinion, which is a funny question to ask. Think about my own experience. I got started contributing to Fedora because I had been using it and here's the thing that I'm getting for free and people are putting effort into doing. And I felt sort of an ethical obligation to give back to the community. And I think that's true to some degree for a lot of people who participate in open source projects. And so the whole idea is that you're putting your code or your documentation or your design or whatever out there for everyone else to also use. And so it's just sort of inherently collaborative. And because there's not these strict boundaries of, yeah, we have Fedora and OpenSusa and Ubuntu and Debian and all these other distributions that have some differences, but we're all using the Linux kernel. We're all using the same basic user land tools. So there's not this strict thing where I'm working in this one community and I'm only going to work here because nothing touches. As soon as you start working on anything that goes into a distribution, you're almost certainly impacting other distributions and then it starts to make sense to work together. So Dan, actually, I think you said you would probably also... Yeah, I mean, I'm afraid Ben already said most of the important parts. Since if you do open source, at least in my opinion, open source or more or less, for me, open source is to a good extent about the community, about the people. It's not all... It's also about the code and that you can modify it, but it's also mostly about the people with whom you work with. And then it's about collaboration and it's kind of wired into the heart of the whole thing. I mean, you can put it out there into the public so that people can reuse it. And I mean, the whole thing started like that. So it's kind of baked into its genes. Okay. So the benefits... I mean, you could see the benefits with collaborating. Do any of you have any specific examples that you could think of? So I guess the earliest example really for me that I think really demonstrates this quite well is when I started working on bringing up support for soft float arm architecture, so the really crappy useless arm architectures for DNF and RPM. So that involved going to, at the DNF level, adding in the markers to identify to that, but also teaching Lipsul how to do it and going through to that, which meant touching both a Red Hat project and an OpenSusa project, and then going to OpenBuild service so that it would learn how to actually sort that and be able to select the correct builders for that as well and those sorts of things. Like when I was doing that architecture bring up for a professional project, thankfully nobody will ever see the light of day of it because it was a terrible thing and I don't ever really want to do that again. But because it was something that involved some, you know, different distributions, different projects, I even found, you know, there was a Linux distribution that was actually shipping RPM based distribution with DNF and all this stuff, you know, for ARM v5 and they were like, yeah, you know, if you can help us out with, you know, getting this all fully working with this new stack, that'd be great. And it's like, all right, sure. And they tested some of my patches and made sure that it worked. And, you know, so like with a little bit of Magia's help and, you know, Michael Schroeder's help in Libsol and Adrian in OpenBuild service and Yaroslav Ronsek, sorry, I screwed up that name, I'm sure I messed that name up. In the DNF team, it went fairly well and I was able to easily like work with a whole bunch of people, a whole bunch of random people to get something that was both simple and complicated at the same time actually done and usable. Ed, no, that's still a terrible architecture. Right. I think another good example is also OpenQA, which was also previously mentioned. I mean, OpenQA is relied on by, I mean, that's the thing that makes OpenSusertumble wheat roll and be actually pretty stable for a rolling release distribution that releases up to five times a week. If you compare that to other rolling release distros, which break, break, break and break. We won't name names. No, we won't. But so it's, and a good chunk of that is achieved by OpenQA. And then Fedora using it, you see, especially Aedam, contributing to OpenQA, also upstream and fixing stuff that's sort of is used by Fedora. And so I think that's one of the, that's a pretty good example. And Adam also added something unique to the OpenQA ecosystem that I found really handy. He wrote a Python client for OpenQA. So I didn't have to write a URL code, it's your face with it. You're not going to get away from that fight. We're going to come back to it. I figured I just pointed out, like, Adam added something that actually made it much more usable for a broader set of people. And that helped me start like, you know, poking at it and testing, like whenever I'm working with OpenQA stuff, like either in OpenSusert or Fedora, I use the Python OpenQA client so that I can, you know, automatically go find something, fetch it and take a look at stuff because, you know, there's a lot of stuff in OpenQA. And navigating by hand is very, very annoying. And if you want to, and if we want to go down this Python versus Pearl route, OpenQA is now also adding a Python API. So in case you don't like Pearl at all and don't want to use the test API, you will be able to use the Python-based test API soon. But it looks exactly the same. I can expect Adam to port everything over to Python. I expect that will happen. It's possible, but it looks the same. So sure. But it's the thought that counts. And then we'll rewrite it in Rust. Okay, you're banned. It works. But if you have a common goal, those have a different one, collaboration works mostly automatically. Most places. So it's the part that you find to come and go and to come and benefit. Yeah. That's I think been the hardest part in general was just finding a way to bring everyone to realize that there is a common goal because a lot of times, you know, people don't realize that, you know, they may be framing the problem differently in their head or they're thinking of it in different terms, but it's still like the same kind of thing and everyone's still and sharing the workload, you know, many hands make light work is, you know, an old phrase that is still true even today. Like collaborating makes it a lot easier for us to do bigger and bigger things together better. Is it in doing the questions, we had one that a lot of you actually responded to that you would want to answer and one was kind of opposite of what we're talking about, but collaboration can sometimes feel like a drag on innovation because of the required effort to communicate and synchronize with many different people. So how do you overcome this feeling? Oh man. That one, that one seems like it speaks to me in particular. The the hard part really is trying to, the hard part about making the collaboration work is trying to not necessarily make it someone else's problem because that's a lot of what sometimes that's what a lot of people kind of default to, but trying to find a way to have everyone see the benefit of solving the problem. And the reason I find it important to do that, even though that it's pretty grungy and and unrewarding work most of the time, is because the end result is I get more people that can help me solve the problem better because in a lot of cases, if I'm doing the thing by myself, I'm not a 10x engineer. I do not, I do not have the capability to make magic out of nothing. I am not that good. And and the only way I'm even halfway successful on anything that I'm doing is because I can get help from others who are willing to chip in and and that make it successful. That's the real 10x engineer right there through one who can convince 10x engineers to work with them, right? I still feel like a failure after drag. I mean so it may sound a little self-serving, but I think having somebody, you know, in that sort of act as like a project or program manager role who their contribution is just managing some of the collaboration and the communication really helps. You know Fedora didn't have somebody in my role originally and it was sort of a crazy disaster. Now it's just like slightly less crazy disaster, but I think, you know, that sort of the same principles apply when you get into, you know, smaller even sort of more ad hoc collaboration is if there's not somebody sort of keeping the cats all going in the same direction, you then do have to communicate with each person individually one-on-one and that becomes an unmanageable overhead. I'm going to echo that for the community based work as well. People are hard. Oh come on Adrian. I think you kind of have the hardest job of all. You have to bake all the things work for everyone. No, no, no, no, I have to excuse that it's all content, all problems are only the content. I'm never going to. I don't know, I distinctly remember a couple of times where I have to go in and ask you like, hey, this thing is not working because stuff's missing. What do? My daughter has solved her insect problem by deciding to go to the mall with her friends. That's an interesting solution. Honestly, I'm proud. Who among us has never said bugs are hard let's go shopping though. Fair. Yeah, but I think what really helps in terms of collaboration, if it feels like a drag is to essentially consider if I push this stuff upstream, yes, it's going to be more work in the short term, but in the long term, I get more eyes. I get more people to look at that. So in the long term solution is very likely going to be better. So that really, that really at least helps me because I know, I mean, sometimes you submit a pull request somewhere and then the discussion starts and it goes around in circles and weeks later, it's still not merged. And at some point you get annoyed and you think, oh, I could just for this thing and be done with it. And then the project lives for 10 years and you think, oh, damn it, I should have been maybe a bit more resilient and actually go through with this pull request. So I think the, I think in the long term, usually collaborating tends out to work, to work out positively for everyone. I have a response to this for myself personally, overcoming that feeling, it's a lot of adaptation and not being so like, don't hold on to your idea of what the outcome of this is going to be, what it evolves and kind of accept that and release it. How I go about it, at least with the non coding teams within Fedora working amongst themselves. I'd probably say coding, programming, that's the easy part. That's the part that honestly does not matter very much most of the time. The actually getting people to sit in a room together and not throw pitchforks at each other, that is the hard part. That is sitting in the room. Oh my goodness. Yes. Like, that's those are the things that really strain my desire to do this kind of stuff because it's really difficult sometimes when you have really unfortunate personalities is the way I'm going to describe this really unfortunate personalities in the room. And whether it's code, whether it's documentation, whether it's design, whether it is advocacy or any of those things, it really doesn't matter what space it is. Like juggling personalities and trying to get people to mesh together is really the hard part because when that falls apart, that is what leads to forks. That's what leads to split efforts. That's what leads to project failures and things like that. And notes you said for both forks and pitchforks. One of the things that's I mean, some of its personalities and some of it is people who maybe need to learn to work better together in collaboration. But a lot of it is this is very personal for so many of us. And it's very hard to not take the smallest things very personally, very quickly, especially when it is something where you put in 100 hours a week, both your work time and your free time and whatever it's been your life and your passion. And it's very easy for somebody's comment that they didn't mean a certain way that sounds dismissive to set you off or for even things that started as technical to become very just adversarial. And so it is a hard thing to deal with. I'm just going to add that codes of conduct make collaboration easier. It's definitely something we implement in the individual community to make things easier for folks to work together. And it's definitely helped. I don't know if codes of conduct in themselves help, but having a process to make that code of conduct work is I think probably more important because I have been involved in a ton of projects that have them, not naming any particular names, but they have code of conducts and things like that. But with no mechanism to make them real, to make them enforceable, they're just words on a digital paper. They don't mean anything. It's quite one thing to just say, we're going to copy the contributor covenant or insert your favorite template of a code of conduct here. It's quite another to say, I'm going to back this up with action when somebody actually like point something out and says, if this is a problem. And in a lot of open source projects and a lot of communities, sometimes you just have one without the other. And that's just, it doesn't work. It's not enough. You have to mean what you say and say what you mean. Can we let in the middle a little bit more? And we'll talk about like, so, you know, with collaboration, what do you all think could be improved? I mean, I think we did on a few things already, for sure. But other things that come to mind. I think that more that we have is our distributions are so large and we have so many individual pieces. And in some cases, it won't be able to collaborate or the additional time effort to agree on something would be way too much that it's inverse, basically. But you have so many packages that there's definitely some stacks where we could aim for that we would be, should be able to work together. I don't know which one to pick because but we, let's say, I don't know something. Just if you want, if you want to this example. Where we have the same pine, same pain and where we should. Yeah. Yeah. I looked at a little Rust project I have that all it does is it makes a little world with trees in it that you can move a little figure around this. It has 236 dependencies. So am I ever going to package that up in the traditional manner? Not if some unless somebody else agrees to package at least 234 of those dependencies. I have written an extension for VS code and including development dependencies. It's over 1000 node models. Yeah. And I have been conservative. So I'm not packaging those. No JS makes me want to replace this water bottle with something way stronger. Yeah, but where I definitely see where we could collaborate where we definitely I think can collaborate and should collaborate most for instance not only the packaging stuff but also image building since that's that's something where essentially every distribution has its own tooling. So in the open build service and for the open source of distribution mostly use Kiwi. Fedora has I think Lorax is the tool. There's nine different build tools. Okay, I wasn't aware that it was so many but essentially Kiwi is already a pretty is a pretty powerful tool and it would be great if we could just if we could also work work together on something there since Kiwi can build Fedora images for instance it can build a Bantu and yeah Neil is Neil is doing his thing since he brought in a good now we have 10 different ways of doing it is what you're saying. I will aggressive yeah I'm not saying we should make a tenth one. That will aggressively rip out all the other ones if we replace it with Kiwi because I am so tired of fixing all of them every time I make a change to them. It's a massive time sink and it sucks and everybody should feel ashamed of how many times I have that this wheel has been spun over and over again with square wheels that are chipped at the edges with sharp spikes on the sides. I think another way we can improve the collaboration is you know we think a lot of our collaboration tends to focus on very technical things whether it's you know packages or how we're building process. I don't know if we necessarily doing a very good job of collaborating on how we build our communities but you know it's really interesting you know I was right before the world shut down last year I was at the scale conference and I spent about an hour in the OpenSUSA booth talking to an ambassador there and every problem that they said they had like that's very familiar and then you know the the OpenSUSA community survey and it was back over the winter maybe you know a lot of the problems I was like oh these are the exact same problems that we face in Fedora and I think it'd be great if we had a way to work together on those things too right because like we said the code is not always the the hard part and I think there's a lot of benefit you know culturally it feels like our projects are very similar we have a lot of shared community members so I think we you know collaborating on the the community parts is a big area that we haven't really tapped yet. I think an unintentional positive of pandemic COVID times has been a nice chance for our communities to engage at a higher level at these virtual events and you know at Nest and the release party we've had you know different organizations come and hang out with Fedora you know and this cross-collaboration panel is a great example of like something that maybe wouldn't necessarily happen just this way if COVID hadn't happened we hadn't put more time and energy into virtual events and I think it's given a different swath of people a chance to connect with communities that are tangential so we've seen more engagement and more folks at our events and I'm guessing that that is part of it it's definitely not intentional it's an observation. Yeah but I like to ask again if we have a shared community somewhere then they need to have a shared goal because then they can talk about the same thing but we would get this if we share some components if we would have for example the same GNOME desktop then of course if you speak about GNOME then it doesn't matter if it's Fedora or Fedora so it would be then the yeah I don't know then who would maybe even need to come name something I don't know. So is Fedora? Fedora is working on a community outreach revamp and when that is all set and done and has a bow on it I'd love to share that with other communities so they can adapt it. We're planning a resource library you know it's gonna it's happening. Yeah like this is something that I kind of ran on in my platform to become a board member was I personally have felt that the OpenSusa community lacked a particular I want to say empowerment to feel like they could communicate and collaborate and communicate and and execute on things in a way where they felt like they could be seen like something that was a problem that I had observed like listening in the OpenSusa Discord server in the matrix rooms talking to folks like pretty much everywhere outside of the OpenSusa Factory IRC channel. The common issue was you know I'm trying to figure out how to do something but I can't figure out how to figure out how to do something and when I even if I could figure out how to figure out how to do something I don't know how to make people know that I did this thing and like maybe look at it or see if that could be made better and things like that and I feel like this is a problem that spoke to me so very hard and and it's something that I'm hoping you know that you know with this closer ties between Fedora and OpenSusa we can learn a little bit from each other about how to do that sort of thing so that our community members can feel empowered to drive change excellence and growth in in both projects like both Fedora and OpenSusa deserve to be able to be successful and and make you know the the people that use these projects to live to work to play to be happy with it. Sounds good. Such a statement Matthew. But you don't just think that this kind of a problem is because we have too many communication channels and they are just too little. So at least with with everything but IRC and the mailing lists they're all bridged together and unified so when someone talks from one avenue the other people every one of the other avenues actually can see each other and respond to each other so it feels a lot more connected than it would like say for example if you go into Fedora's matrix discord or telegram rooms they're a lot more fragmented because there wasn't an intentional approach of connecting them all up front whereas when we were introducing the real-time chats across the different platforms in OpenSusa a distinct goal was making sure they were all connected to each other it's generally not allowed to introduce a communications platform that is not connected to the other ones and so if it's a real-time chat platform where users and developers and other folks would be you know communicating with each other there needs to be a way to connect them all so that nobody is missing from the conversation and that was something that I think alleviates a lot of that problem at least on the OpenSusa side the main actual problem has been that we have not been able to connect IRC to all the rest of them because of a whole bunch of historical mess with how the IRC channels were managed with the unfortunate situation around IRC that has happened it has been an opportunity to actually fix that too so like one of the things that I'm hoping we can actually resolve soon is bringing everybody you know into the onto the same page so that you know someone who unfortunately is on IRC can talk to everyone who's on Discord Matrix Telegram and whatnot and and be able to to do that sort of thing um but the second part of it is not just the real-time chats but like people don't know how to get started to do something in a lot of cases and I don't know the answer to that problem I don't know how to solve how to help people like get started with their thing other than case by case things and that doesn't scale so Neil with kind of sort of sort of solve this problem a little bit Fedora with the join-sig it's a like a group of people it's a chat room and a repo that's a very like low commitment effort and it's simply a place to send newcomers and the folks that hang out in that chat room help them open a ticket like a little intro ticket trying to like get some information out of them and they connect them with the group that makes sense so it's a big problem we're solving it with people and it's working pretty well okay cool something to look into then I already noted too um so it looks like we're sort of hitting that's a top of the hour and I think we'll probably finish it up is there any last words that anyone would like to say famous last words about that I think Neil's uh Salil a quick from a few minutes ago was an excellent ending point so if you go back to edit this video just stop it there chop it and that that I think that says I'm gonna do a shame most love uh everyone's invited to nest 2021 and open season will be there so should be a great time and we'll give a great talk about obs stuff and how they're supporting the door building fedora on it I mean uh should we do this panel at nest I think it'd be fun it'd be a different it'd be a second audience call call for participation still still open right now it is open until July 16 get your proposal said sure but yeah like I I'm really happy that we did this this is this is really nice and it lets us show like the perspectives from both of our communities and hey adrian is here which like I have I barely ever see adrian on anything and that just excites me in a whole different level I know our time zones do not align at all all right well I want to thank you all and enjoy the rest of the conference and thank you for being here thank you thank you thank you thank you thanks for having us thanks bye
 All right. So welcome back everyone. So we're going to have a cross collaboration panel  and we might have a few people joining us and who might come a little late. But anyway,  I want to introduce you to the panel that we have. So we have myself, Douglas,  and I'm going to sort of moderate this. And then we have Adrian Schroeder.  Adrian works on OBS. So Ben Cotton, Ben's the release manager for Fedora.  We have Matthew Miller just coming in here. We also have Dan Sermak. I'm not sure if I'm  saying that correct, Dan. Please feel free to. That depends on how you define that. But close enough.  Yeah. And it looks like Neil Gampas joined us as well. So thank you. Thank you all for being a part  of this panel. So if you wouldn't, actually, yeah, looks like Maria. Marie just joined us. Thank you.  So Marie and I are kind of going to moderate this, but we're obviously going to have input as well.  And so the format is kind of go, we're roughly going to be here for an hour. If we could go that  long, we could go much further. But just to start off a few questions, could you each like tell me a  little bit about yourself and what you work on? And we could start probably, we'll start with Ben.  Hi. So I'm the Fedora program manager. I've been in this role for about three years. And basically  my job is the chief cat herding officer. Basically managing Fedora release schedule and making sure  we're actually sticking to it and gently encouraging people to come along and follow the processes.  I've been a member of the Fedora community for over 10 years. And I sort of tangentially  perspate in several other open source projects as well. I guess we'll go with Dan. Hi, I'm Dan.  I'm a software developer at SUSE. I work in the developer engagement program that I've started  out working on vagrant boxes, then build few Visual Studio Code extensions and sites that I  also contribute to a few open source projects. I'm active in the open SUSE community as a package  maintainer. And then I'm also quite active in the Fedora community. So I'm part of the I3  special interest group where we release the I3 spin of Fedora. I recently got elected in  the FESCO and the Fedora engineering steering committee. So yeah, I have my feet also there.  And Marie, if you want to like tell us a little bit about yourself and take over.  My name is Marie Norden. I am Fedora's community action and impact coordinator. That is a community  agreed upon name. It's very long. It means community manager, support person. So I help with a lot of  the administrative type things behind Fedora. But I also work on community initiatives. And I work  with a lot of the non coding teams, giving them guidance and support. And also work with  mental projects. So I do a whole bunch of stuff with Fedora. I'm going to pick on Matt next.  Okay, I'm Matthew Miller. I am the Fedora project leader. That means I throw cats at Ben. I think  that's the main responsibility. In seriousness, Fedora is a community directed and community led  project. So it means that as a leader, my job isn't to tell the community where to go, but to help the  community find its collective direction and actually go there. Because I think the nature of  communities is to go in every direction at once. And that's not always the most effective way to get  somewhere. It's an interesting search technique, but not the most efficient one. So my job is to help  us be more efficient. Really? Do I pick on the next person? Is that how we're doing this? Why not?  Neil, go. All right, then. Lovely. So my name is Neil Gampa. I am someone who's all over open  source stuff. Professionally, I am a senior DevOps engineer at Datto working on software delivery  stuff and release engineering type things. But in the open source world, I am a long time member of  both Fedora and OpenSUSA communities. I've been a member of VESCO for a year now. But also I am a  member of the OpenSUSA board since January. And I do quite a lot in the KDE, SIG and Fedora, as well  as the workstation working group, cloud working group, a number of other places. And I'm also in  the OpenSUSA heroes team, helping build up infrastructure and setting up solutions to help  support the OpenSUSA community to do great things. And you work so much with Adrian, you probably  couldn't just pass it on to him. That's true. Hi, Adrian. Hi, Neil. I'm Adrian Schulter. I'm an  OpenSUSA. I'm in first place responsible for the open build service. Yeah. And then someone gets new  ideas how to build distributions like this with the jump concept. And I have to work on that. And  apart from that, I take care about all the configurations from the non-SUSA distributions in  the build service. So I try to help out people. And in the old times, I actually was a project  manager in the OpenSUSA project when it started. And before that, I was doing the KDE desktop at  SUSA. My daughter is messaging me that there's a large bug in the backyard and to come and save  them. But I think this is one that she can probably handle herself. Sorry for the distraction.  So it's not a paneled walking bug? Right. Well, so, you know, we kind of have some  listed questions that we kind of came up with together. And I guess, you know, we've already  talked a little bit about the background, but sort of one of the aspects of that was, you know,  what sort of cross collaboration do you have within the project you're working with? Anyone  wants to take that, feel free to just answer. So we certainly have some tooling level cross  collaboration where a really big example is the OpenQA service that we use that came from SUSA.  And it's really instrumental to Fedora's testing efforts at this point. And then we also have  some key shared code like the library that DNF uses to actually do its work comes from SUSA.  And then of course, we also have a lot of interested shared upstream projects which we work  together. I think that's the highlight. We have Neil and Neil does all cross collaboration with  all projects ever. And we've got Dan. So we've got actually some Venn diagram overlap of people as  well, which is awesome. Yeah. And since you mentioned, since you mentioned lip solve,  which is at the heart of lip DNF, if I understood it correctly nowadays, the micro opens with the  micro s, which is using transactional updates that in turn is now using lip DNF. So it's interesting.  Earlier this year, Richard Brown and I switched OpenSUSA micro as for desktops from using the  weird shell wrapper thingy calling zipper to using a LibDNF plugin that implements the whole  transactional updates mechanism so that package kit can use it. And so GNOME software can actually  do upgrades this way, as well as Plasma Discover and any of those other things like that. We also  have micro DNF now able to do transactional updates with Butterfast and things like that. And  there's some further development down the pipeline to enable it for cockpit as well. And so there's  some there's some fun stuff going on with that. And I'm hoping that we will have, you know,  the DNF package manager with the transactional updates enablement actually used throughout  micro os on all flavors desktop server, whatever, because it's actually worked out real nice and  made for a much cleaner experience for integrating in the stack. Yeah, I think we also got some,  we have some decent collaboration in the Rust ecosystem. As far as I know, the Fedora Rust  special interest group was leveraging the open build service quite a lot. I guess Adrian flinches  now because I think it cost quite some strain on on the build service in a few at a few times.  But there's been some some call up in there as far as I mean, where I think they've also boxed  some resources there, not read and taking endless but yeah, that's got salt. It gave Michael some  heartburn when when we when we started doing the dynamic build dependencies and OBS because the  processes will get hung. It will just kind of stay there. Keeping the VMs up even after the  builds were done. Because every time the every time you ran the build process, it starts up a  brand new virtual machine and runs the whole build process inside of there. But because it didn't  know that the dynamic build dependencies cycle was complete, it keep the VM running, doing nothing.  So I have a question about this at a higher level. Rust is an example, but basically,  every every language, especially every new language has its own package manager, its own way of  doing things. And as Fedora, we've struggled with what the best way to provide a good experience  for users, both developers who are using those languages and then consumers of those developers  work, like in our in our world. And our traditional answer has been, you know, to cram all of that  through RPM with increasing automation, which is definitely cramming it all through RPM with one  with automation is definitely one approach. There might be other approaches to do that. So I guess  on the rust thing in specific, I think people see little value, it's hard for us to convince the  world of the value of putting rust crates into RPM format. Arguably, there's value. But it's a  it's an argument that we have to continually make if we want to say that. I think that there is  almost zero value in both the Fedora community doing it and the open SUSE community doing it. So  how could we make it so that it's just one community doing that packaging?  We would have to sit down and essentially come up with a way how we want to do that. But there's  also there's also a limit to how far we should go since we can't package the world anymore, at least  with Rust, with Node.js. These these ecosystems, they have really small standard libraries, and they  have huge dependency stacks. And at a certain point, you really come down to think, does packaging  everything as RPMs, does that really provide additional value? It does if you can, if you can,  for instance, so what there's been some recent development for the in the build service to  provide to provide vending of the of the dependencies where you can replace certain parts of them  with with RPMs that you ship and certain parts that you take from upstream. And there you could  say, okay, we have these core libraries where we provide support as a distribution or as an enterprise.  And then and the rest you can just grab from from npm or from from crates IO.  But yeah, okay, three different problems from my point of view. I mean, you're right that  I mean, the build part is one problem. And we could automate it or do it in RPMs or not RPMs.  This is one discussion. But the other problem is if you do it button up or not in a unified format,  then you would need to deal with the cost dependency between the RPM world and  Ruby gems and Python pit and so on. So if you update my my SQL, it has an effect on all the  other formats. And then you need to transfer this logic into into the update tooling. So  DNF, super and so on would need to learn about it. I mean, I think there's no wrong or right answer  here. I think we can should work in all directions and evaluate. We can't know can't limit people  to use other formats. We can't force people to RPMs. But on the other side, there is the additional  value I completely agree. But what I think comes to the origin of the question, when working together  with packages, no matter if it's past or whatever, I think there is really the big amount of our  packages we could really maintain together. And I think our tooling is is blocking us there at the  moment because build services so much different than Koji and whatever copper whatever else you have.  And that's definitely an area where we should. I mean, we don't need to consolidate the build  tooling, but we should maybe consolidate how we maintain sources. So yeah, and I think what's  also blocking us is convention. So we have really this really different conventions between open  SUSE and Fedora in certain regards. So in Rust, it's relatively comparable since they have essentially  the same heritage. But if you take a look at if you take a look at Python packaging, it works  it works kind of differently. If you take a look at Koji, I think that's completely different. If  you take a look at Ruby gems, it's also quite different. Although open SUSE uses a fork version  of gem to RPM. So I think to really to really come to share our sources together, we'd have to all  sit together at one table and decide we are going to do it now this way and reduce the  reduced differences that we have. So we did this once before, about a decade ago, where we came  together and reconciliated the way that our structures and guidelines in our packages were  I think was 2011 or something like that, where basically open SUSE and Fedora communities after  like, it was a desktop summit or something like that, where the guidelines were reforked from  Fedora to start over the guidelines in open SUSE. And things were built off from there. What I think  failed after that effort was there was no ongoing communication between the two projects to  solve problems together. So like something that when the Fedora Rust was being assembled that  Igor, Rates and I did together was we made sure that the stakeholders from the different  distributions that started the project together stayed in contact throughout the time that the  stuff evolved. And that's something that we probably should be doing if we want to like  make that successful this time around. From a tooling perspective, I agree with Adrienne that  like we don't necessarily need to be like, all right, Koji, OBS, Smush, done. That's probably  one not going to happen and two very, very difficult. But we can do things like there's  experiments going on with pulling the sources out of OBS's version control system into a  diskit style system with code.openSUSE.org, which is a Pagger instance. Like we can start and the  OBS team has actually started working on SCM integration with Forges and they're working on  Pagger integration that could actually be used in this manner. So we could start leveraging some of  the same conventions and blending a little bit of best practices to try to make this better and  easier to share between the two projects. One place that would make sense to me is the source  git thing that some people in Fedora and Red Hat have been working on, which is basically  maintain the spec file stuff in an upstream repository rather than in diskit, an actual  fork of the upstream source and then use automation to go to diskit and then to build from there.  It seems like maybe it could be made so the same source git tree could then go to  SUSE's git and build system from there, but using the same original source, that would be a huge  win. Yeah, I just want to correct a bit the things about continuous integration support at the moment.  This is not, I mean, this has a different purpose, but we are working also on an  alternative asset stores. So the entire project even can be replaced with the git, so that an  external git can be or other protocols can be used. So that would be the vehicle in our case.  And yes, then we could exactly achieve this and it doesn't matter if it's git or if we agree on  anything else. It's more important, I think, that we do the sources, the source hosting together,  and what's especially important, I think, is that we agree how do we validate which sources we can  trust? How do we trust upstream projects and so on, because I see there's the biggest value in  our aggregation. The build is, yeah, that's also something we need to deal with, but the  really, really important and valuable work we do is collecting the trust. Actually, that brings to  mind another thing, which is license identification in packages. I have no idea what you folks do  in SUSE with that. They're SPDX. Okay, which is excellent, because we are actually working on  a project to possibly move Fedora to SPDX for our license tagging. There's a lot of work to go there,  but that means that we can audit everything. Yeah, I think that would help collaboration there as well.  Yeah, and especially we could also leverage projects like Cable, which is a license, that's a  license checker module that runs in the open build service. It essentially scans all the sources that  you throw into a package and tries to find out all kinds of licenses. It's pretty good. It finds  all kinds of interesting things. About a year ago, I've packaged our studio for Open SUSE, and then  through that it came into Fedora. It found a lot of licenses that upstream bundled and  wasn't aware of. It also found a bunch of stuff that upstream still thought they had bundled and  didn't anymore for five years. What is that tool called? Cable. C-A-V-I-L.  I think it's known as license sticker. Oh yeah, license sticker was the old name.  So something that might be interesting would be if we could get Cable deployed in Fedora  infrastructure and then wire up a bot to do it on Pagger PRs. We could also replace the thing that's  done in Fedora review with using Cable and see, start using it that way. We could also share the  license corpus that we use. We actually discussed this in Tristan a little bit, Matthew.  Yeah. So Red Hat has a tool called PELC, which is used internally, P-E-L-C,  that is part of the REL process. It's an internal tool, not a public tool. We've had some talks about  open sourcing this. I feel like SUSE does this and we could collaborate. Might be something  we could move towards. I saw Neil's face. There's a tool that exists that I've never  heard of before. Yes. What language is it written in out of curiosity?  It's been around for a long time. So my guess is Python, but there's a small chance of it being  Pearl, right? I don't actually know. I haven't actually looked. Cable is Pearl. We could merge  all the pearls. Excellent. Yeah. If merge all the pearls is one thing, but then if it's Python,  then we'll have to have a shootout and figure out which code base is less awful to work with.  The one that's not in Pearl is the one that's less awful to work with. Come on.  And I'm not even wearing my Pearl shirt today, but I have one that I wear all the time. I love  Pearl. Matthew, I've tried to be diplomatic here. That's not too bad. It's really hard to  transition though, because we have a lot of other things we want to hit on.  Sorry. I'm going to go to Ben here. I'm going to read this because I couldn't break it up  so easily. The messages really collaborating amongst the projects really resonates. I think we see it  as we talk about it. Ben, do you have any opinion about why that is?  You just started out with Ben. Ben, yes. Ben said he would do it.  So you asked if I have an opinion, which is a funny question to ask.  Think about my own experience. I got started contributing to Fedora because I had been using  it and here's the thing that I'm getting for free and people are putting effort into doing.  And I felt sort of an ethical obligation to give back to the community. And I think that's true  to some degree for a lot of people who participate in open source projects. And so the whole idea is  that you're putting your code or your documentation or your design or whatever out there for everyone  else to also use. And so it's just sort of inherently collaborative. And because there's not  these strict boundaries of, yeah, we have Fedora and OpenSusa and Ubuntu and Debian and all these  other distributions that have some differences, but we're all using the Linux kernel. We're all  using the same basic user land tools. So there's not this strict thing where I'm working in this  one community and I'm only going to work here because nothing touches. As soon as you start  working on anything that goes into a distribution, you're almost certainly impacting other  distributions and then it starts to make sense to work together.  So Dan, actually, I think you said you would probably also...  Yeah, I mean, I'm afraid Ben already said most of the important parts. Since if you do open source,  at least in my opinion, open source or more or less, for me, open source is to a good extent  about the community, about the people. It's not all... It's also about the code and that you can  modify it, but it's also mostly about the people with whom you work with. And then it's about  collaboration and it's kind of wired into the heart of the whole thing. I mean, you can put it  out there into the public so that people can reuse it. And I mean, the whole thing started like that.  So it's kind of baked into its genes.  Okay. So the benefits... I mean, you could see the benefits with collaborating.  Do any of you have any specific examples that you could think of?  So I guess the earliest example really for me that I think really demonstrates this quite well  is when I started working on bringing up support for soft float arm architecture,  so the really crappy useless arm architectures for DNF and RPM. So that involved going to,  at the DNF level, adding in the markers to identify to that, but also teaching Lipsul how to do it  and going through to that, which meant touching both a Red Hat project and an OpenSusa project,  and then going to OpenBuild service so that it would learn how to actually sort that  and be able to select the correct builders for that as well and those sorts of things.  Like when I was doing that architecture bring up for a professional project, thankfully nobody  will ever see the light of day of it because it was a terrible thing and I don't ever really want to  do that again. But because it was something that involved some, you know, different distributions,  different projects, I even found, you know, there was a Linux distribution that was actually shipping  RPM based distribution with DNF and all this stuff, you know, for ARM v5 and they were like,  yeah, you know, if you can help us out with, you know, getting this all fully working with  this new stack, that'd be great. And it's like, all right, sure. And they tested some of my patches  and made sure that it worked. And, you know, so like with a little bit of Magia's help and, you know,  Michael Schroeder's help in Libsol and Adrian in OpenBuild service and Yaroslav Ronsek,  sorry, I screwed up that name, I'm sure I messed that name up. In the DNF team,  it went fairly well and I was able to easily like work with a whole bunch of people,  a whole bunch of random people to get something that was both simple and complicated at the same  time actually done and usable. Ed, no, that's still a terrible architecture.  Right. I think another good example is also OpenQA, which was also previously mentioned. I mean,  OpenQA is relied on by, I mean, that's the thing that makes OpenSusertumble wheat  roll and be actually pretty stable for a rolling release distribution that releases  up to five times a week. If you compare that to other rolling release distros, which  break, break, break and break. We won't name names. No, we won't. But so it's, and a good chunk of that  is achieved by OpenQA. And then Fedora using it, you see, especially Aedam,  contributing to OpenQA, also upstream and fixing stuff that's sort of is used by Fedora.  And so I think that's one of the, that's a pretty good example.  And Adam also added something unique to the OpenQA ecosystem that I found really handy.  He wrote a Python client for OpenQA. So I didn't have to write a URL code,  it's your face with it. You're not going to get away from that fight. We're going to come back to it.  I figured I just pointed out, like, Adam added something that actually made it much more usable  for a broader set of people. And that helped me start like, you know, poking at it and testing,  like whenever I'm working with OpenQA stuff, like either in OpenSusert or Fedora, I use the  Python OpenQA client so that I can, you know, automatically go find something, fetch it and  take a look at stuff because, you know, there's a lot of stuff in OpenQA. And navigating by hand is  very, very annoying. And if you want to, and if we want to go down this Python versus Pearl route,  OpenQA is now also adding a Python API. So in case you don't like Pearl at all and don't want to  use the test API, you will be able to use the Python-based test API soon. But it looks exactly  the same. I can expect Adam to port everything over to Python. I expect that will happen.  It's possible, but it looks the same. So sure. But it's the thought that counts.  And then we'll rewrite it in Rust. Okay, you're banned.  It works. But if you have a common goal, those have a different one,  collaboration works mostly automatically. Most places.  So it's the part that you find to come and go and to come and benefit.  Yeah. That's I think been the hardest part in general was just finding a way to bring everyone  to realize that there is a common goal because a lot of times, you know, people don't realize that,  you know, they may be framing the problem differently in their head or they're thinking  of it in different terms, but it's still like the same kind of thing and everyone's still  and sharing the workload, you know, many hands make light work is, you know,  an old phrase that is still true even today. Like collaborating makes it a lot easier for us to do  bigger and bigger things together better.  Is it in doing the questions, we had one that a lot of you actually responded to that you  would want to answer and one was kind of opposite of what we're talking about, but  collaboration can sometimes feel like a drag on innovation because of the required effort  to communicate and synchronize with many different people. So how do you overcome this feeling?  Oh man. That one, that one seems like it speaks to me in particular. The  the hard part really is trying to, the hard part about making the collaboration work is trying to  not necessarily make it someone else's problem because that's a lot of what sometimes that's  what a lot of people kind of default to, but trying to find a way to have everyone see the  benefit of solving the problem. And the reason I find it important to do that, even though that it's  pretty grungy and and unrewarding work most of the time, is because the end result is I get  more people that can help me solve the problem better because in a lot of cases, if I'm doing the  thing by myself, I'm not a 10x engineer. I do not, I do not have the capability to make magic  out of nothing. I am not that good. And and the only way I'm even halfway successful on anything  that I'm doing is because I can get help from others who are willing to chip in and and that  make it successful. That's the real 10x engineer right there through one who can convince 10x  engineers to work with them, right? I still feel like a failure after drag.  I mean so it may sound a little self-serving, but I think having somebody, you know, in that  sort of act as like a project or program manager role who their contribution is just managing  some of the collaboration and the communication really helps. You know Fedora didn't have somebody  in my role originally and it was sort of a crazy disaster. Now it's just like slightly less crazy  disaster, but I think, you know, that sort of the same principles apply when you get into, you know,  smaller even sort of more ad hoc collaboration is if there's not somebody sort of keeping the  cats all going in the same direction, you then do have to communicate with each person individually  one-on-one and that becomes an unmanageable overhead. I'm going to echo that for the community  based work as well. People are hard.  Oh come on Adrian. I think you kind of have the hardest job of all. You have to bake all the  things work for everyone. No, no, no, no, I have to excuse that it's all content, all problems are  only the content. I'm never going to. I don't know, I distinctly remember a couple of times  where I have to go in and ask you like, hey, this thing is not working because stuff's missing.  What do?  My daughter has solved her insect problem by deciding to go to the mall with her friends.  That's an interesting solution.  Honestly, I'm proud. Who among us has never said bugs are hard let's go shopping though.  Fair.  Yeah, but I think what really helps in terms of collaboration, if it feels like a drag is  to essentially consider if I push this stuff upstream, yes, it's going to be more work in  the short term, but in the long term, I get more eyes. I get more people to look at that.  So in the long term solution is very likely going to be better. So that really,  that really at least helps me because I know, I mean, sometimes you submit a pull request somewhere  and then the discussion starts and it goes around in circles and weeks later, it's still not merged.  And at some point you get annoyed and you think, oh, I could just for this thing and be done with it.  And then the project lives for 10 years and you think, oh, damn it, I should have been maybe a  bit more resilient and actually go through with this pull request. So I think the,  I think in the long term, usually collaborating tends out to work, to work out positively for  everyone. I have a response to this for myself personally, overcoming that feeling, it's a lot of  adaptation and not being so like, don't hold on to your idea of what the outcome of this is going to  be, what it evolves and kind of accept that and release it. How I go about it, at least  with the non coding teams within Fedora working amongst themselves.  I'd probably say coding, programming, that's the easy part. That's the part that honestly  does not matter very much most of the time. The actually getting people to sit in a room together  and not throw pitchforks at each other, that is the hard part. That is sitting in the room.  Oh my goodness. Yes. Like, that's those are the things that really strain my desire to do this  kind of stuff because it's really difficult sometimes when you have really unfortunate  personalities is the way I'm going to describe this really unfortunate personalities in the room.  And whether it's code, whether it's documentation, whether it's design, whether it is  advocacy or any of those things, it really doesn't matter what space it is.  Like juggling personalities and trying to get people to mesh together is really the hard part  because when that falls apart, that is what leads to forks. That's what leads to split efforts.  That's what leads to project failures and things like that.  And notes you said for both forks and pitchforks. One of the things that's I mean,  some of its personalities and some of it is  people who  maybe need to learn to work better together in collaboration. But a lot of it is  this is very personal for so many of us. And it's very hard to not take the smallest things  very personally, very quickly, especially when it is something where you put in 100 hours a week,  both your work time and your free time and whatever it's been your life and your passion.  And it's very easy for somebody's comment that they didn't mean a certain way that sounds  dismissive to set you off or for even things that started as technical to become very  just adversarial. And so it is a hard thing to deal with.  I'm just going to add that codes of conduct make collaboration easier.  It's definitely something we implement in the individual community to make things  easier for folks to work together. And it's definitely helped.  I don't know if codes of conduct in themselves help, but having a process to make that code of  conduct work is I think probably more important because I have been involved in a ton of projects  that have them, not naming any particular names, but they have code of conducts and things like that.  But with no mechanism to make them real, to make them enforceable, they're just words on a digital  paper. They don't mean anything. It's quite one thing to just say, we're going to copy the  contributor covenant or insert your favorite template of a code of conduct here. It's quite  another to say, I'm going to back this up with action when somebody actually  like point something out and says, if this is a problem. And in a lot of open source projects  and a lot of communities, sometimes you just have one without the other. And that's just,  it doesn't work. It's not enough. You have to mean what you say and say what you mean.  Can we let in the middle a little bit more? And we'll talk about like, so,  you know, with collaboration, what do you all think could be improved? I mean, I think we did  on a few things already, for sure. But other things that come to mind.  I think that more that we have is our distributions are so large and we have so many individual  pieces. And in some cases, it won't be able to collaborate or the additional time effort to  agree on something would be way too much that it's inverse, basically. But you have so many  packages that there's definitely some stacks where we could aim for that we would be,  should be able to work together. I don't know which one to pick because but we,  let's say, I don't know something. Just if you want, if you want to this example.  Where we have the same pine, same pain and where we should. Yeah.  Yeah. I looked at a little Rust project I have that all it does is it makes a little world with  trees in it that you can move a little figure around this. It has 236 dependencies.  So am I ever going to package that up in the traditional manner? Not if some unless somebody  else agrees to package at least 234 of those dependencies. I have written an extension for  VS code and including development dependencies. It's over 1000 node models. Yeah. And I have been  conservative. So I'm not packaging those. No JS makes me want to replace this water bottle with  something way stronger. Yeah, but where I definitely see where we could collaborate where we definitely  I think can collaborate and should collaborate most for instance not only the packaging stuff but  also image building since that's that's something where essentially every distribution has its own  tooling. So in the open build service and for the open source of distribution mostly use Kiwi.  Fedora has I think Lorax is the tool. There's nine different build tools.  Okay, I wasn't aware that it was so many but essentially Kiwi is already a pretty is a pretty  powerful tool and it would be great if we could just if we could also work work together on  something there since Kiwi can build Fedora images for instance it can build a Bantu and yeah  Neil is Neil is doing his thing since he brought in a good now we have 10 different ways of doing  it is what you're saying. I will aggressive yeah I'm not saying we should make a tenth one.  That will aggressively rip out all the other ones if we replace it with Kiwi because I am so tired  of fixing all of them every time I make a change to them. It's a massive time sink and it sucks and  everybody should feel ashamed of how many times I have that this wheel has been spun over and over  again with square wheels that are chipped at the edges with sharp spikes on the sides.  I think another way we can improve the collaboration is you know we think a lot of our collaboration  tends to focus on very technical things whether it's you know packages or how we're building  process. I don't know if we necessarily doing a very good job of collaborating on how we build our  communities but you know it's really interesting you know I was right before the world shut down  last year I was at the scale conference and I spent about an hour in the OpenSUSA booth  talking to an ambassador there and every problem that they said they had like that's very familiar  and then you know the the OpenSUSA community survey and it was back over the winter maybe  you know a lot of the problems I was like oh these are the exact same problems that we face in  Fedora and I think it'd be great if we had a way to work together on those things too right because  like we said the code is not always the the hard part and I think there's a lot of benefit you  know culturally it feels like our projects are very similar we have a lot of shared community  members so I think we you know collaborating on the the community parts is a big area that we  haven't really tapped yet. I think an unintentional positive of pandemic COVID times has been a nice  chance for our communities to engage at a higher level at these virtual events  and you know at Nest and the release party we've had you know different organizations come and hang  out with Fedora you know and this cross-collaboration panel is a great example of like something that  maybe wouldn't necessarily happen just this way if COVID hadn't happened we hadn't put more time  and energy into virtual events and I think it's given a different swath of people a chance to  connect with communities that are tangential so we've seen more engagement and more folks at our  events and I'm guessing that that is part of it it's definitely not intentional it's an observation.  Yeah but I like to ask again if we have a shared community somewhere then they need to have a shared  goal because then they can talk about the same thing but we would get this if we share some  components if we would have for example the same GNOME desktop then of course if you speak about  GNOME then it doesn't matter if it's Fedora or Fedora so it would be then the yeah I don't know  then who would maybe even need to come name something I don't know. So is Fedora? Fedora is  working on a community outreach revamp and when that is all set and done and has a bow on it I'd  love to share that with other communities so they can adapt it. We're planning a resource library  you know it's gonna it's happening. Yeah like this is something that I kind of ran on in my  platform to become a board member was I personally have felt that the OpenSusa community lacked a  particular I want to say empowerment to feel like they could communicate and collaborate and  communicate and and execute on things in a way where they felt like they could be seen like  something that was a problem that I had observed like listening in the OpenSusa Discord server  in the matrix rooms talking to folks like pretty much everywhere outside of the OpenSusa Factory  IRC channel. The common issue was you know I'm trying to figure out how to do something but I  can't figure out how to figure out how to do something and when I even if I could figure out  how to figure out how to do something I don't know how to make people know that I did this thing  and like maybe look at it or see if that could be made better and things like that and I feel like  this is a problem that spoke to me so very hard and and it's something that I'm hoping you know  that you know with this closer ties between Fedora and OpenSusa we can learn a little bit from each  other about how to do that sort of thing so that our community members can feel empowered to drive  change excellence and growth in in both projects like both Fedora and OpenSusa deserve to be able  to be successful and and make you know the the people that use these projects to live to work  to play to be happy with it. Sounds good. Such a statement Matthew.  But you don't just think that this kind of a problem is because we have too many communication  channels and they are just too little. So at least with with everything but IRC and the mailing  lists they're all bridged together and unified so when someone talks from one avenue the other  people every one of the other avenues actually can see each other and respond to each other  so it feels a lot more connected than it would like say for example if you go into Fedora's  matrix discord or telegram rooms they're a lot more fragmented because there wasn't an intentional  approach of connecting them all up front whereas when we were introducing the real-time chats  across the different platforms in OpenSusa a distinct goal was making sure they were all connected  to each other it's generally not allowed to introduce a communications platform that is not  connected to the other ones and so if it's a real-time chat platform where users and developers  and other folks would be you know communicating with each other there needs to be a way to connect  them all so that nobody is missing from the conversation and that was something that I think  alleviates a lot of that problem at least on the OpenSusa side the main actual problem has been  that we have not been able to connect IRC to all the rest of them because of a whole bunch of historical  mess with how the IRC channels were managed with the unfortunate situation around IRC that has  happened it has been an opportunity to actually fix that too so like one of the things that I'm  hoping we can actually resolve soon is bringing everybody you know into the onto the same page  so that you know someone who unfortunately is on IRC can talk to everyone who's on Discord  Matrix Telegram and whatnot and and be able to to do that sort of thing um but the second part  of it is not just the real-time chats but like people don't know how to get started to do something  in a lot of cases and I don't know the answer to that problem I don't know how to solve how to help  people like get started with their thing other than case by case things and that doesn't scale  so Neil with kind of sort of sort of solve this problem a little bit Fedora with the join-sig  it's a like a group of people it's a chat room and a repo that's a very like low commitment effort  and it's simply a place to send newcomers and the folks that hang out in that chat room help them open  a ticket like a little intro ticket trying to like get some information out of them and they  connect them with the group that makes sense so it's a big problem we're solving it with people  and it's working pretty well okay cool something to look into then  I already noted too um so it looks like we're sort of hitting that's a  top of the hour and I think we'll probably finish it up is there any last words that  anyone would like to say famous last words about that I think Neil's uh  Salil a quick from a few minutes ago was an excellent ending point so if you go back to edit  this video just stop it there chop it and that that I think that says I'm gonna do a shame  most love uh everyone's invited to nest 2021 and open season will be there so should be a great time  and we'll give a great talk about obs stuff and how they're supporting the door building fedora on it  I mean uh should we do this panel at nest I think it'd be fun it'd be a different it'd be a second  audience call call for participation still still open right now it is open until July 16 get your  proposal said sure but yeah like I I'm really happy that we did this this is this is really nice  and it lets us show like the perspectives from both of our communities and hey adrian is here  which like I have I barely ever see adrian on anything and that just excites me in a whole  different level I know our time zones do not align at all  all right well I want to thank you all and enjoy the rest of the conference and thank you  for being here thank you thank you thank you thank you thanks for having us thanks bye
openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_0_155.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_156_1187.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_1188_1427.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_1428_1475.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_1476_1523.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_1524_2027.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_2028_2075.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_2076_4235.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_4236_5507.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_5508_6611.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_6612_11495.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_11496_13271.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_13272_13367.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_13368_14531.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_14532_24119.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_24120_30875.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_30876_35891.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_35892_35999.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_36000_37427.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_37428_37967.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_37968_38123.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_38124_38867.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_38868_39107.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_39108_40451.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_40452_41291.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_41292_44387.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_44388_46535.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_46536_48551.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_48552_48671.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_48672_48719.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_48720_49727.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_49728_49775.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_49776_49859.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_49860_50555.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_50556_54695.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_54696_55631.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_55632_68999.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_69000_71051.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_71052_74471.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_74472_74687.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_74688_74927.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_74928_76127.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_76128_76319.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_76320_76535.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_76536_76727.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_76728_83507.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_83508_87479.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_87480_87563.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_87564_87611.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_87612_87743.png openSUSE_Virtual_Conference_2021_-_Live__Cross_Col_Ru2i3EbNmcI_87744_87744.png