Recent Ubuntu Community Refinements

Our community is at the heart of how we build Ubuntu. Recently there were some concerns expressed about some aspects of our community and I have been working with various community members and internally at Canonical to resolve some of these issues to make things smoother.

I just wanted to summarize some updates:

  • Regular, transparent planning – we want to improve how we plan the delivery of work items, and make that planning more nimble. While the major decisions are reserved for primary discussion at UDS, we want to regularly and transparently checkpoint progress on those projects, and ensure things are moving along. To do this the engineering managers at Canonical will perform this planning on a monthly basis with our community. An an example, with my team, we will decide at UDS what major projects we will work on and document the work items in those blueprints, and every month I will ask the team to commit to delivering an agreed set of work items that month and update the blueprints accordingly. This will make it easier to understand who is working on what, what needs to be done, and areas in which people can participate. This entire process will be completely open and transparent and I would like to encourage our wider community to use the same approach. As an example, this could be a useful technique for our LoCo community to use for planning their work too around advocacy campaigns. All of this work will continue to be tracked openly in status.ubuntu.com.
  • Training our engineers – our engineers at Canonical are expected to openly and transparently perform all work that is not considered customer/company confidential. While this expectation is clear, there are sometimes cases when this doesn’t happen (e.g. if someone joins Canonical without the experience of working in an open environment and isn’t really sure how to do this). I have prepared an internal slide deck with these expectations and workflows clearly laid out; my team will be working to ensure everyone gets the deck, reads it, and gets an answer to any of their questions.
  • Regular leadership problem solving meetings – one problem we have today is that we don’t have a regular problem solving meeting in our community in which our governing leaders are present at. Instead our different leadership boards (e.g. Community Council, Forums Council) tend to resolve issues pertinent to that specific board. We think it could be useful to have a meeting every two weeks that has representatives from our different governance boards and our community can join and raise topics for discussion. We are going to run the first one of these sessions tomorrow (Tue 19th March 2013) on Ubuntu On Air at 8pm UTC. We invite you to bring your topics there on IRC for discussion.
  • Online UDS refinements – as I blogged about last week we have released a survey to gather feedback about how to refine and improve UDS. We have already made some plans for some improvements but I plan on organizing a community meeting to discuss this more next week (I can’t later this week as I am at an event). I think there is an opportunity to refine the format of UDS into a form that becomes a useful and repeatable way of coordinating meetings in a community.
  • Weekly Updates – I have reached out to the engineering managers on some of the core projects at Canonical and asked them to provide weekly updates of work going on. We have already seen the first updates for Ubuntu Touch and Mir.
  • Prepping announcements better – while the major announcements are now out, one piece of feedback I received is that our community felt ill-prepared around things such as the Ubuntu Touch announcement, and people such as our IRC/Forums/Community councils were inundated with questions and didn’t have good answers to those questions. If we need to make future announcements in the same way again, I am going to ensure our core governance boards are clued up first and we provide a FAQ for our community to refer to when getting these kinds of questions. This should relieve this concern.
  • Improving our community on-ramp – one area where I want to drive some improvements is making it easier for people to join the community. We started some work a while back to improve the community landing page on ubuntu.com and I have asked Daniel Holbach to drive that work to completion. I am also working with the Ubuntu Touch and Mir teams to ensure that they have awesome documentation and guidance for how people can participate. A good example of the progress being made here is the Mir documentation. If you would like to help improve these docs, then feel free to dig in and help, or share your ideas on the mailing lists.

I want to get as much feedback on these steps moving forward as well as other ideas and areas in which we can focus. You can always grab me on IRC on freenode (my nick is jono) and I hang out in #ubuntu-community-team. Also feel free to drop me an email and join my regular Q+A session every week. Unfortunately, this week’s Q+A session is canceled as I need to be at an event, but I will be back in the regular slot next week on Wednesday at 7pm UTC on Ubuntu On Air.

  • Anonymous

    Wow. The answer to “We feel out of the loop/not involved in the process” is not “write a better FAQ”. Clueless.

  • http://benjaminkerensa.com/ Benjamin Kerensa

    I think its fair to point out that Jono has been working closely with leaders in the Ubuntu Community to address the concern of members feeling out of the loop or not as involved and the roadmap that I have seen discussed will do a lot to address those issues.

    It is not possible to make everyone happy but I do think Jono is doing his best to address at least a good portion of the issues that lead to the recent Blogalanche (Jono Terminology) and there will definately be interactions with Ubuntu Community Leaders and Jono’s team that continue as time goes on to ensure that we are moving towards improved processes and to do health checks.

  • Anonymous

    No. The answer to being more in the loop is the first point about transparent monthly planning and continuing to ease how topics are discussed and planned at UDS.

    Now, there are likely to be some projects that will be private in the future. This is just the nature of some business-to-business relationships, but for those announcements we will better prep our community first as I outlined.

  • http://benjaminkerensa.com/ Benjamin Kerensa

    +1

  • alistair munro

    It strikes me that some of the recent problem has been how to handle the difficult moment when people want to leave/move on from the Ubuntu project. One common theme I noted running through most of the critical blog posts in this episode, is that the folk who had taken the decision to leave the project felt the need to go as public as they could in order to get the ‘catharsis’ of having their point heard. A couple of years ago, I left a job where several problems weren’t effectively resolved, and by the time I made the decision to leave, I was feeling quite bitter towards the company because I felt that hadn’t been given the chance to air my grievances to the company in a way that might be listened to. With hindsight I realise that this situation is difficult for any large group of colleagues to avoid. But at the time the anger and frustration was very real, and difficult to contain. One thing that company did do well, is they got somebody from another area of the organisation, who was dispassionate either to my position or that of my management, to sit down with me on my last day. The aim was a “tell it as it is/get it off your chest” session. My comments were faithfully recorded expletive warts and all. I thought that worked really well as a venting valve. And probably helped me feel less inclined to go public with my disagreements. Maybe the “community/company off ramp” might be worth some attention also? Granted it’s a lot harder to gracefully manage people leaving than joining.

  • Anonymous

    Thanks, Alistair, I think you make some really interesting points. This could potentially be a useful thing for our governance boards to do – e.g. if someone leaves the LoCo community, the LoCo Council could do an “exit interview”.

  • Bruno Girin

    On the last point, there is some brilliant work currently being done on testing with things like Autopilot: it would be great if that type of initiatives was more publicised. I was going to say it would also be great if it had better documentation but I just had a look at it and it’s been seriously improved since I last looked at it :-)

  • smspillaz

    Hey Jono,

    I think these are definitely a step in the right direction. One of the things which caused me so much concern which has effectively put my Ubuntu involvement on-hold at this point is the fact that there seems to be very little stability within the Canonical-run projects which means that you don’t know if you’re going to get pushed back over some internal thing that you don’t know about.

    For example, the straw the really broke the camels back for me was when I worked on a number of substantial performance improvements in compiz. Granted, these changes were large, and they were not trivial. But they were developed using all of the quality practices that we come to expect (a large suite of unit and integration tests, peer reviewed code, lots of manual testing in a ppa, proven benchmarks), and at the end of the day they were rejected suddenly because “we’ve decided we don’t care about compiz anymore”. That felt very inconsistent with what I was told one week prior which was that the maintainers of the compiz and unity-legacy stack were very interested in getting my work in-tree. And then suddenly I’m being told that maybe we do want this work in-tree again, but we’ve spent so long deciding whether or not we want it that its past feature-freeze and I have to go through yet more paperwork. It was really at that point I just got frustrated with the whole thing to put my attention on other areas.

    If those working on these projects were told that they were soon to be deprecated, or that Ubuntu didn’t want any more changes to them, then I wouldn’t have spent all that time fruitlessly doing something which turned out to be a waste of my time.

    Is it possible that in the future the decision to take projects behind closed doors is going to be managed from a community perspective? -Eg, is it possible to tell the community “we’re going to replace this with something, but we don’t want to share it yet because its not done”

    At the end of the day, sharing that kind of information early is something that I don’t think harms either the company or the community. The community isn’t led on by a false set of circumstances, and the company doesn’t have to spread information that it potentially doesn’t want its competitors to see.

    Thanks.

  • http://twitter.com/jspaleta Jef Spaleta

    I’m missing how the transparent monthly planning is going to translate in easing blowback from the “ta-da” product strategy announcements. I mean mir.. 8 months of dev and no heads up at the last UDS to externals prior to vUDS? And the post feature freeze codedrop of the amazon results in the dash which was not brought up for discussion at a planning UDS.

    I’m missing how internal Canonical teams are going to feel comfortable communicating that sort of stuff in a more transparent fashion. It’s one thing to hold up your team’s work items moving forward and pledge to be more transparent about. But I’m pretty sure your teams work items aren’t the crux of the problem. Well other than the short notice for vUDS, which was a problem. The real hard issues are the product strategy issues, and I’m just not seeing how the the new transparency push outlined here is going to help bridge that divide. People are still going to be blindsided by product strategy announcements and code drops, and those are exactly thing things people are struggling with understanding and planning for.

    -jef

  • Anonymous

    Jef, I am not going to deny that there may be other Tada announcements. I would prefer if we did everything out in the open from the beginning too, but I don’t get to make the decisions, and that is sometimes the nature of the business/community balance. It is what it is, and it is reasonably out of my control.

    My point about monthly planning is that outside of the Tada announcements there is still room for us to be even more open about how we build Ubuntu and welcome people to participate. vUDS feeds into that: in reality the Tada announcements account for small proportion of all the development we do. All of those projects are now fully open and my goal is to ensure they function and operate like awesome Open Source projects.

  • http://twitter.com/jspaleta Jef Spaleta

    I look forward to seeing the CLA dropped so all the Canonical projects can “operated like awesome Open Source projects.” Good luck with that.

    There’s a pretty big difference in “open from the beginning” and “announced 8 months after initial development started”, well at least to me.

    But other than the fact that “tada” moments exist, which you have no control over, the timing of them seems pretty out of balance. Dropping announcements close to feature freeze deadline for a release, and asking for feature freeze waivers is not setting a good example for externals to follow.

    Late breaking announcements, esp. “game changers”, makes community contributors feel like they are wasting their time working on things that no longer relevant to the direction Canonical is pushing. If the community can’t know 6 months out what technology Ubuntu is going to be using, then you can’t really expect community to plan their invaluable volunteer hours doing support tasks associated with that tech.

    You have to find a way to deal with the tada announcements in a different way that how its been handled if you want people to continue to contribute in technical areas. Because as it stands its not balanced, its pretty one sided. If you can’t build technology roadmaps in a way that includes your volunteer contributors, then you aren’t making optimal use of that resource and it will whither.

    -jef

  • smspillaz

    So to be honest, I think a realistic expectation needs to be set here. Canonical can do code-drops and development behind closed doors, but I don’t think its fair to expect that people will want to invest into something which isn’t stable from a project lifetime point of view. For me – I think mir is a great project, the design is fantastic, the development process is great. But I can’t contribute unless someone can assume me that its not going to be scrapped and rewritten in six months.

    Jef speaks for what I think is the (quiet) group of technical contributors which I think Ubuntu is trying to attract. Lots of people has been framing this debate as a matter of community control and consultation. That frame is erroneous. The community realizes that those who put the money in get to set the direction. What they want a guarantee about is whether or not their effort is well-spent here as opposed to other places. Recent history hasn’t made that case out.

  • Philipp Gassmann

    «I have prepared an internal slide deck with these expectations and workflows clearly laid out; my team will be working to ensure everyone gets the deck, reads it, and gets an answer to any of their questions.»

    I would be interested in these slides, can’t they be shared to the public? Such workflows on how to work with a community could also be of interest to other companys and individuals.

  • http://IronPatriotNY.me/ Ricardo N Feliciano

    Sounds like welcome improvements.

  • Andrew

    I have to agree with Jef. Often times, really poor Canonical [the company] decisions get covered up by words like “awesome,” “transparent,” “cadence,” – but events like the Mir announcement brings everyone back to the fact that Canonical IS a business. They aren’t leading the community, but steering it in their best interest. There is nothing wrong with this, as it’s Canonical’s financial backing to do what they please – but pretending it’s anything else is just rude to those of us who ARE open-source enthusiasts. Things like the CLA, continuing to push upstart, starting from scratch with Mir, are all Canonical [the company!] driven projects in the effort to make a profitable product. I mean, compare the Mir announcement to the discussion of porting gnome to wayland, on the desktop-devel-list. Several inputs from various distributions and developers – it’s easy to get excited and understand what to look forward to as a user.

    So, the company operates behind their closed doors, but employes a community leader to attempt to convince people that their actions are reasonable? The fact that you felt this blog post was even necessary (and admit that these events will even happen again!) is disheartening enough. It’s nothing against you personally, as it would be incredibly difficult to wear a Canonical shoe on your right foot, and Community shoe on your left.

  • http://twitter.com/jspaleta Jef Spaleta

    The Mir development stands out compared to other decision-making in one important way. In 2010, Shuttleworth made Canonical’s plans to support Wayland public. Very public.

    http://www.markshuttleworth.com/archives/551

    This wasn’t a simple case of internal business decision making. Canonical previously communicated a direction to the wider external community, and said “hey heads up, we are going this way next.” And then with pretty much no further communication on the matte for a couple of years, instead of zigging like they told everyone they were going to do, the got back together behind closed doors and zagged.

    Mark wrote: “We’ll help GNOME and KDE with the transition, there’s no reason for them not to be there on day one either.”

    This is the sort of thing that is worrisome. Not that Mark made a promise, that Canonical was unable to fulfill. Not that Canonical changed course after re-evaluating. No, that’s par for the course. No what is worrisome is that Canonical completely failed to cycle back and warn anyone that he previously encouraged that Canonical was changing course again. What is worrisome is that for some reason Canonical has stopped thinking of the larger community as stakeholders important enough to communicate decisions to in a timely manner. Somewhere between Nov 2010 and now… something changed with regard to the company’s regard to keeping external contributors informed about roadmap changes. Something fundamental in the balance in the corporate/external dynamic has broken. That is very worrisome.

    And I don’t see the proposed changes Jono is talking about helping resolve that fundamental corporate/community dynamic. I mean the changes sound great, but they aren’t going to be applied as widely as needed. I’m slightly concerned that people are going to read this and assume that the process changes are going to take root across all of Canonical’s teams, which will further caused an expectation/realtiy mismatch and lead to more external contributor frustration, not less. The core problem is the new Canonical concept of product strategy and how product development is being forced into Ubuntu processes that pre-date the “product strategy” push when those processes were not designed with the “tada” drops in mind.

    -jef

  • http://www.happyassassin.net/ AdamW

    Jef: there’s a clear reason to do the tada moments close to freezes, though: the whole point of the ‘tada’ philosophy (which I don’t agree with at all, BTW, but I am the kind of guy who likes to argue all sides of a debate) is to surprise everyone with a big exciting New Thing. It clearly has more impact if your tada moment is “Here’s our big exciting New Thing that we’re shipping NEXT MONTH!” than “Here’s our big exciting New Thing that we’re shipping…in four months’ time when you’ll have forgotten about it so we’ll have to re-announce it!”

    The ideal is of course “Here’s our big exciting New Thing that we’re shipping RIGHT NOW!” – which is what Apple got great at doing, the “…and one more thing” model – but obviously Canonical has decided that would be impractical in a project like Ubuntu. I think their approach is to try and figure what’s the shortest delay between ‘tada announcement’ and ‘ship date’ they can plausibly manage, and go for that.

  • http://www.happyassassin.net/ AdamW

    You might find some of the materials at opensource.com useful – there’s a whole book on The Open Source Way buried in there somewhere. RH has a mandatory multi-day orientation for all employees (including lawyers, accountants, HR people etc etc) which includes a bunch of stuff on open source, but (ironically!) I don’t know if we have the materials for that available in public. Maybe we do, but if so, I dunno where.

  • http://twitter.com/jspaleta Jef Spaleta

    Sure, I understand the motivation. It’s the execution that is damaging. The problem is they co-opting of the established Ubuntu processes and brand to do the Appe-ish excitement drops. The Ubuntu processes are not designed for tada. Hell man, even the CoC codifies the ideal of being early communicators. CoC 2.0 and CoC 1.1 section of collaboration. Personally I think the tada approach as executed right now is contrary to the intent of the CoC’s commentary on collaborative best practices.

    Want to do tada announcements? Need to do it to win mindshare in the OEM marketplace? Make a new brand and a product space that is separate from the established Ubuntu processes and brand… a parallel track..a product track.. and go absolutely crazy with the wow factor and jazz hands and bedazzled belt buckles (seriously Mark’s belt buckle in the UTouch announcement video was to die for) in that new space.. for that new thing.. with the new approach.

    What Canonical is engaging in right now is brand value destruction. The value of the Ubuntu brand is the “community” and everything they are doing with the new platform approach is not that. Unity as a brand could be the new “platform” brand it could mean everything Canonical needs a product strategy to mean in one word. Ubuntu as a brand means something else, it means the wider product.. the collaborative project… not the tight integrated platform.

    In the same way that Canonical owns the Kubuntu brand, but doesn’t control Kubuntu technical decision making. Canonical could bequeath the Ubuntu brand to the wider community and could pour their engineering resources into the Unity brand and do all the close door decision making they want around the Unity brand because there isn’t the same collaborative sense to the brand. Everyone would respect Canonical’s need to make the Unity brand a very tightly controlled product brand.

    -jef

  • alistair munro

    Even if Canonical and Ubuntu are well aware of the issue(s), it would be an opportunity to express thanks for what you’ve contributed and sorry that we couldn’t resolve this before it got to this stage.

  • Matt Jones

    Why are people equating this to the end of the Ubuntu project? Its just growing pains.

    First point: Why is everyone saying Canonical killed Compiz when they decided to go with Mir? @smspillaz acknowledged in his own blog post in December 2012 (http://smspillaz.wordpress.com/2012/12/24/sideways/) that he was basically done with Compiz, and that Canolical was going to drop it when they moved to Wayland. So them going with Mir instead of Wayland changed nothing for Compiz.

    Second point: If you have ever developed software, you know things are always being created months before they are announced. That is because you don’t know if they will work yet, and you need a contingency plan if you external dependencies don’t work out.

    Third point: Ubuntu is not a pure community project. It is a company/community project. Canonical will have veto power over the community. If they did not, then Ubuntu would be just like Debian. That was the whole point when Marc created the project.

    There is no reason to bail-out of the project, just because you don’t always get your way. Anything as large as Ubuntu will have to have compromises. Jono is doing a good job of trying to change things to work better with the community.

    Why don’t people work with Jono and the Council to try and make things better?

  • smspillaz

    So sure, I understood that compiz was EOL by that point. Fair disclosure: i knew about the Mir project when I worked at Canonical, I knew that a Unity QML rewrite was probably going to be on the horizon (though no real decision was made by the time I left, but I’m sure my, and an number of other employees leaving certainly tipped things in that favor). I was also told (quietly) before anyone else that the plan was to drop Unity/Nux and Compiz at some point, but was not really told when.

    My main beef was that I was under the impression that:

    a) it was still going to be used for 13.04 b) a number of my performance-improving changes were going to be reviewed(!) [they were not even reviewed] and accepted into 13.04

    Then all of the sudden, two things happen at vUDS:

    1) The rolling release kerfuffle, and I get a huge amount of uncertainty from people internal at Canonical basically telling me that they won’t be able to accept patches because there won’t be a release 2) Canonical says they’re going to pull all resources off of both compiz and Unity/Nux, effectively stonewalling any pending contributions and development.

    Now the presiding view is that there will in fact, be a 13.04 release, and maybe we do want to accept these changes. But wait, its past feature freeze, so now its going to be impossible to get anything into the distro, except if you operate under the company name “Canonical Ltd.” at which point you can have yet another feature freeze exception put through because you have priority over everyone else.

    Its a real shame, because these improvements would have benefited a large cross section of users, and now Canonical is probably going to get slammed by Phoronix again in their 13.04 benchmarks because windowed OpenGL performance is still awful compared to everything else.

    Can you imagine how frustrating it is when you put 3 solid months of work into something only to see it go to waste because of mis-management of expectations?

    Can you imagine how much worse this would have been for those other contributors which were not even told that this was going to happen? I’m lucky, and I’m still annoyed and still don’t put a whole lot of trust in Canonical after that. For those people, it could have been so much worse.

    My point is this – if you want to engage the services of the community, that’s fine. But at that point, you are under a duty to act in good faith and to disclose to them your general plans and policy on how the community relationship will be handled. A breach of this duty is something that harms that relationship, and is understandably why a lot of people are very shy to contribute at this point. A guarantee that things like this won’t happen again through a sound and transparent process. And it is so easy – Canonical doesn’t need to give the community any control whatsoever. They just need to not mislead them and act in good faith. When decision making happens which might affect the community on a technical or project-management level, due consideration and a plan must be developed for how that relationship is going to be handled.

  • Matt Jones

    AFIK the “kerfuffle” was about people misunderstanding Rick Spencer’s rolling release proposal. Many people assumed that it was a set-in-stone plan, and not a proposal.

    This confusion happened from the announcement (mid February) till after UDS (March 6th). Feature freeze happened March 7th. So that means you either did not have your new Compiz code into 13.04 before mid February, or people were actively stopping you from adding it before rolling was proposed.

    This does not ad up. You couldn’t have gotten the 3 months of work done, and had your work prevented from entering 13.04 in that 2 week window before UDS.

    Did you find out about rolling before it was proposed to be at vUDS? On what date did you get prevented from adding your code into 13.04? And by whom?

  • smspillaz

    At the time, the impression I got from all of the developers I spoke to (I won’t name names) was that:

    a) There was not going to be a 13.04 release. b) Unity-Legacy was going to be replaced immediately by Unity-Next.

    Those impressions later turned out to be false (to the suprise of both myself and many other people within Canonical). The problem was this – that code had been ready for about 3 months and despite the fact that I engaged with several areas of product strategy and distro during those three months, resources were simply not allocated even when I was told that they were going to be (likely in anticipation that . Then all of the sudden, resources are not allocated and compiz is forked before feature freeze (code.launchpad.net/~compiz-team/compiz/raring) at which point I’m informed that my changes are actively not going to be considered or accepted because of “internal developments”. And then, a few weeks later a member of product-strategy tells me that those changes will be reviewed and considered.

    The point is this – stonewalling some of your main contributors over internal management or politics is an incredibly bad exercise of project management. One you are engaging volunteer contributors and have plans set in stone with them, you are under a duty to either ensure that they are clearly informed as to all relevant project management related matters (again, they do not need to have a role to play in the decision, they just need to be informed), and also to ensure that you stick to the plans and expectations set out in pre-release planning. Not adhering to these duties is a reason for volunteer contributors to lose their trust in the entity that they are putting in volunteer hours for, adhering to these duties is a way to gain volunteer trust.

  • Matt Jones

    That seems to clear everything up. I’ve also been talking with some people at Canonical. Many have said the same things.

    Some people decided they wanted rolling during the 13.04 cycle (in prep for 13.04+/phablet), and changed things that were already planned for 13.04. They expected there to be no 13.04. This meant a lot of work was wasted.

    Others said they knew about the rolling plan, but expected it to be later and not affect 13.04. They then found out that 13.04 was “canceled” around mid February with the virtual UDS announcement. And finally they were again caught off guard when it was decided to do 13.04.

    This sounds like a major communication screw up. It really sucks that you lost 3 months of hard work.

    So now we can understand that the problem was that the already established plans for 13.04 were changed secretly, and no one outside was told about it. Hopefully the things Jono presented above will alleviate problems like this in the future.

  • smspillaz

    Right,

    My suspicion is that the problem isn’t really with the canonical-community conversation, but that the problem is entirely internal to canonical and is just starting to manifest itself in its relationship with the community.

    To be fair, the review process for those changes has started again, and maybe it looks like they’ll go in this time.

  • Elfy Esquire

    Shame that canonical web design don’t understand community … http://design.canonical.com/2013/03/three-main-types-of-user-behaviour-found-in-ubuntu-com-testing/