The Genesis Of Free Software Projects

Regarding Mark’s post about participating in private projects, I just wanted to weigh in on a few things.

First, let’s take a look at how Ubuntu is developed today.

Like other Linux distributions, we take a range of upstream components and assemble them together into an integrated system. Throughout this integration work our community actively participates in a range of different areas – development, testing, bug-fixing, translations, documentation, and more.

Now, over the years Canonical has invested extensively in building components to help grow and improve the Ubuntu experience. Examples of this include Unity, Juju, Launchpad, Bazaar, Ubuntu One, and various other projects. The majority of these projects are fully open and anyone can participate in them.

In some of these projects Canonical prefer to keep some new features secret until they are released. Now, I know some of you have a fundamental ethical objection to this, and a subset of those folks deem this so reprehensible that it warrants an abandonment of Ubuntu.

Just don’t abandon Ubuntu gangnam style.

As part of my job, I have had consistent feedback from a small number of unhappy people about these private projects, and I have communicated these concerns to Canonical as appropriate. Now, like many of you, I always like to opt for an open by default approach to Free Software, but in some cases there is reasonable justification for keeping things secret until they are ready. Examples of such reasons can include keeping the development and design focused to get a first version together, having a big splash with the feature (and thus waiting for the splash), or if the feature is being created for a partner product and then later added to the main platform if appropriate.

Now, I can hear some of you balking at these reasons, and you have every right to disagree. I have personally always seen it this way though: when sometime decides to create Free Software either as an individual or as a company, they have the right to create the first iteration of that feature however they choose. Their investment of time, money, or both in building Free Software earns them a right to put together a first cut that meets their needs…this is the very nature of scratching an itch.

Taking the analogy too far…choose your own tree, or bear.

Contributing Earns Decisions

Let me give you an example. Some time ago I built the first cut of Ubuntu Accomplishments. I spent a few weeks hacking like mad getting the vision into a runnable form that I could then share with others. Now, as soon as I mentioned it publicly, some people were very complimentary of the work, and some decided to question every decision I made. Why did you use Ubuntu One as the back-end? Why are you not supporting Mozilla Open Badges? Why are you using GTK3 for the client? Why don’t accomplishments allow you to do XYZ?

The way I saw it was pretty simple. I took a few weeks out of my life to build Free Software that I would share with others, and as far as I was concerned, I had earned the right to make my own decisions about the project and how it would be constructed.

Importantly, this right does not extend to the full history of the project; the social expectations change when you release a project and want people to participate. Continuing my Ubuntu Accomplishments example, as soon as I had released my first cut and was asking people to participate, I could no longer just make unilateral decisions as I could when it was just me; for me to attract others to my project I needed to be collaborative.

Collaboration…gangnam style.

Shortly after I announced the project, Rafal Cieslak joined and started contributing code. Rafal and I had a series of discussions about the core system, discussing every decision I had made previously. I considered Rafal’s feedback on this strategic focus of the project as an equal to my own; Rafal deserved the right to have a say because he too had now taken time away from his family and friends to contribute to this Free Software project.

For new Free Software feature development, Canonical (or Red Hat, or Collabora, or Xamarin etc) has every right to keep things private to get the project in shape if they wish to. If the company is investing in hiring people to write a new piece of Free Software, they have earned the right to decide on that first cut. When the company then releases the project and if they want to have the community participate, the company then has a responsibility to be collaborative in their work with these active contributors.

Ubuntu Will Always Be Open

Getting back to Mark’s post, Ubuntu is not becoming less open; you can still participate in pretty much every part of Ubuntu if you choose to. The point of his post is that there are indeed some projects (some, certainly not all) in which Canonical is creating new features in which these projects are private, and we want to open up and include community participation in those projects. This is a step towards increased community participation, not less.

Now, again, I know some of you will be boiling in your boots about the fact that these projects are not open from the beginning. You are welcome to slam me in the comments if you choose to, but I don’t make the decisions on keeping those projects private, so you might as well save your fingers.

What I am going to be responsible for is helping to get these private projects in a position where community folks can participate, where participating is fun and enjoyable, and and helping to get the right community people connected to the right projects. As Mark mentioned in his post, Michael Hall (who is mhall119 on IRC) is going to be the point-man if you want to take part in these projects, and Michael sits on my team. We have already fleshed out an on-ramp for how community members can express interest in this and we will be following up more this week about how to get involved.

Thanks for reading!

  • dylan-m

    I’m all for teams starting projects privately (or, better, just quietly) and making the big release when they feel happy about it! I think it’s a great way to get something in motion, and people are really deluding themselves if they think that’s any different from how open source software usually happens. I also think it’s nice that you’re reaching outside Canonical here. Could be a great way to get what you’ve generally been doing while empowering the wider developer community. I hope it works well.

    Of course, I do have some concerns. I always do :)

    Does Canonical have any advantage doing this sort of thing to other developers? Specifically, if some community member outside Canonical decides to lead a big secret project and announce it a week before feature freeze, will they be able to have that feature shipping in 13.04? Will Canonical be able to? Who do they need to talk to first?

    As a GNOME Shell user, the Online Accounts thing really bothered me, and I think that one is an example of where this can go very wrong. A little while ago, we had just GNOME Online Accounts, which was developed quite openly. It was released and working happily in Ubuntu 12.04. In Quantal, Ubuntu Online Accounts appeared and it was as if GNOME Online Accounts never existed. Except it did, and it still does, which is why I keep needing to specify which one I’m talking about.

    So, we have these two completely redundant features, and it’s a mess because neither project was developed with the other in mind at all. GOA was released long before I recall UOA being talked about publicly, so I don’t think they deserve blame here; meanwhile, UOA looks to have ignored GOA for reasons I cannot begin to imagine. I’m sure smart people on either end will sort it out soon enough, but I don’t think anyone wants to see what happened with Online Accounts happen. It’s a huge waste of resources that could be better spent saving the world (and reaching 200M users).

    I think the UOA example demonstrates that there can be serious scale issues with secret projects and Ubuntu. After all, an operating system is a big thing, and Canonical and Ubuntu aren’t the only pieces that need to fit together here: GNOME, Debian, Firefox and LibreOffice are all huge, fast-moving projects and Ubuntu depends on them. It’s probably easy enough to guide plans in Ubuntu around a secret project (eg: GOA panel hidden in Unity), but it’s a lot harder when you’re silently competing with upstreams.

    Do you know if these projects you’re talking about will generally be small, detached, honestly Ubuntu-specific projects, or should we be concerned about a GOA/UOA repeat?

  • Jef Spaleta

    Or put it another way…. projects crafted in private.. still need public review prior to integration at the distribution level to drive feedback into the development process ahead of committing to integration.

    Build any of these skunkwork items as a distribution agnostic projects. Reveal it as an early adopter consumable ppa and compilable source tarball. Then, and only then, propose it for inclusion at the distribution level just as any other feature blueprint. Gather feedback from the blueprinting process and make adjustments.

    Don’t short-circuit that feature inclusion process by allowing Ubuntu the project commit to private deliverables before there is public discussion.

    It’s okay to have private development, just don’t short-circuit the integration feedback when its time to actually integrate the project into the Ubuntu releases. Let these projects live and breath as upstream projects not tied or committed to inclusion into an Ubuntu release until their is the public discussion. Commit to making sure all the skunkworks projects go through that public discussion. Don’t let them be exceptions to the release policies.


  • Chad McCullough

    Thanks for this, Jono. I will admit that I am one of those that got “bent out of shape” after reading an article by an online technical magazine. I’m sure you all know the one I’m talking about. Shame on me for letting an article get me so upset. I should know by now that not every writer is fair or does his or her homework.


  • Matt Fischer

    And that’s exactly what you’ll be getting at UDS.

  • Jef Spaleta

    i look forward to the blueprints getting filed in launchpad… just like we saw happen for the “interesting” features for 12.10.. oh wait..nevermind.

  • Jeremy Bicha

    There’s almost nothing stopping any Ubuntu developer with MOTU or Core upload privileges from uploading any new feature they like at any time before Feature Freeze. If it involves a new binary package, the Release Team does need to manually approve that in the queue.

    Canonical does have a huge advantage to being able to land things post-Feature Freeze which was pretty clear during the Quantal cycle. By being so aggressive this cycle, they’ve drawn more attention to that issue and we may very well see more resistance to those attempts in future releases (I expect user interface freeze in particular to be more respected).

  • Henrik Ingo

    Fact: Linus Torvalds worked on Linux in private for 12 months before releasing the first version in public.

    As long as the end result is under an open source license, it’s open source. I personally encourage FOSS actors to try different methods of getting to that point. We need this to find out what is the most efficient way to develop and distribute free and open source software to the world.

    For instance, it’s a fact that Android won over MeeGo. What can you do.

  • Martin Owens

    I agree Jono that this is a good move forwards.

    I do share the concerns with Canonical’s special ability to brute force features into the distro before they’ve done through a bit of tempering in the cold hard glare of community peer review.

    Consider the Online Accounts, I’m currently trying to write a new provider to allow deviantArt integration into the desktop. This is made almost impossible by lack of guides, help files, or even README files in the code repositories. It’s an oversight because if the project had been mauled a little before release, it would have been more rounded in collaboration aspects.

    Consider code that’s made open source after made years of being a private codebase. Usually the code is a complete dogs breakfast and it takes years of developers hacking with it to get it into some sort of order. Canonical’s private code bases won’t ever be quite that bad, but it’s something that community peer review can really help with.

    Regarding Online Accounts, I’m going to have to wait for the three Canonical guys to get online on Monday to ask them about how it should be hacked on.

  • Marcus

    It seems that the images are Copyrighted (I guess you have asked the original author if you are allowed to use them). Anyhow, I would expect that one who is promoting free software is using Images that are available under a free license.

  • Anonymous

    I suggested that you take a look at Open Badges; I hope you didn’t think I was “questioning every decision you made”. I completely agree that if you are writing software, you get to decide how it works.

  • Adam Williamson

    Bit late, but this was a thoughtful and well-reasoned post, and broadly I agree with everything you said, thanks Jono. It would be nice if Mark was equally thoughtful and less confrontational in how he put stuff. If you could ask him to stop bashing other F/OSS companies unnecessarily in discussing decisions like this, it would be nice. I thought that’s what we’d agreed on in the past.

  • Anonymous

    Gev, that wasn’t aimed at you – many people mentioned the Open Badges thing. :-)