Thinking about GNOME 3.0

As many of you will know, I am quite the usability pervert. Understanding how people use computers and creating better and more intuitive interfaces fires me up, and the mere idea of GNOME 3.0 is interesting to me. The reason why I find GNOME 3.0 exciting is that it presents a dream for us; we are not entirely sure where we are going, but we know it needs to be different, intuitive and better for our users.

While at GUADEC I sat down for a while with MacSlow, and he gave me a demo of Lowfat. For quite some time I have had some vague ideas for the approximate direction of GNOME 3.0, and some of Mirco’s work triggered some of these latent concepts and mental scribblings. I am still not 100% sure where I would like to see GNOME 3.0 go, but some of the fundamental concepts are solidifying, and with my recent addition to the mighty Planet GNOME, I figured I should share some ideas and hopefully cause some useful discussion. I am going to wander through a few different concepts and discuss how we should make use of them in GNOME 3.0, culminating in some ideas and food for thought.

A more organic desktop

One of the problems with the current GNOME is that it largely ignores true spatial interaction. Sure, we have spatial nautilus (which is switched off in many distros), but spatial interaction goes much further. If you look at many desktop users across all platforms, the actual desktop itself serves as a ground for immediate thinking and ad-hoc planning in many different ways:

  • Immediate Document Handling – the desktop acts as an initial ground for dealing with documents. Items are downloaded to the desktop and poked at before entering more important parts of the computer such as the all-important organised filesystem.
  • Grouping – users use the desktop to group related things together. This is pure and simple pile theory. The idea is that people naturally group related things together into piles. Look at your desk right now – I bet you have things grouped together in related piles and collections. We don’t maximise this natural human construct on our desktop. More on this later.
  • Deferred Choices – the desktop serves as a means to defer choices. This is when the user does not want to immediately tend to a task or needs to attend to the task later. An example of this is if you need to remember to take a DVD to work the next day – you typically leave it next to the front door or with your car keys. The analogy with the current desktop is that you would set an alarm in a special reminder tool. More people set ad-hoc reminders than alarms.
  • Midstream Changes – it is common for users to begin doing a task, and then get distracted or start doing something else. An example of this is if you start making a website. You may make the initial design, and then need to create graphics and get distracted playing with different things in Inkscape. The desktop often acts as a midstream dumping ground for these things. Work in progress in documents are often placed on the desktop, and this acts as a reminder to pick it up later (see deferred choices earlier).

It is evident that the desktop is an area that provides a lot of utility, and this utility maps to organic human interaction – collecting things together, making piles, creating collections, setting informal reminders, grouping related things. These are all operations on things, and are the same kind of operations we do in our normal lives.

Part of the problem with our current desktop is that there is a dividing line between things on the desktop and things elsewhere. It is a mixed maze of meta-data, and inter-connected entities that should be part of the desktop itself. As an example, when I was writing my book, I created word processor documents, kept notes about the book in TomBoy, saved bookmarks in Firefox and kept communications in GMail and Gaim. The singular effort of writing a book involved each of these disparate unconnected resources storing different elements of my project. I would instead like to see these things much more integrated into (a) contextual projects, and (b) manageable at a desktop level. More on contextual projects later.

Blurring the line between files, functions and applications

The problem we have right now is that the desktop is just not as integrated as it could be. If you want to manage files, you do it in a file manager, if you want to do something with those files you do it in an application, if you want to collect together files into a unit, you use an archive manager. Much of this can be done on the desktop itself, but we need to identify use cases and approach the problem from a document-type level.

Let me give you an example. A common type of media are pictures, photographs and other images. The different things you may want to do with those images include:

  • Open them
  • Edit them
  • Compare them
  • Collect them together by some form of relevance (such as photos from a trip, or pictures of mum and dad)
  • Search for them

These tasks involve a combination of file management, photo editing applications, photo viewing applications and desktop search. Imagine this use case instead:

I want to look through my photos. To do this I jump to the ‘photo collection’ part of my desktop (no directories) and my collection has different piles of photos. I can then double-click on a pile and open up in front of me. Each photo can be picked up and moved around in a physically similar way to a normal desk (this is inspired from MacSlow’s LowFat). I can also spin photos over and write notes and details on the back of them. Using my photos I can put two side by side and increase the size to compare them, or select a number of photos and make them into a pile. This pile can then be transported around as a unit, and maybe flicked through like a photo album. All of this functionality is occurring at the desktop level – I never double-click a photo to load it into a photo viewer, I just make the photo bigger to look at it. All of the manipulation and addition of meta-data (by writing on the back of the photo) is within the realm of real world object manipulation, and obeying pile theory and spatial orientation.

The point here is that the objects on the desktop (which are currently thought of as icons in today’s desktop) are actual real world objects that can be interacted with in a way that is relevant to their type. In the above use case you can make the items bigger to view them, compare them side by side, and scribble notes on them. These are unique to certain documents and not others. You would not zoom, compare and scribble notes on audio for example, but you would certainly use pile theory on audio to collect related audio together (such as albums).


So, if we are trying to keep interaction with objects at the desktop level, how exactly do we edit them and create new content? How do today’s applications fit into this picture? Well, let me explain…

The problem with many applications is that they provide an unorganised collections of modal tools that are not related to context in any way. I have been thinking about this a lot recently with regards to Jokosher, and this was discussed in my talk at GUADEC. Take for example, Steinberg’s Cubase:

In Cubase, if you want to perform an operation, you need to enable a tool, perform the operation, disable the tool and then do something else. There is a lot of tool switching going on, and toolbar icons are always displayed, often in situations when that tool can either not be used or just would not make sense to be used. The problem is that it obeys the philosophy of always show lots of tools onscreen as it makes the app look more professional. Sure, it may look professional, but it has a detrimental impact on usability.

I believe that tools should only ever be displayed when pertinent to the context. As an example, in Jokosher we have a bunch of waveforms:

The first point here is that we don’t display the typical waveforms you see in other applications. Waveforms are usually used to indicate the level in a piece of audio, and as such, we figured that musicians just want to see essentially a line graph, instead of the spiky waveform in most applications. This immediately cuts down the amount of irrelevant information on screen. Now, if you select a portion of the wave in Jokosher, a tray drops down:

(well, at the moment, it drops down, but does not visually look like a tray, so run with me on this for a bit!)

Inside this tray are buttons for tools that are relevant to that specific selection. Here we are only ever displaying the tools that are pertinent to the context, and this has a few different benefits:

  • We don’t bombard our users with endless toolbars
  • Tools are always contextual, which makes the interface more discoverable and intuitive
  • We restrict the potential of error by restricting the number of tools available for a given context
  • There are fewer buttons to accidentally click on, and this lowers the hamfistability of our desktop 😛

Now, take this theory of contextual tools, and apply it at the desktop level. Using our example from earlier with photos, I would like to see contextual tools appear when you view a photo at a particular size. So, as an example, if I have my collection of photos and I increase the size of a photo to look at it, I would like to see some context toolbars float up when I hover my mouse over the photo to allow me to make selections. When I have made a selection I should then see more tools appear. There are two important points here:

  • Firstly, you don’t load the photo into an application. As you view the photo at the desktop level, the functionality associated with an editing application seamlessly appears as contextual tools. This banishes the concept of applications. Instead, you deal with things, and interact with those things immediately.
  • Secondly, tools are always contextual, and relevant to the media type. For a photo and a document, selections make sense, for an audio file, you should be able to apply effects and adjust the volume, for a text editor you should be able to change font properties. Everything is relevant to the context.

The contextual desktop at a project level

To really make the project feel contextual, we need to be able to make it sensitive to projects and tasks. At the moment, tasks and sub-tasks are dealt with on a per-task basis as opposed to being part of a bigger, grander picture. Let me give you a use case with today’s desktop:

I am working on a project with a client to build a website for him. I decide I want to send some emails to him, so I fire up my email client and dig through my inbox to find the mail he sent me yesterday. I then reply to him. As I work, I realise that I need to speak to him urgently, so I log on to IM. Within my buddy list I see that he is there, so I have a chat with him. While chatting, my friend pops up to discuss the most recent Overkill album. As I am working, I don’t really want to talk about the album right now, so I either ignore him or make my excuses. After finishing the discussion with the client, I load up Firefox and hunt through my bookmarks to find a relevant page and start merging the content into the customers site. To do this I load up Bluefish and look through the filesystem to find my work files and begin the job.

The problem here is that the relevant work is buried deep in other irrelevant items. To make matters worse, some resources such as IM can just prove too distracting, and may never get used (remember midstream changes earlier). As such, the really valuable medium of IM is never used for fear of distraction. Now imagine the use case as this:

I am working on a project with a client to build a website for him. I find my collection of projects and enable it, and my entire desktop switches context to that specific project. Irrelevant applications such as games are hidden, and relevant resources are prioritised. When I communicate with the client, only emails and buddies relevant to that project are displayed. When I want to find resources (such as documents) to work on, only those documents that are part of the project are displayed. The entire desktop switches to become aligned to my current working project. This makes me less distracted, more focused and there is less clutter to trip over.

I actually had this idea a while back, and wrote an article describing it in more detail. Feel free to have a read of it.


The point of this blog post is not to sell you these concepts, but to identify some better ways of working which are more intuitive and more discoverable. Importantly, we need to make our desktop feel familiar. This was a point Jef Raskin made as part of his work, and I agree. Some people have been proposing some pretty wacky ideas for GNOME 3.0, but grandiose UI statements mean nothing unless they feel familiar and intuitive. What I am proposing is an implementation of real world context, relevance and physics into our desktop. This will make it more intuitive, less cluttered, less distracting and a better user experience.

I really want to encourage some genuine discussion around this, so feel free to add comments to this blog post, or reply via Planet GNOME. Have fun!

  • Tom


    I wholeheartedly agree with your points about projects. I wrote something similar last year about my frustrations when writing papers and articles.

    I’m not sure I’d want my “entire desktop” to switch context, partly because I often like to keep more than one “project” going at the same time. But we definitely need our computers to make sense of our work patterns rather than forcing us to work around the limitations of software.

    I’d also steer clear of relating the computer too closely to physical examples. Ted Nelson’s rants on the office-duplication are really interesting in this respect, as are Aaron Seigo’s attempts to drop the entire desktop-as-an-icon-dump idea altogether.

  • Phil Bull

    Interesting read, thanks! Just one small point.

    “I believe that tools should only ever be displayed when pertinent to the context.”

    Say a user is looking for a particular tool. That tool happens to be ‘irrelevant’ to the current selection (and so it’s not displayed, because you only show relevant/enabled tools). The user keeps on looking for the tool (wasting lots of time), and gets frustrated. By not displaying irrelevant tools somewhere you lose some important feedback – how can you tell the user that they can’t perform a particular operation on the current object/selection/etc? This, I believe, is the logic behind not hiding toolbar buttons, and instead disabling them when they don’t apply.

    Of course, there are loads of ways around this… it’s just that you’d have to make sure that you categorise ‘relevant’ and ‘irrelevant’ very carefully when you design the interface to avoid frustration or excessive clicking. For example, going back to:

    “You would not zoom, compare and scribble notes on audio for example”

    Well, maybe a minority of people would scribble notes on audio…


  • Raphaël Quinet

    Very good suggestions, but you will have to pay attention to the discoverability of some tools/actions.

    Some actions may not be available unless the document/object/whatever has some properties or is in a specific mode. Some inexperienced users may be frustrated if they want to perform some action that is not available in that context and they have no idea how to change the object in such a way that the operation becomes possible.

  • jono

    Phil / Raphaël- the solution to the problem of non-discoverable contextual tools is not to make them non-contextual, but to improve feedback in other ways. As an example, in Jokosher we constantly show instructions in the status bar, a technique inspired by Inkscape.

  • Anthony C.

    A contextual desktop seems nice but also seems hard to implement. Basicly you’ll get a desktop that knows what you want to do. It’s great in theory but i don’t think it’s really possible. (please prove me wrong :wink:)

  • stuhood

    Jono, you bring up some excellent points. I read the O’Reilly article as well, but I think the project management tool you suggested might end up making too many assumptions about what the user wants to do. Sometimes people DO want to multitask, and only being able to switch back and forth between projects one at a time might be rough.

    What immediately came to mind when I read this though, was the concept of Tags. Better integration of tags into the desktop (tagging people, tagging all documents, tagging websites) could have the same kind of effect. Instead of the ‘project tool’ (“In the top right hand side of his GNOME desktop he clicks on the project icon and a drop down list of his current projects are displayed”), how about a widget like the one in F-Spot that can show/hide documents, people, EVERYTHING… based on tags.

    Of course, we don’t want to make the task of tagging new ‘objects’ cumbersome for the user, so some kind of algorithim to guess tags for the user, based on what other objects they have open (and maybe integrated into the file-save dialog / a hover whenever you drag to create something) could be very useful as well.

    I know that David Trowbridge ( has been busy with tags lately, and has even begun work at integrating them into Gimmie ( Maybe this would be a good first step?

  • Ole Laursen

    Interesting thoughts. I think the idea of being able to organise stuff by projects or contexts is great, as long as you can still do nothing in particular. I hope you figure out how to do this more concretely. Maybe have the concept integrated so you could click “Start a new project” and have the desktop show a new workspace for you and setup any applications you start so that they know that they are dealing with a particular project. The state should then be preserved and be mobile.

    But like Tom I don’t think taking physical metafors too far is a good idea. If you emulate a real desktop down to being able to turn photos around and use the backside, then what have you gained from using a computer except for a lousy mouse interface?

    If I annotate a photo, I most likely want to be able to look at the notes while I’m looking at the photo itself. The computer can do this easily for me. Not so in the real world where I would have to store a separate piece of paper.

    I think it is ironic that some people are actually working on implementing piles. If there’s something I hate, it is having to shuffle piles around and clean them up because there’s not enough room on my desktop. :-)

  • Hender

    I think you’ve got a great thing going on here with your idea of a context-sensitive desktop!

    A Dashboard-like app should ideally be given prominence in this task-driven future, while trying to make the more advanced tools on the desktop equally streamlined. For instance, being able to enter who my client/boss is and Dashboard presenting the option to IM or E-mail that person with important information … without necessarily firing up all of the bulky and intrusive Gaim or Evolution and distracting me. This could potentially turn into a vast performance win as well, as for example loading a conversation window in Gaim is less resource-intensive than loading the contact list as well.

    The catch would be that all apps need to expose very basic interfaces for task-specific access as well as the alternative, full-featured (non-task-specific) app in order to actually be useful to this vision.

  • Jo Vermeulen

    Some good ideas, considering organizing music, here is a nice app for inspiration:

    Furthermore, a lot of what you’re saying comes down to context-awareness. A nice article on context awareness in general was written by Henry Lieberman and Ted Selker:

  • Jo Vermeulen

    Just another small comment, I guess you based your idea of making piles of stuff and using physics to do so on BumpTop (which MacSlow also mentioned as one of his main competitors)?

  • Brent

    I like the idea a lot. I have posted about similar ideas, basically making projects as seperate desktops. This would make a project similar to a viewport except that everything in this desktop would be localized, and all relevant information would be contained in it. When the project is completed, the whole project’s desktop will be archived.

    This can also define collaborative work, as people on the same project can work with the same files and are easily contacted. Management can be implemented by including subprojects; these would define the information available at the subproject level (not everyone needs access to all data) and the project progress updated when each subproject is completed. This would require a wizard for the creation of a project, which would described the resources required, people involved, and subprojects, then keep track of the progress.

  • nobody

    interesting discussion. I think it’s worth reviewing the history of Gnome over the last few years, however. The jump in responsiveness in the desktop and speeding up of development have coincided with breaking down monolithic apps into smaller pieces that do specific jobs.

    Nautilus, Mozilla suite, Star Office suite, etc. are now smaller, more specific versions hardly recognizable by their monolithic forebears. Grand visions too often result in bloated jack-of-all-trades-master-of-none software.

    I would be interested to read a discussion of your vision in a way that moves forward from where we are rather than back to the state where nautilus showed pix, played sound and movies, and vacuumed your carpet.

  • jono

    Thanks for the great discussion folks – keep it coming! :)

    stuhood – the problem with tags is that they are basically meta-data, and you need to think in tags to make them useful. In the context desktop you would need to manually specify which resources are associated with the project, but this could be tags under the covers. I have lots of ideas of how this could be eased and automated.

    ole – I am not saying the project should only work in project mode, but be able to work in that mode. As for physical metaphors, I don’t see anything wrong with modelling the real world so long as it makes sense. Each of the use needs to be realistically considered. The example of scribbling notes on the back of a piece of paper was just an example to illustrate the potential of thinking of items as real world items.

    Brent – collaboration is where it really gets exciting. When the desktop is hot-rodded to broker connections with related projects and workers at the desktop level, things really get exciting. My mind is so full of examples and cases where a collaborative, contextual desktop would be useful.

    nobody – if there was some interest within the GNOME community to implement some of these ideas, then I would be more than happy to work on some designs for a migration step. I think we need to head towards the contextual desktop in an iterative way to make it work, and I think we can. In terms of the ideas I mentioned of real-world items and a more expansive desktop, that would require some fundamental changes, but Lowfat is heading in the right

  • Gergely Nagy

    Hi folks, excellent thoughts. I’m very excited about getting rid of the burdens of today’s computer desktops and finally make the machine work for us instead of the other way around.

    Seems to me we are getting lost in the details while still discussing a rather high level view of a future interfarce. Things like tags are just an implementation detail (they define relationships between your files).

    Now ideas like focusing on the data instead of the tools (i.e. docs instead of apps), the contextual workspaces, more natural grouping and handling (may they be stacks or whatever) are the real point here.

    Instead of having isolated silos of data managed by individual applications, we need some kind of a common data platform. Something more high level than files (byte streams). Also, something distributed. Nobody works exclusively with local data anymore.

    A common distributed data platform would enable us to blur the lines between applications, push them back as second-class desktop citizens, and make the novel higher-level, data centric user interaction possible.

  • behavedave

    Wow you have quite a following now. These are some extremely good idea’s however I’d just like to extend the idea from documents are more important than applications to documents & contacts are the important parts of the desktop. I really do like the ‘applications should be interacted with and not dominate the task’ feel. I wonder if you’l influence the KDE guys as they appear to be implementing right now and Aron has already suggested a organic interface is what is being aimed towards at ‘KDE four Core’. Infact ask him for a Lugradio update seeing as the Core meetings were so important.

  • nobody

    If you’re proposing a document-centric view, I would like to suggest something too: build it on top of a (personal, distributed, fast) revision control system like git. This would allow people not to worry about “previewing” changes they’re about to do, and wouldn’t interrupt people with confirmation dialog boxes or anything like that. Backing up, saving state, collaborating with other people would all become easier. And it would give a lot of history information that can be harvested for search (a la beagle) or related systems.

  • nobody

    Another remark on the previous comment: if you ever tried to keep your home dir in sync on multiple machines (laptop, desktop, server, PDA, mobile phone, …) through unison, rsync, cvs/svn, nfs or even coda/intermezzo – you’re already aware that it can be a real pain. If the Desktop was built on something like git, dealing with documents spread over different systems would become a lot easier.

  • jono

    Gergely – exactly, I think its important that in the initial stages we define how contextual linkage should occur, and there needs to be a (dare I say it) framework in which applications can hook together. I could imagine something akin to HAL which can report capabilities to different applications. This framework would indeed need to be network aware, and could bring in all the potential of real networked collaboration.

    Everyone – I am pleased that there is some discussion going on, and this thread is bringing some great ideas up. It would be great if we can solidify these ideas into something implementable. 😀

  • Greg

    To me, it sounds like you’re describing two related things: a new systemwide metaphor for organizing information (an alternative to the hierarchical filesystem), and more document centric, contextual interfaces. They’re related in that applications can certainly utilize the alternative filesystem metaphor, but it extends beyond individual applications.

    The first part sounds similar to the general shift away from organizing information in trees and towards organizing it in sets that’s been seen in many contexts lately. I love this, because I think human brains tend to organize information in sets, making these kinds of interfaces much more natural than the hierarchical filesystems that have been ingrained in computer science since at least 1950. This shift probably started with rise of the search engine concept, which, at a fundamental level, lets people find stuff by taking the intersection of various sets (along with ranking the results). More lately, its been evident in the tagging craze and the idea of virtual folders that applications like evolution and liferea (both are just sets by another name). Moving to a completely object based or relational filesystem is probably beyond our capabilities for now, simply because of inertia, but providing a set api on top of it, as you and these other efforts are trying to do, is one way of alleviating some of the computer/brain impedance mismatch. Anyway, what I guess I’m saying is that your contextual projects sound like sets too, and could probably be representing by a tagging system, although one in which tags aren’t necessarily applied by the user. The leaftag project is attempting to add something like this to gnome, although I worry that its isn’t deep enough into the gnome infrastructure (being made part of gnome-vfs might be preferrable) and might not be powerful enough for some of the things, like this, that people will want to use it for (a namespace system for tags would help).

  • Greg

    Oops, wrote that without having read any of the comments. I completely agree that “tags” are an implementation detail… that’s kinda why I use the more academic term “sets”. It should be a developer api, not necessarily something the user sees.

  • Erich Schubert

    Hi, Another “natural” way of organizing things is that we tend to put important stuff in the front, and less important stuff somewhere to the back. Many people are cluttering their desktop (at least on windows), and then will need to clean up every now and then. They usually have two things on their desktop (apart from cruft like automatically generated icons): – frequently accessed application icons – “fresh” files they just dropped on the desktop and not in an appropriate folder yet.

    You “piling” aims at that a bit, but I think we could do better. Give the user a “natural” way of making stuff more or less important. Use depth.

    What I’m basically thinking of is that icons will become smaller when they are placed at the upper end of the screen, and larger when they are placed “in front”. I guess that can look awesome with lowfat, too. 😉

    But it’s not just about graphics. Think of how people use piles on their desktop – the piles grow and grow, and the stuff lower in the pile gets obscured, until they actually clean up. If I had a “current documents” pile on my desktop – I have disabled all icons, since I run my windows full screen anyway, so I never would see them anyway – I would want it to behave similarly, that basically by dropping more and more stuff “atop” of each other, some stuff will go into the background.

    (It might be useful to have different views on your desktop then – including a zoom in function and a birds eye, for cleaning up, or finding “covered” files)

  • pollycoke :) » Pensierini post-GUADEC

    […] Jono Bacon on contextual user interfaces. […]

  • jonobacon@home » Busy days

    […] Thinking more and more about the contextual desktop. The blog post I wrote garnered some really awesome comments and some interesting ideas. When I get a moment for my head to breath I am going to think about it in some more detail. […]

  • Lauro Moura » Desktop contextual

    […] Nesse post, Jono Bacon (um dos autores do Jokosher) comenta sobre um possível GNOME 3.0. Segundo suas idéias, o ideal seria algo mais próximo de como trabalhamos no dia a dia, abstraindo o conceito (separado) de aplicação. Ao invés de aplicações propriamente ditas, elas apenas disponibilizariam serviços que seriam exibidos de acordo com a necessidade, sem necessidade de uma “janela” própria para o programa. Dessa forma, a diferença para o usuário entre aplicações locais e aplicações remotas poderia estar ainda menor. […]

  • GrumZ

    Like lots of people here, I love these ideas. A desktop where your workflow will be smooth and not always disturbed by anything would be really really great.

    Maybe, you will be interresting by the Atomato project, initiated by Rodrigo Moya, which could be the first steps towards this ideal desktop. The first goal of it is to “allow non programmers to build their own scripts for automating tasks without having to write a single line of code, much like Apple’s Automator.” This will be done by letting application export their functionnalities throught DBUS functions.

    But if we push further this project idea we can imagine that each application could be only a set of exported function that can be registered to the desktop. So when you click on your photo pile, it will display all the application’s functions that have registered for photo related tasks.

    More details here :

  • Nathaniel

    Piling’s obviously a big area of research in HCI, but your discussion of the photo management stuff made me in particular think of Dan Bauer’s recent disseration. It comes out of both the cognitive ethnography and zoomable-user-interface traditions; he built a photo/document sorting tool, with piles, zooming for inspection, various bits of physics (lens for sorting, etc.), and then analyzed how it worked in practice, how tiny design choices affected usage, etc. I don’t think the diss is online, but you could probably get a copy by emailing him at Might be a useful source of food for thought…

  • Alex

    Hello, I really like the idea of sets and that etherogenous applications objects must be groupable and movable. I tend to think an implementation for that is tag for grouping but also bookmarks as symbolink links to objects. Via dbus you could see applications which offers functions for an object (open mostly, but also offers icon …). for example email://mailboxlocation#number could represent an email, body://adress a buddy etc… OS as always seen as responsible for manipulating files, I think that today it should handle objects (although not directly but offering common api to programs via dbus).

  • GrumZ

    Did you see this post about a research project for this kind of metaphor : BumpTop :

    It looks really great ! Watch the video, it is amazing !