Category Archives: February 2010

Writing Multimedia Applications with Vala

Jim Nelson describes using Vala to write multimedia applications for the Linux desktop.

 

Introduction

A user recently notified me about a build problem he encountered with a multimedia application I’ve been developing in Vala, a new object-oriented language designed for GNOME.

When I suggested he upgrade to the latest version of the Vala compiler, his response was “Why on earth don’t you use a standard language?”

This is my attempt to answer his question.

Vala

The story of Vala begins with Jürg Billeter. He noted that writing and refactoring GObject-based code in C required a lot of extra work that more modern languages automated. He also noted that these modern languages do not produce libraries that can be easily used in other environments. He wanted a full-featured object-oriented language that could produce C-style interfaces automatically. (See the Gee collection class for a perfect example of Vala solving this problem.) He mentioned his idea to fellow student Raffaele Sandrini, who jumped in to help build the early compiler.

In May 2006, valac became completely self-hosted. Work continues on it today.

Building a Better Multimedia App

Vala first came to my attention in January 2009, when I joined the Yorba Foundation. Yorba was founded that same month by Adam Dingle. His goal was to make high-quality open-source multimedia applications for GNOME. His whiteboard list of applications to develop included a multitrack audio recorder, a video editor, and a photo organizer. Yorba’s challenge is to craft open-source multimedia applications as fluid and friendly as the best-of-breed in the commercial world.

When I joined, our first task was to select a programming language. Adam and I both had plenty of experience with C, C++, and Java. Adam had worked with C# as well. In considering what language would best fit Yorba’s needs, we devised these criteria:

  • A modern object-oriented language. We both admired Java and C#’s interpretation of the object-oriented paradigm. We were less enthusiastic about developing desktop multimedia applications in C++ due to its quirks, many due to its C legacy.
  • Strict performance and memory-usage characteristics. Thinking about real-time applications like audio recording and video editing, we had concerns about background garbage collection pauses and virtual machine/script interpreter performance. With an application like a photo organizer, which might require tens of thousands of object instances to represent a large collection, we wanted fine-tuned control over heap traffic and allocation sizes.
  • Language support for GObject and GType. Since we planned on targeting the GNOME desktop, we wanted full access to all of Gtk’s features, including signals, exceptions, threads, and reference counting. If you look at any Gtk project written in C, it’s apparent the price for these features is a lot of boilerplate and bookkeeping. We hoped to minimize this work or avoid it entirely.
  • Strong type checking. Adam and I agreed at the outset: We’re not fans of weakly-typed languages. Too often we’d been burned on code errors that could have been caught by strict parsing. Forgiving languages have their place in the world, but rock-solid desktop applications may not be one of them.
  • Broad and mature class libraries. In addition to the sort of libraries most modern languages provide—collections, threads, asynchronous I/O and streams—any multimedia desktop application will rely on the hard work of many other projects in the open-source community, such as GStreamer, gPhoto, SQLite, Pango, Cairo, and more. We wanted that code to be transparently available, either through bindings or native support.

We found that Vala meets all these requirements, and more.

Introducing Vala

Vala is described on its home page as a “compiler for the GObject type system”. This description is accurate but unappetizing. To put it another way: Imagine a language that offers GObject’s inheritance and signaling, GType’s run-time type checking, and GLib’s exceptions and thread support, and makes them available as native language features. That language is Vala.

It’s beyond the scope of this article to detail Vala in its entirety. The executive summary is, if you know C# or Mono you know 95% of Vala’s syntax; if you know Java, you’re 90% there.

Build and run-time environment

Unlike Java/C#/Mono, Vala programs build to a native executable. There is no virtual machine or bytecode interpreter. The current Vala compiler does this by generating C code and running it through the machine’s C compiler (typically gcc). In addition to translating the application’s logic to C, valac produces the GObject boilerplate a GNOME C programmer would have to crank out. It also automates the reference-count juggling normally done by hand.

However, Vala is not merely a fancy C preprocessor. In the future, valac may become a direct compiler or a gcc back-end. (This is similar to the historical development path of C++.)

As mentioned, we had concerns about garbage collection in our applications. GObject’s solution is to use reference-counting. The deallocation happens as soon as the last reference is dropped; there is no background thread interrupting the application. This means we have been able to characterize our code’s performance knowing it operates in a deterministic way.

Reference counting does come with a price, however. Most notable is the problem of circular references (where two objects hold references to each other and their reference counts never drop to zero). Vala data structures should be designed to avoid this situation. If such references are inevitable, Vala offers the weak keyword as a way to break this loop.

Class libraries

Vala is distributed with nearly one hundred bindings for various libraries, including Cairo, GStreamer, HAL, libglade, WebKit, and of course Gtk and GLib. Other bindings are always under development.

Bindings are located in VAPI files, which are Vala code with embedded directives that enable valac to generate proper C code for any library out there. (Writing one is much like writing an interface.) Additionally, if the library is based on GObject (such as gstreamer), a tool distributed with Vala called vapigen can use GObject introspection to discover the library’s interface and automatically produce a VAPI file.

Benefits & Pitfalls

Yorba started Shotwell, a digital photo organizer, in March of 2009. We’ve made great progress on Lombard, a video editor, and Fillmore, an audio recorder. We’ve also built a gedit plug-in, Valencia, which offers Vala code completion, method signature lookup, and jump to symbol. All of these applications are coded in 100% Vala.

Our experience with Vala has been largely positive. As the syntax is so familiar, new programmers can jump in and make contributions quickly. Our summer intern committed his first substantial patch his second week—in a language he’d never heard of before arriving. Standard development tools (such as gdb and sysprof) work without modification. We also coded up bindings for libgphoto2 and libexif in a short amount of time. We plan on submitting these to the Vala group for inclusion in the distribution.

The pitfalls we’ve faced all deal with the newness of the Vala project. The compiler is still in development and there are some bugs we’ve had to work around. There have also been some slight shifts in the language syntax, but the dust is settling as the language matures. There are also problems with the generated C code causing gcc warnings. (These usually deal with const issues in C.) Jürg is aware of these issues and is working to correct them.

Another problem has been the bindings themselves—when they’re incorrect, valac will produce incorrect C code, causing bugs or compilation failures. These issues are being fixed as they’re discovered, but they do crop up.

Still, the productivity gains we’ve realized overshadow these shortcomings. Development and debugging speed is akin to other modern languages.

Vala has proven to be a great choice for building GNOME multimedia applications.

Getting Started with Vala

To check out Vala, your first stop should be the Vala home page at gnome.org. Check out the tutorial and sample code to get you started. The FAQ and syntax pages are important reads to learn some of Vala’s fine details. Valadoc is a vital Web-based reference guide to bindings distributed with Vala. And, of course, subscribe to the mailing list to keep current.

About the Author

Jim Nelson works at Yorba developing multimedia applications for the Linux desktop.

Discuss this story with other readers on the GNOME forums.

What’s New with Banshee

The Banshee Project released 1.5.3 on January 27th. This release brings a slew of improvements to the already impressive media player that gives iTunes a run for its money, if not hype.

The Banshee Project released 1.5.3 on January 27th. This release brings a slew of improvements to the already impressive media player that gives iTunes a run for its money, if not hype.

What is Banshee?

For those not familiar with Banshee, it’s a media player written in C# on the Mono platform, started by Aaron Bockover. Since its debut in 2005, Banshee has seen contributions from nearly 100 developers. Banshee is Free Software, licensed under the MIT/X11 license, and has been packaged for all major Linux distributions as well as recent releases for Mac OS X. The most recent Mac OS X release removes the requirement for the Mono and GTK frameworks, bundling everything in one binary for installation on Mac OS X. Note that Mac builds are beta quality and may be missing some features present in Linux releases, or may not be quite as stable.

Initially designed for music, Banshee took on video support with the 1.0 release. Banshee uses GStreamer as its backend, so its support for formats depends on what GStreamer packages are installed on your distribution.

Banshee supports music sharing over a network using DAAP, the protocol introduced by Apple for music sharing. Unfortunately, Apple has modified the protocol since its introduction and now has broken compatibility with Banshee in iTunes 7, so sharing music between Banshee and iTunes is no longer possible. However, users with several computers in the house can enjoy streaming music between instances of Banshee.

The version numbering might be a bit confusing for newcomers. On the Banshee site, the 1.5.x series carries a parenthetical version number of “1.6 beta x”—so, for example, Banshee 1.5.3 is tagged “aka 1.6 beta 3”. This shouldn’t be taken to read that the release is actually beta quality. It’s an indicator that most of the new features are being added in the odd releases (such as 1.5.x and presumably 1.7.x once the 1.6 series is out) and that the 1.6 series will be mostly bugfixes and performance enhancements.

Speaking of performance, one of Banshee’s key features is its speed. Banshee can read in a media library in the tens of gigabytes in minutes. My music collection consists of more than 60GB of audio, and Banshee is able to import the library in just a few minutes.

Banshee also works with most popular MP3 players and devices like the HTC Android and Nexus One. Unfortunately, Banshee doesn’t work with iPhones and iPod Touches because they’re not standard storage devices and can’t be mounted the same way as iPods. However, most other MP3/media players work well with Banshee.

Banshee sports a very clean, uncluttered interface that is easy to navigate. In 2009 the Banshee team worked a bit on a mobile interface for Banshee called Cubano. Some of the interface enhancements for Cubano are making their way into mainline Banshee to help improve the player for not only netbooks, but also for standard desktop users.

You don’t need to have a vast music library to enjoy Banshee. It sports support for podcasts, so you can get all the episodes of your favorite podcasts via Banshee, or just stream some tunes via Last.fm or Internet radio.

What’s New in 1.5x?

A lot of improvements have been introduced to Banshee in the last year in the 1.5.x series. The first release in the 1.5 series came out in June of 2009. It brought improvements in media importing and handling, by allowing users to migrate from Rhythmbox and providing for separate library locations for music, video, podcasts, and so on. The 1.5.3 release adds support for organizing audiobooks separately from the music collection to make it easier to find books.

You’ll find quite a few handy features to make it easier to organize your music and create playlists, or shuffle music to avoid what we used to call “train wrecks” in radio: Songs back to back that don’t belong together. Who wants to hear Brahms segue into R.E.M.?

Banshee can detect the Beats Per Minute (BPM) of songs in your library to help sort, search, and create playlists based on the BPM of your songs. It also has shuffle modes that will take into account your scoring of songs and play back your favorite songs more often. Don’t want to take the time to rate songs on your own? Banshee can automatically assign scores based on whether you skip songs or not and how often they’re played.

If you don’t feel like loading your play queue manually, Banshee has an Auto DJ feature that allows you to fill the queue by artist, album, song, rating, etc. This way, Banshee can jam out tunes all the day long without any need for user intervention.

With the 1.5.3 release, Banshee will also generate smart playlists to sync music to your player, or you can pick your music manually as always. The latest release also allows Banshee to make use of the GIO file backend, so Banshee can import non-local files off of network shares.

Finally, the Banshee team has been putting a lot of effort into accessibility (a11y). The 1.5.x series includes improvements in focus handling, button naming, and so on. According to the release notes for 1.5.2, Banshee “should now be fully accessible.”

Try it Out

Banshee is installed by default on openSUSE if you install the GNOME desktop. Users on Ubuntu and other distributions can find Banshee in their package repositories.

What’s in the repos may not be the most recent releases, though, so be sure to visit the download pages and see if there’s a more recent version of Banshee for your preferred distribution.

About the Author

Joe ‘Zonker’ Brockmeier is a longtime FLOSS advocate, and is GNOME PR team lead. Brockmeier is a freelance consultant and author. Brockmeier has worked as a community manager for openSUSE, and prior to that was Editor-in-Chief of Linux Magazine, and has also written for Linux.com, OStatic, Ars Technica, IBM developerWorks, LWN, Sys Admin, UnixReview.com, Enterprise Linux Magazine, and many others. You can learn more about Joe or contact him here.

Discuss this story with other readers on the GNOME forums.

Interview with Jonathan Thomas, creator of the OpenShot video editor

Paul Cutler interviews Jonathan Thomas, creator of the OpenShot video editor. OpenShot is a non-linear video editor with support for many audio and video codecs, GNOME drag and drop support, titles, transitions video encoding and transcoding and more.

 

Would you please introduce yourself?

My name is Jonathan Thomas, and I am the creator of OpenShot Video Editor. I am a professional software developer, and spend much of my day working on proprietary software for a private company in Texas. I divide the rest of my time between OpenShot and my family.

According to the OpenShot website, you started using Ubuntu in early 2008 and the lack of a video editor led you to create OpenShot. How did you teach yourself Linux and create OpenShot in less than two years?

Soon after evaluating Ubuntu, I had it installed as my primary operating system. It captivated me, just as it has captivated so many other people. I am still not a Linux “expert”, but I am extremely focused, and only learned what was necessary for OpenShot to succeed.

OpenShot uses GTK, Python and MLT (Media Lovin’ Toolkit – an open source multimedia framework). What process did you go through to choose these languages and tools to develop OpenShot?

Well, having never used any of those technologies, I had to start from the beginning. Since I was more familiar with Gnome, GTK seemed like a natural choice. After a brief look at the Mono Framework, I decided on Python. One of the main factors for choosing Python was its impressive support for multimedia library bindings, such as GStreamer and MLT.

What advantages did you feel MLT had over Gstreamer?

I had originally chosen GStreamer and Gnonlin as the multimedia framework for OpenShot. But I quickly ran into many issues, such as no support for transitions, no support for compositing multiple videos and images, inactive mailing list, and a complicated API. I invested a lot of time into GStreamer, and it was a good learning experience, but just not right for OpenShot. Had MLT not existed, this would have been the end of OpenShot. Not wanting to give up, I then re-evaluated the MLT framework.

The MLT framework has a wonderful API, great documentation and examples, an active mailing list, support for transitions, compositing, key-frames, audio and video effects, and much more. For a more detailed comparison of the two frameworks, you can read my comparison,
http://www.openshotvideo.com/2009/03/mlt-vs-gnonlin-lets-get-it-on.html.

As someone who is fairly new to GTK, what do you like about GTK?

I choose GTK primarily because it was the standard toolkit of Gnome. I wanted OpenShot to look like a native application in Gnome, and so I chose GTK. In addition, KDE already has a great video editor based on the MLT framework called Kdenlive (which uses the QT toolkit).

What do you feel were the disadvantages in using (or learning) GTK?

I do not feel there were any disadvantages in using GTK. I am very happy with that decision.

A number of video editors for Linux have come and gone, such as Diva. In less than two years, how have you built a community around OpenShot?

So many open-source projects do not communicate with their users. They infrequently update their websites, don’t use their mailing lists, or just flat out don’t listen to their users. From the beginning, I decided to create a blog, and write about every decision, idea, and feature I thought up. When users spoke, I listened and responded. It did not take long before I had many dedicated users offering advice, feedback, and suggestions. Now, we have hundreds of users offering feedback every month, and I am very proud of the community we have become.

What were the biggest challenges you had to overcome throughout the development process?

The biggest challenge I have had to overcome has been trying to build an installer for OpenShot. Packaging requires a lot of knowledge about how Linux works, and this was probably why I had so much difficulty learning it. I spent 2 full months trying to learn Debian packaging. However, if I had not learned how to package OpenShot, tens of thousands of users would not have been able to evaluate it, and OpenShot would have a much smaller community.

As a side note, I am still waiting on a Debian sponsor for OpenShot. This is necessary so Ubuntu can include OpenShot in Lucid Lynx

What led you to choose using Launchpad for development and how did it make it easier?

I was impressed with Launchpad’s integration of Bazaar, bug tracking, and translations. Bazaar has been awesome, and is much better than any professional source control package I have used. The translations module in Launchpad is also great, and has allowed many OpenShot users to contribute to their favorite language. I imagine we would not have translations for 42 languages if I had not choosen Launchpad.

With the recent release of OpenShot 1.0 this past January, what does the future roadmap hold for OpenShot?

We are in the process of building a roadmap for version 1.1 on LaunchPad. However, we are doing a bit of house cleaning on our bug reports at the moment. In the next week or so, we will start discussing the specifics of our next version. All I can say, is there are many awesome features we have in the works, so stay tuned.

You can learn more about OpenShot at its homepage at www.openshotvideo.com, download OpenShot or visit OpenShot’s project page on Launchpad.

Discuss this story with other readers on the GNOME forums.

GNOME Boston Summit 2009

Jason Clinton recaps the talks given at the Boston Summit in October 2009.

 

Introduction

Each year over Columbus Day weekend, GNOME developers — especially North Americans — gather for a casual hackfest in Boston, Massachusetts. The atmosphere is friendly and informal; most of the attendees know each other from other GNOME events. There are no booths at the event; companies are only represented by their employees.

This year’s Summit was especially productive. With a clear goal in sight (GNOME 3.0), everyone seemed to be on-task. Discussions were cordial and not too tangential. Ideas were rapidly brought up and filtered. New objectives were set and agreed upon. And, perhaps best of all, this Summit was very cohesive: almost everyone was able to attend every session that they needed to.

During and just after the summit, I posted notes on several sessions to my weblog—about 7,500 words worth. What follows is a much shorter summary of the Summit. I’ve left out the discussion to give you just the sessions’ conclusions and interesting factoids. Rather than a linear view of the Summit, I have provided an executive summary, merging like sessions where possible.

Logistics

Around 9:30 on the first day, Saturday, attendees began to wander into the MIT Sloan building in Cambridge, enjoying the coffee, tea and bagels provided by Red Hat (headquartered in Boston and well-represented at the Summit). The Summit began at 10AM with a short session, moderated by Jon McCann of Red Hat, in which attendees threw out ideas for half-hour or hourlong sessions. The attendees sorted proposals, voted on them, and scheduled the first two days of the Summit schedule, trying to ensure that popular sessions wouldn’t conflict. The last day, Monday, was reserved for hacking.

The Plumbing

GTK+ 3.0 and Client-Side Decorations

Matthias Clasen talked about the plans to intentionally break the GTK+ API for GTK+ 3.0, and the new features the backwards incompability would allow: XInput2, the drawing API, and the ability to change theme engines without breaking applications. One goal was to make deep hacks less appealing, to put an end to the current string of abusive theme engines that must break with each new release of GNOME, and to provide a feature-rich foundation on which upstreams can do solid work.

Cody Russell discussed client-side-decorations (CSD), a feature which is already being developed on a side-branch. The authors aim to merge this work in time for GTK+ 3.0. CSD enables GTK+ support for things like Google Chrome’s tabs-in-window-decorator, and allows GTK+ to render the window border theme (rather than leaving it to the window manager). Cody said that he had been working with Nokia to ensure that Qt’s approach would be similar, to avoid compatibility issues between Qt and GTK+ applications that provide their own CSD. To ensure that windows can be closed even when frozen, some people have discussed extending EHWM to specify an area of the window that triggers “close” events. Having the CSD code handle resize events will prevent glitchy drawing when X11 windows resize before the border finishes drawing. There are no plans in the cards for a “Glib 3.”

Glib/D-Bus and GSettings

Ryan Lortie began with a brief history of some existing methods for D-Bus access in GNOME. gbus was a libdbus replacement based on gvariant, libdbus being the core protocol library that the daemon, and the glib, Qt, python bindings, use. Two different people had worked on implementations called “gbus” and another on eggdbus. These implementations are not good enough, and Ryan proposed three steps to move forward.

  1. merge GVariant (G-ified D-Bus’s data types) into Glib
  2. have libgio link to libdbus1, possibly via a GIO module extension point
  3. pick some of the best parts of gbus and eggbus and merge them into GIO as GDBus

At Colin Walters’ request, it was agreed that this new binding would get filtering and path registration. The attendees also agreed that it would be a good idea to put libunique in GIO on top of this new binding.

Ryan then demonstrated GSettings and its API. At the time, GSettings was a proposed replacement for gconf; it has since been accepted. Participants said they wanted to add Gsettings to GIO, too, after GVariant was merged. GSettings uses atomic changed signals to change many settings at once, rather than a signal for each setting. GSettings’ signals do prevent recursion, and a feature was planned to allow filtering returned signals. The GSettings schema format is a simple plain text file (in a python-ish format with indentation and colons). GSettings supports inheritance, as expressed in the schema format (schema inheritance).

Ryan demonstrated dconf as a replacement for gconf (the daemon that merges application writes into the on-disk data-store). He went over the simple API as it stood then. Compared to gconf, dconf was 10-50 times faster on reads, and had high-performance writes as well. Not only is dconf’s memory usage an order of magnitude smaller than gconf, but dconf is only in memory during the writing process. The dconf writer process uses fsync() a lot, but you don’t care because it’s not blocking your application.

To perform regression tests of applications which use dconf, one would set proxy settings in gconf/dconf without affecting the user session, or use a custom GSettings plugin can redirect the requests to a temporary database. Ryan said he’d be adding PolKit support soon.

For distributions that override upstream defaults in the distro packager, Ryan suggested modifying the schema via a package patch, rather than adding vendor-specific defaults databases to the system (aside from the standard default database).

Finally, the session wrapped up with a few details. NFS home directories with dconf were buggy because using mmap() over nfs is generally ill-defined, but Ryan was working on it and committed to finding a solution; the documentation would soon be going up on library.gnome.org, including some documentation on how to migrate modules. A gconf-to-dconf bridge was also in-progress. The bridge ran some simple applications, but there were some serious problems to solve. Ryan needed and still needs help, especially with this part.

Hallway Track: GTK+ Font Dialog Revamp

via Alberto Ruiz Alberto Ruiz and Máirín Duffy teamed up to improve the usability of the Font Dialog. Máirín did some mockups, Alberto implemented them in pygtk, and they ran usability tests with the prototypes. They promised a first prototype within a week, and a few days later, Máirín posted the mockups and prototype and asked for comments .

GObject Introspection

Colin Walters started the session with a short introduction to GObject introspection. In Colin’s opinion, GObject was ripe for this treatment: the type system already had a lot of prerequisites for an introspection tool, thanks to features designed to make it easier for us to script our C libraries. We needed a few more things, though—mostly annotations. To that end, Colin demonstrated how to annotate C libraries to make them accessible from GObject Introspection. This annotation data declares (for example) what containers were being passed around by which pointers, and whether you, the consumer, would need to unreference the pointers when done.

When you run the annotation parser, an XML document is written to a .gir file; the .gir is then compiled to a .typelib, an efficient binary format containing the same information. These two parts would make a language binding work. Several language bindings were being worked on, and a GJS binding was already ready. PyBank, the GOI-based Python binding, was fairly complete at the time, though some call conventions had to be worked out for compatibility. Colin thought that the legacy bindings that had already been written shouldn’t be replaced. New bindings should use PyBank, and PyBank will preserve call compatibility between the old and new bindings.

Colin was looking forward to the day that gtkdocs were supplemented by annotation data, but some work remained to get to that point. As a closing thought, Colin left us with the suggestion to try polyglot programming in a project: just by using GObject, you would already be 85% of the way there.

Clutter 1.2 API

This session discussed API additions wanted for Clutter 1.2. Emmanuele Bassi listed some highly visible new features: texture atlasing, a unified texture cache, and layout management. Emmanuele was thinking of adding a modular plug-in architecture to handle input methods, similar to the one used by GIO: API stubs that could be implemented in inherited classes. This would allow nearly any functionality added via GtkModules to be re-implemented in ClutterModules.

Some upcoming performance work was also noteworthy. Ongoing work on frame buffer objects would make partial uploads of changed windows more efficient (for example, when the Evolution throbber is running, there’s no reason to upload the entire window again – we should just upload the changed sections of a texture), but there was no estimated completion date. A lot of people were working on getting VSync to work correctly on top of a GL window manager on top of buggy hardware. The problem lay deep in the hardware drivers and sometimes the hardware itself. There were three separate ideas about how to improve the performance of finding which object the end-user clicked on.

Emmanuele closed the session by getting a sense of the attendees that Clutter and GTK+ could peacefully coexist, targetting different kinds of applications and both having the full support of GNOME. By sharing code where possible and providing a usability guide, API references, and programming guides for each toolkit, we can provide a rich, diverse platform in which Clutter and GTK+ apps can fit together, and in which any type of application is possible.

GNOME Shell

Jon McCann from Red Hat ran the first Shell session, on design. An amazing amount of progress had been made since the hackfast one year prior that spawned the Shell project, but some major tasks remained. Fortunately, Red Hat decided to fund full-time work on the Shell, so progress was being made quickly (and still is).

Jon focused on the Shell UI as it stood then, and still is today. He said that the top bar had been redesigned. The right-hand side showed session-oriented actions (things that would apply to the entire session). A clock and date widget were in the center, “balancing” the top bar to provide an even distribution of content across the width of the screen. The far left-hand side held the “Activities” menu, which activated an overview mode. Between “Activities” and the clock widget, a dropdown menu would show the current application.

Some examples of session-oriented actions:

  1. Your presence (now implemented)
  2. Login and -out
  3. Messaging
  4. The control center
  5. Your own account information

The notification tray stays in the same area as the session-oriented menu, but with encouragement to use a new messaging API that would display it in a different location.

The overview mode accessed through the “Activities” menu is a work-space layout view similar to Expose. Work-spaces will not be forced on users, but would be immediately accessible should a user choose them: by default, there will be only one work-space. This would prevent a user from accidentally hitting the “switch work-space” hot-key, having all their windows vanish, and panicking. The left-hand pane of the overview mode displays a search box, lists of your favorite and of all applications, recent documents, and bookmarked places (from Nautilus/GIO).

The new UI is application-based, and displays the current application in the top bar. There were plans for a new API that would show application-level actions: quit, preferences, new window, etc. The Shell requests a new window from any application by invoking the binary again (hoping that it uses libunique or a variant). The new Alt-Tab interface also cycled among applications, with Alt-` showing an extended view of the individual windows grouped beneath their application icon. Mouse navigation and mouse wheel access is also possible in the Alt-Tab UI.

Owen Taylor was concerned about the discoverability of the session menu and offered one proposed solution: a help bubble on first login that would say “Click here to set your name.”

The overview search was an open problem at the time. Then, only menus, recent docs, and places were searchable. Since then, a semi-formal commitment has been made to allow plugable search back-ends.

A major objective that was just beginning was messaging: everyone was tired of having chat, SELinux, etc. stealing application focus. Canonical deserved credit for trying to solve this in GNOME 2, and the messaging design was inspired by this earlier work. Since the Summit, an initial implementation has been committed to a public branch. A message queue in the lower right hand-corner displays things waiting for your attention: chat applications, mail notification, system warnings and music players. Unlike with the notification area, acknowledging receipt of a message is not the only way to get rid of it: a “message” can be suppressed temporarily without discarding it.

Newly arriving notifications slide up from the bottom of the display to show a single line of text, remain for three seconds, and then slide out. To see the pending messages needing your attention, you can move your mouse to the lower right corner and the message queue will slide in. You can also examine the message queue in the overview mode.

Telepathy chat apps add a simple reply window to their message notification, so that an interruption can be immediately taken care of and the user can return to their primary focus. The IM client itself will continue to handle more involved chat. A libnotify compatibility layer is implemented to support legacy applications.

Owen Taylor and Colin Walters opened the second session about Shell: tech. Colin showed how the internals of Mutter became accessible to GJS/Javascript inside the GNOME Shell’s Mutter plug-in. He demonstrated the interactive JavaScript console inside GNOME Shell and used the Looking Glass tool to inspect the stage, to get a reference the the JS object associated with an actor, and to change its properties.

Owen took over and showed how to take the property you just changed, change it in the code on-disk and then type “restart” to reload the shell with your new code—no compiling. He also demonstrated changing the CSS file for the calendar widget and then restarting the shell. At the time of this writing, the majority of the UI is theme-able with CSS. Shell’s CSS implementation stays as close as possible to the behavior that you’d expect on the web, making it easier for new people to understand. The Shell Toolkit also supports inheritance of CSS properties.

Accessibility then came up. Owen said that once Clutter and the Shell Toolkit were accessible, very little work would remain. There was a new magnifier inside Shell that was already being implemented. Someone asked about “Show the Desktop”. That elicited a slightly sheepish response: they didn’t really know what to do about it then. As it stood, “Show the Desktop” sucked. If you were to accidentally click on the desktop and press Delete, you would delete a file. There was some thinking that showing the desktop at all would be a bad idea. Since Summit, Alex Larson and others have been discussing this on the mailing list.

Jon McCann said that they would like to manage tabs within applications at the Shell level, and that some application API was needed to make this happen. For example, it would not really make sense to group GMail with the rest of your browser tabs. One idea was to support single-instance tabs: clicking your GMail icon would yield your existing GMail tab if you already had GMail open. Obviously this problem requires some cooperation from the browser. Another idea was to filter all tabs at the level of the Shell, allowing you to pick a tab from the overlay. The GtkNotebook widget could communicate this to the Shell. Another useful feature would be to allow you to search window titles with the search bar.

The developers gave a short list of things to work on: the messaging tray (already partially implemented), application browsing in the “More” menu (just rewritten), recent documents, people-based search results, fly-over pop-ups for more information, the sidebar, the application menu (on the top bar) and extensions.

Owen and Colin saw existing panel applets as a multi-faceted problem. Things like Keyboard Layout applet would need to become first-class citizens. On the other hand, the Deskbar could be removed if the search field were made extensible. Other plugins, like a stock ticker, would need to be sidebar extensions.

Miscellaneous

Splinter and git bz

Owen opened the session by demoing Splinter, the web UI for reviewing patches.

Splinter’s UI makes it easier to add your own comments and to view other people’s comments in context with the code. The UI provides a patch view in which clicking a line adds an annotation.

Splinter stores the results in a normal Bugzilla comment that is human readable and parseable by the Bugzilla extension. As you work, Splinter stores your work in DOM storage so you won’t lose it if your browser crashes.

Owen said that he didn’t intend to push it upstream, but to advocate and push it as an extension‚ especially to the Bugzillas he is forced to use.

Owen then spoke on git bz, the git bridge to Bugzilla that allows users to post patches for review. In response to requests for a branch-per-bug, Owen said that this would be hard because, for example, what would happen on a `git bz pull` in this branch? Would it pull the current bug status?

Owen said he thought that git bz would never move upstream: git bz is in Python and upstream is C, Perl and lots of Shell.

GNOME Women Outreach / Marketing

Marina Zhurakhinskaya from Red Hat ran the session.

In 2006, a Google-supported, 3-slot version of Summer of Code was run exclusively for women. Marina said that the Board would like to repeat that using GNOME Foundation funds. The first task would be to contact the women who participated in the 2006 program and get an idea of where they stood today regarding open source involvement. (Since the Summit, GNOME Journal has published an article about this very topic.) Some people had a perception that, generally, these programs work best when they select participants who have already contributed in some regard. Marina said that this has been a challenge, because there haven’t been a lot of women in GNOME.

The second task would be developing better introductory materials for both GSoC and Women’s Outreach. Earlier meetings had suggested making resources for students that mentors should review. For example, a mentor should introduce their student to certain aspects of the GNOME community, such as IRC and Planet GNOME—the social side of things. Also, we shouldn’t make it a scavenger hunt for students to find the the technical documents they need to accomplish their goals.

Third would be proactively finding students by contacting universities and asking: “Do you have students who would fit this Women Outreach program?” This could make a huge difference in the level of participation.

Finally, another focus area would be finding mentors and working with them before the program starts. (You can sign up as a possible mentor.)

An attendee pointed out that, generally, we have been bad about helping new contributors, regardless of gender. Most of us haven’t had time to do handholding for new contributors with no experience. Everyone agreed to revive GNOME Love. Everyone remembered this being a success, until the people (or person) providing the man-power could no longer run it.

A proposal for an effort to get more Windows-to-GNOME conversions started a segue to the topic of marketing. Some wondered whether GNOME should even be brought up in such a conversation, pointing to a “What is a web browser?” video that Google had published the previous week. Since five of the six most popular distros gave users GNOME by default, most people would have no idea that they’re using GNOME, not having selected it.

The discussion got very lively at this point, switching among many tangential topics. Some argued that advertising should be only a tiny portion of marketing, and that a huge part of marketing has been listening to users and implementing what they want. One person pointed out that many people have started tweeting their frustrations, allowing us to mine social networks for “hot spots” of usability problems, and even to engage users directly and help them solve their issues.

There was disagreement about which message to emphasize in advertising: cool stuff you could do with free software, or freedom itself. One participant mentioned iPhone commercials as a good example of showing off cool stuff that makes people want to have the product. This tangent was tabled but there appeared to be agreement that short videos would be better than introductory documents.

We returned to the theme of trying to get more users and there was agreement that it’s a very hard task to accomplish. A point was raised that, as with hybrid cars, a portion of the population would respond to a message that “this is good for the world.” There was agreement that this would work for some, but not all audiences. The discussion ran out of time, but the GNOME Marketing Hackfest several weeks later took up most of its themes.

Telepathy BoF (Will Thompson)

I attended this Birds-of-a-Feather session. Below you’ll find my summary of Will Thompson’s session notes.

The session opened with a discussion about gnome-games’ tubes code, which was broken when Empathy moved to Mission Control 5. Developers were working at the time to get it working again, and are working still. Making gnome-games use a newly written contact selector widget that supports MC5 will get it to work again; this selector could also form the basis of the long-anticipated telepathy-gtk.

The Tubes API currently makes it difficult to request a channel for yourself. Empathy contains helper code to make this easier, code that can be moved the telepathy-glib.

We discussed new ways to use Telepathy. I wanted a way to share high scores with your contacts, and they suggested using tubes. The design: gnome-games would come with a tiny daemon that would handle tubes for the “org.gnome.games.highscores” service, serving up your high scores to any contact who asks (if you select “Share my scores with my contacts”). When you open your high scores table, it would use ContactCapabilities to discover which of your online contacts supportted that service, and then would ask them for their latest high scores (with a cache and a rate limit).

Sjoerd noted that Danielle had written a helper which would let you say: “give me a stream to this person,” and that this could conceivably be extended to set up a socket automatically. gnome-games will be documenting how they’re using tubes. Rob suggested pushing this into telepathy-book.

John Palmeri suggested another way to improve documentation: make writing documentation a requirement for Google Summer of Code. Sandy said they’d been discussing documenting requirements and standards for GSoC students, and thought that it would be a great idea to ask students to blog stream-of-consciousness “this is what I did” updates. People would have to make sure that they do this as they go along, because you would forget the learning process after a few weeks.

Rob noted that Telepathy hackers know that you would need to use, e.g., a Handler and a Tube, but people who aren’t immersed in the stack have a lot of prerequisites to learn in order to understand that stuff.

The conversation moved on to other possible uses for Telepathy tubes: integration with DevHelp would be nice to let people post examples.

A closing thought on the future Telepathy tubes was that it should use UPnP to make file transfer and tubes fast in more cases. This was very much on the to-do list.

Geolocation with Emerillon

Pierre-Luc Beaudoin ran this session on Emerillon, the map and location application. It is available and in GNOME git. Pierre-Luc demonstrated Emerillon and libchamplain (the map widget) for the audience.

With Emerillon, when browsers and websites request your geolocation, a dialog prompts you before sending your location to the requester, giving you control over your privacy.

Pierre-Luc envisioned several uses, immediate ones including viewing contacts on a map or geotagging photos. Other ideas included updating social networking apps with your geotag on Twitter. Some suggested integrating with Mojito on Moblin.

Pierre-Luc said that OpenStreetMaps was offering pixmap tiles, but there was a GSoC project to download the meta-data in raw XML format and render it via Cairo, locally. Pierre-Luc said that OSM’s servers were rather overloaded. Ted Gould from Canonical suggested pre-seeding the cache with all North American data, for example, for a netbook. To let Emerillon submit edits to OSM, use the right-click menu to open OSM in a browser and edit the data.

Pierre-Luc demonstrated some comparisons of OpenStreetMap with other online maps. There were som cases where the data in OSM was clearly better. He demonstrated how geographical data is integrated, as well as public transit (mostly in Europe).

He requested that everyone give Emerillon a try.

Usability Kit

The session began with a demo of the “usability kit” developed by Máirín Duffy and Ray Strode. The usability kit records the keyboard, screen, and the user’s face. The box contains a DVR-style device, running embedded Linux, which stores all four video inputs. The total cost of all hardware, including VGA scan converter, was $800.

The user’s behavior was recorded as they performed a set of tasks given by the test giver. The analysis was done after-the-fact by filling in the user’s behavior while he was attempting to complete the task. The videos have provided actual documentation of the lag between a user’s click and UI’s response (especially helpful for a web site design).

Máirín tested a wide range of users, from developers to regular users.

Máirín recommended narrowing the testing scope to a specific set of tasks. Rather than saying, “I’m going to test GNOME,” narrow it down to “File Management” and then come up with a list of tasks related to that. Then measure those.

Ray finished the session by showing how they used GStreamer to stitch the four separate video feeds in to a single video. In a subsequent blog post he explained how this was achieved:

Summary

The GNOME 3.0-specific sessions dominated hallway and dinner conversations; this was clearly where the excitement was at. GNOME Shell was a major factor, but many were excited by other upcoming changes, like GSettings and even the client-side windows work. The sense of momentum was palpable and this energy has carried us forward for the past two months. Thanks to the success of the Boston Summit 2009, we’ve made major progress towards our objectives, and have positively influenced other teams and hackfests.

About the Author

Jason works for Advanced Clustering Technologies building beowulf clusters used in all kinds of scientific research; as a part of the job, he occasionally travels. He also published a book on the Ruby programming language called “Ruby Phrasebook”. In GNOME, he is a Games module maintainer. Part time, he attends UMKC for a BS in CS.

Discuss this story with other readers on the GNOME forums.

PiTiVi

Jono Bacon provides an overview of PiTiVi, an open source video editor for the Linux desktop.

For many of us, using Linux is like being a kid in a candy shop. Want office productivity? Check. Want to listen to and watch multimedia? Check. Want to get online and communicate over a raft of mediums? Check. Among the many shelves, stocked with treats aplenty, has been one strangely lacking shelf: that of video editing.

While Linux video purists will argue that there have always been video editing solutions, these have typically been the domain of rocket scientists or have lacked the necessary features to put together most video projects. For those of us passionate about the GNOME project, traditionally we have only had Kino as a real option, and while a strong and solid application for capturing and performing basic edits, Kino runs out of steam for more complex projects and requires a certain amount of Kino mojo to operate.

Back in 2006, the video editing situation was looking far more exciting. Michael Dominik was working on the hugely exciting Diva project and Edward Hervey was working on PiTiVi. Both combined exciting technologies, being built on the formidable foundations of GTK, GNOME, GStreamer and Cairo. Diva was developed using Mono, and PiTiVi using Python. With the video buzz in the air, Michael and Edward both demoed their projects at the Villanova GUADEC to rapturous applause.

Unfortunately, the Diva project shut up shop as Michael moved onto other projects, but PiTiVi plowed on, primarily driven by Edward Hervey and his dream to make easy-to-use Open Source video editing a reality. Today Edward and his cohort have made tremendous progress.

The Technology

Part of the reason PiTiVi has taken so long to mature is that Edward has been conscious to build PiTiVi “the right way” and ensure that the underlying infrastructure that PiTiVi is built upon is mature and stable.

At the heart of PiTiVi is the incredible GStreamer framework: a technology that enables developers to string together different modules with different capabilities (such as decoding a file, applying an effect, adjusting volume etc.) into a pipeline that can perform a given operation. Wizened UNIX-heads will remember that this is the heart of how UNIX was designed: create a series of tools that do one thing really, really well. GStreamer takes this approach to multimedia, offering a huge array of different elements that can be added to the pipeline. This has resulted in a devilishly simple but powerful framework for application developers.

For Edward to build PiTiVi he needed to create his own set of GStreamer elements in the form of GNonLin: a set of elements designed to make non-linear audio and video editing a possibility. These elements were needed to handle the complex requirements of a non-linear editor. These editors take source video and audio files and allow you to edit their lengths, overlay them, apply effects and other features, and all without destructively changing the source media. GNonLin provides an incredibly powerful set of components that were designed in a truly extensible and flexible way to allow anyone to build a non-linear editor, be it video or audio.

For Edward to have created the first incarnation of PiTiVi he needed to ensure that GStreamer and GNonLin were mature and stable enough to use for his application. Back in 2006 and 2007 GStreamer had been through a significant period of change and GNonLin was newly created to help build PiTiVi. Edward entered a potentially project-killing write-some-code-for-pitivi-and-then-fix-the-gstreamer-bugs-to-make-it-work period. Although GStreamer had enjoyed a healthy period of active development and was stable and mature in many areas, PiTiVi was pushing the framework toward less-tested grounds. Most GStreamer applications were media players as opposed to media production environments and PiTiVi was blowing the cobwebs off some rarely used elements. Fortunately Edward had the smarts to get in there and fix those components.

Around this time I had announced the formation of the Jokosher project, the development of an easy-to-use audio production environment that was fundamentally built on the same technology as PiTiVi: Python, GStreamer, GNonLin, GNOME, GTK and Cairo. As our project kicked off, we entered into a similar round of writing code and uncovering bugs in lesser known parts of the GStreamer stack. As the only other application using GNonLin, we in the Jokosher camp worked closely with Edward, and he was remarkably responsive in fixing bugs, and rolling out new releases. Not only that but the GStreamer developers were also hugely responsive in fixing bugs in the platform. Although the bugs blocked progress in both the Jokosher and PiTiVi camp, as each bug was fixed GStreamer and GNonLin became an ever more exciting and compelling platform for multimedia.

Today

Since those days, GStreamer and GNonLin have continued to mature into a hugely powerful and stable framework for multimedia production, and we are now starting to see PiTiVi show off the fruits of that labour. The latest release, 0.13.3, is already showing promise, with a great set of features on this firm foundation:

  • Simple to use interface.
  • Import a variety of different types of media.
  • Tools for cutting, moving and layering video and audio.
  • Audio mixing.
  • Ripple/roll editing.
  • Undo/Redo.

PiTiVi is now really beginning to come into its own. Regular releases are flowing, each bringing new features and bug-fixes, and the development team has expanded from just Edward Hervey to include full-time developers and community members. It is time for video in GNOME to rock and PiTiVi is leading the way.

To learn more about PiTiVi, visit the PiTiVi homepage at http://www.pitivi.org.

About the Author

Jono Bacon is the Ubuntu Community Manager, author of The Art of Community by O’Reilly, founder of the Community Leadership Summit and contributor to various GNOME projects, including Jokosher.

Discuss this story with other readers on the GNOME forums.