PyGTK, GObject, and GNOME 3
Sumana Harihareswara interviews Tomeu Vizoso and John “J5” Palmieri about PyGTK, GObject, introspection and PyGObject. What’s new, what’s been hard, and what’s next?
Sumana Harihareswara: It’s my perception that the PyGObject project is part of what’s exciting for developers about GNOME 3—under-the-hood improvements that give application developers more power and flexibility. Is that right?
Tomeu Vizoso: I think so. During the development of Sugar (which makes extensive use of Python, GNOME and Freedesktop), I have felt several times the limitations imposed by the unavailability of Python bindings and their lack of maintenance and packaging. There’s great work going onto the GNOME platform, and it was frustrating not being able to fully use it. And, as a nice side effect, with PyGObject we were able to address the terribly slow startup and memory usage of PyGtk applications.
John [J5] Palmieri: I would say that Introspection in general is very exciting. The idea that you can write a relatively small kernel of code to interface with any language runtime, and automatically get access to a large number of GObject-based libraries, means that the barriers to developing for GNOME are significantly dropped. The ability to move between Python and C fairly quickly means you can prototype in a scripting language and then write the slow bits in C for performance enhancements. And, as we have seen with the libpeas development, a library for creating a Python plugin frameworks in PyGObject Introspection, it makes providing extension points for your apps easy. Python has historically been the most popular GNOME scripting language, so it is logical for it to be the most complete at this point.
Python, C, Overrides, and the switch
SH: What misconceptions do you want to clear up about PyGTK or PyGObject?
TV: I would like to make it clearer that there’s no need any more for bindings for each library. You just need a recent PyGObject to be able to use GObject-based libraries from Python code. Though an introspection-enabled PyGObject is already stable and well-supported, it will take a while until everything works at 100%, so now is the moment for applications to be ported and bugs filed.
JP: I would ask this as, what are the differences between PyGTK and PyGObject, and why did we decide to use PyGObject over PyGTK for the future of GNOME?
It is first important to point out to the reader that PyGTK is actually based on top of PyGObject. At one point PyGTK was one monolithic code base. However, as gobject became useful outside of just building GUI applications, PyGTK lagged behind, requiring a dependency on GTK+ for apps that just wanted to use some of the non-GUI modules.
It was decided at one point to split the modules into PyGObject and PyGTK, where PyGTK would just be another static binding that was built on top of the PyGObject core.
During the 2010 Python Hackfest in Cambridge, Massachusetts, it was decided to continue work on PyGI, the Introspection fork of PyGObject, and eventually move it back into the PyGObject library. Again, PyGObject would be the base for other gobject-based libraries, except this time you wouldn’t have to write glue code to get them to work.
This is the major reason we decided to move to Introspection instead of continuing patching the static bindings. For instance, we were able to support GTK3 along with the new APIs such as gdbus and GtkApplication with minimal work. The great thing is, a lot of the code that went into making GTK+ work with Introspection benefits other gobject libraries as well. It also makes Python 3 support much easier to maintain, since all the Python 3 code is localized to the PyGObject code base. The modules that run on top of the main PyGObject code don’t have to worry about supporting C Python APIs. They only need to understand the base GObject Introspection concepts such as annotation and modelling API’s so that they are wrappable. This base understanding actually makes GObject developers write clearer public APIs by restricting the use of, or confining to private APIs, some of the more esoteric code patterns that C allows.
Unlike PyGTK, we also support overrides written in Python itself. The two uses of Python overrides are (1) to support legacy APIs such as a PyGTK compatibility layer, and (2) to make APIs more Pythonic by hiding some of the C’isms that direct API mapping exports. This has been a great motivator for getting people to contribute to the project. Since the barrier to contributing has been lowered by moving to a Python API from C Python API, any person who has ever wrote a PyGTK app can now write overrides with very little extra effort. We have seen a lot of contributors start with the Python overrides and then start diving into the actual core C bits.
That isn’t to say PyGObject Introspection is perfect. We still don’t support 100% compatibility in PyGTK and never will. Overrides come with some overhead and we prefer C authors to write APIs that are usable without them, bringing the advantages to other scripting languages as well. PyGTK will also be closer to the Python way of doing things, because it focuses on individual APIs more than the generic approach we take with Introspection, trading off maintainablitly for a kick-ass interface. There is also the dependency issue, which isn’t fully worked out yet. PyGObject’s dependencies are soft dependencies, so there can be issues if C library APIs changes aren’t caught before they are deployed. With PyGTK, there are hard compile time dependencies, so packagers are alerted early in the process if they are updating a package that may break things. PyGTK has also been rock solid for ages now. It will be awhile for critical path PyGTK apps, such as Fedora’s Anaconda installer, to fully trust the new way of doing things.
Sponsors, contributors, and upstream/downstream
SH: What companies and organizations have been sponsoring this work?
TV: I started my work on PyGObject as part of my involvement in Sugar Labs, and when I was later employed by Collabora, they sponsored this work. It has been critical as well to get the support from Red Hat by putting J5 full-time to work on this. Lately Martin Pitt from Canonical joined the team, and has been doing a really awesome work while porting his apps to introspection.
JP: Collabora has been great with providing us with funding for hackfests and giving Tomeu the time to hack on major parts of PyGObject. I also can’t thank Red Hat enough for allowing me to work full time leading the project (even though I probably should be working on more Fedora’y type projects). Martin Pitt was also at the last hackfest representing Canonical. He is responsible for making the GDBus overrides easy to use. However, to a large extent our developers are independent contributors from the community who started by submitting patches or bug reports and then slowly started diving into the internals. We welcome anyone who is interested in contributing. Come join us on #python at the GIMPNet IRC servers.
SH: …and who would you like to see step up and offer more support?
TV: I’m quite happy at the current support from both companies and volunteers alike (the GEdit/libpeas hackers have been great) but I must say that I have been quite disappointed at the big efforts that I have had to do to get downstreams involved. I believe it should be obvious that upstreams having to reach downstreams is not scalable: it should be the other way around. If you package software that is being obsoleted, you need to make some effort to understand what is the replacement. If you develop on top of some software, you need to understand where upstream is going and participate on its development. Money enters into the free software ecosystem through downstreams, and we need to get much better at channelling it to where it can have the most impact.
SH: The current porting status/TODO page shows that a lot of apps haven’t been ported yet. What do you expect the timeline will be for that? Is this an activity that we should have a hackfest for?
TV: From talking with application maintainers, they seem to have been waiting for PyGObject to be ready, as if it could have ever matured without their involvement. Fortunately, as part of the push for GNOME 3, a few have gotten involved and made it possible. But it could have happened way earlier…
I’m pretty sure a hackfest would be great for that, based on the outcome of the last one we had.
JP: There isn’t really a timeline. We started on this focus during the last hackfest in Prague, Czech Republic, except we were focusing on what porting applications needed fixed rather than actually porting applications (though that did happen at the hackfest). At Red Hat, we have a push to port modules to Python 3, so internally this will push a lot of our Python tools to move to PyGObject as well. In this next cycle I am concentrating on finishing my invoke path rewrite, which will simplify the codebase, speed up the core, and allow us to support some of the harder introspection use cases. Other than that, we should be focusing on documentation and porting. It is at least my hope that by the next major GNOME release, all of the most-used GNOME Python apps are either fully ported or on their way to be ported. The next hackfest should be 100% dedicated to this effort.
SH: What unexpected challenges or opportunities have you run across while working on this project?
TV: It was a challenge to pick up a piece of software that, though it was widely used, had been practically unmaintained for a long time. Part of what it made it difficult was that most users of PyGObject are reticent to read and write C code, unlike users of the C APIs.
It has been a revelation how hackfests can help bring new people to a project, mainly by helping stabilize personal relationships between old-timers and newcomers. After the last hackfest, I’m certain I won’t be missed in the project when I move to other stuff.
JP: The biggest challenge has been coordinating with the different libraries and applications which support or are supported by PyGObject. Introspection is very new and with that comes growing pains. Before the binding developers could work in a bubble, catching up to whatever the API developers added or changed however they saw best. However, now everything is so highly integrated that the binding developers have to closely work with the API developers. This can cause a lot of frustration and a bit of extra work for all involved, but in the long run the closer collaboration will reveal itself in a better end user experience. If we have learned anything in GNOME over the years, it is the end user that matters and using Introspection helps move towards that goal.
Sumana Harihareswara manages projects and people. She has worked for the GNOME Foundation, the Wikimedia Foundation, and Collabora, and contributed to the AltLaw,Empathy, Miro, and Zeitgeist open source projects. She is currently an editor and the release organizer for GNOME Journal and a blogger at GeekFeminism. She blogs atCogito, Ergo Sumana and microblogs at Identi.ca.