Grilo: Integrating Multimedia Content in Your Application

Iago Toral introduces Grilo, a framework for making media discovery and integration easy.

This article introduces the reader to one of the challenges that media application developers have to face when they attempt to design and implement modern multimedia applications: the integration of content provided by many heterogeneous sources, and how Grilo, a framework focused on making media discovery and integration easy, helps developers face that challenge successfully.

Modern life and the new multimedia challenges

Traditionally, software platforms addressed the problem of multimedia by focusing mostly on providing means for application developers to deal with the huge variety and complexity of media formats and streaming protocols available: RTSPMMSOGGMP3MatroskaTheoraVorbisMPEGH264, etc. Nowadays, thanks to projects like theGStreamer multimedia framework, developers can create multimedia software capable of rendering a wide variety of content without having to learn about the underlying complexity of all these formats and protocols or deal with other low-level considerations. GStreamer makes all of those details transparent for them, remarkably easing application development. It is no wonder then that any software platform today ships a multimedia framework: developers demand this from their platforms just as they demand a graphical toolkit and many other commodities.

However, thanks to the technological progress during the last decade or so, and in particular, to the growth of multimedia services on the Internet, a new challenge for media application developers has popped up: users all over the world consume massive amounts of media content every day. Users enjoy their personal collections of media, which can be stored in different places like hard drives, usb keys, network shares or UPnP servers. Users can also watch videos on YouTube or Vimeo, listen to music from services like JamendoShoutcast or Last FM, and like being up-to-date by subscribing to podcasting services. These are just a few examples as there are many media providers which most people regularly use. Of course, this changes the way application developers face the development of multimedia software today; it is not only about playing back content in various formats and protocols, it is also about content integration: users want access to all of these services. Users want them properly integrated and they want the same experience no matter the service they consume the content from or the device they use to consume it. Whether it is a personal computer, a smartphone, their new connected TV or a Set-Top-Box, they only care about being able to consume the content from the services they like and have a great experience while they consuming it.

Unfortunately, existing platforms do not provide developers with appropriate tools to manage this situation yet, and thus, application developers interested in integrating multimedia content from these services are in for a lot of hard work. They must learn about the different APIs, protocols and technologies involved as well as coding application specific solutions for each individual service. Well known media applications in the GNOME desktop like Totem or Rhythmbox suffer this problem, but this is notGNOME specific at all, many other platforms lack support for this emerging issue.

It is only a matter of time before modern software platforms start shipping solutions to help developers face this new challenge. Just as we all expect any platform worth our consideration to provide a multimedia framework, there will be a moment too in which all of us will come to expect any software platform to ship a framework to help us with integrating multimedia content. This is because such a framework will be fundamental for creating successful multimedia applications in the near future and will be a tool developers will not want to be without.

Grilo

Grilo is a new project which aims to make media integration easy for application developers. The way it operates is very simple and familiar. Grilo abstracts the differences among media providers, hiding them from application developers by exposing a single high-level API that can be used to interact with all of them. This remarkably eases the effort required on the application development side. Application developers can now code their solution once and know that it will work for all the media providers supported by Grilo.

Because of the familiarity of the solution used and the high level nature of the framework, most developers should feel fairly comfortable understanding how Grilo works. This also helps with getting started very quickly.

Grilo is split into two parts: the core (grilo) and a collection of plugins (grilo-plugins).

The core defines the fundamental concepts used by the framework, like the concept of media (GrlMedia), which packs metadata associated with a particular media resource (a specific audio, video or image item). By metadata we mean information like Title, Artist, Album, Duration or the URI of the actual media resource. It is a representation of the media that is independent of the service that provides it.

Another important concept defined in the Grilo core are the media providers (GrlMetadataSourceGrlMediaSource). These objects expose APIs that application developers can use to gain access to the media content exposed by the various providers. These APIs specify how you can browse, search, upload or remove content among other things. The key idea is that no matter the service you want to interact with, it will offer a well known interface. Because of this, application developers do not need to code specific solutions for specific services. Instead, Grilo makes all of them behave alike. Interacting with YouTube is no different than interacting with Jamendo as they both expose the same API and thus behave the same. The only difference between them is the content that they expose is different.

With the fundamental concepts defined, all that is needed are GrlMetadataSource and GrlMediaSource implementations of interesting media providers. This is achieved through a plugin system through which developers can write implementations of these objects as plugins for Grilo. A Grilo plugin is nothing more than a wrapper of the public APIs exposed by a particular media provider that adapts them to the cross-service APIs defined by Grilo. The actual implementations of the plugins live in the grilo-pluginssplit.

Right now Grilo has plugins for many services: YouTubeVimeoJamendoFlickrShoutcastUPnPMedia BookmarksPodcasts and more, with a varying level of completeness and maturity. This means that application developers who choose to use Grilo can create applications that consume content from all of these services in one go and with very little effort.

Grilo and GNOME

Grilo started as a standalone development, not integrated with any specific platform, because we wanted to create a solution useful for as many developers as possible,GNOME or otherwise. However, because of the close relationship of its main developers with the GNOME ecosystem, Grilo was developed using GNOME technologies (GLiband GObject mainly) and GNOME was also the first and natural choice for the development team to explore. Indeed, Grilo is kindly hosted by the GNOME project, providing us with all the resources we need: wiki, bugzilla, a mailing list and Git repositories. All of this makes Grilo very natural for GNOME developers as this is a very familiar set of tools.

GNOME should define what its vision is on how media applications should consume and share multimedia content. Grilo can help by providing the building blocks to implement that vision. Indeed, some GNOME developers are already working on defining how this should work which is represented by the GNOME Media Server Specification. The idea is to separate the content provider code from the actual application consuming the content and communicate them over D-Bus.

This vision has benefits and drawbacks when compared to a non-distributed approach that should be apparent to most developers, but let’s focus on two of its advantages: it does not require applications to link against the media provider code which helps with stability for example, and enables application developers to use any language they like to implement their application. There is no need to use language bindings to exploit the media provider library.

With these principles in mind Grilo is being used to create a daemon that implements the provider side of the GNOME Media Server Specification. This daemon exposes the D-Bus APIs defined in this specification for application developers to consume. This is the Rygel-Grilo project (Rygel is the first consumer of this interface), a name we will change soon to something more representative. Just like Rygel, other GNOME applications can use these D-Bus interfaces to consume content exposed by this Grilo-powered daemon. The team already has plugins for Totem and Rhythmbox that we hope to push upstream soon.

It is important to note that the D-Bus APIs defined by the GNOME Media Server Specification are different from Grilo’s and are restricted to browsing and searching content while Grilo has a wider scope. It may turn out that some GNOME developers will still prefer to use Grilo directly. It is up to application developers to choose the approach that fits their needs better.

Future Plans

Nowadays our efforts are focused mainly on these areas:

  • Polish the APIs to support more use cases or fix limitations. Ideally, feedback from application developers should guide this work.
  • Complete and improve the plugins available, adding support for more features offered by the underlying services.
  • Provide GObject-introspection based bindings to be more developer friendly. Grilo is written in C_, but many application developers prefer to use more modern languages to code their applications. As we mentioned before, developers have the option to use _Rygel-Grilo and D-Bus from any language, although this approach has some limitations at the moment. Therefore having language bindings is still an important milestone for us. We are working hard on that and we expect to have related news very soon.
  • Integrate with the GNOME desktop. Rygel-Grilo and the Totem and Rhythmbox plugins are the first step in this direction. We are working on finishing a first version of these projects and then we will try to integrate them upstream. Another idea in this regard is to start moving plugins from these projects to Grilo, which would have two advantages: the plugins would become available for any other application using Grilo and all interested developers could cooperate in the maintenance work.
  • Help application developers to use Grilo successfully. This last bullet is particularly important since every framework developer needs feedback from application developers to improve and evolve the framework in the right direction.

Learn More

For general information about the Grilo project, please visit the wiki page or join us on IRC in the #grilo channel on the GIMPNet server or our mailing list.

For readers interested in learning more about Grilo and how they can use it to create modern media applications I suggest that they look at the documentation available in the project repository. This documentation includes a quick start guide for application developers that will get the interested reader started in a few minutes through a few simple code examples.

About the Author

Iago Toral is a partner and employee of Igalia where he works as Software Engineer in the Desktop and Mobile group.

Discuss this story with other readers on the GNOME forums.

Advertisements

Posted on September 1, 2010, in September 2010. Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: