Fun with GStreamer Audio Effects
Stefan Kost describes GStreamer features that have been implemented and that are in the works, and he steps users through setting up an example with which to play.
Recent changes make the framework more useful for creative media processing. GStreamer-0.10.12.1 (cvs) got latency handling. This allows realtime effects on live sources (alsasrc ! effects ! alsasink). Also, lots of work went into flexible data format support [int (8/16/32 bit) and float (32/64 bit)] in audio processing effects. Finally, the new jacksrc provides a bridge between the desktop applications and the professional Linux Audio world. Using the jackd daemon allows routing audio signals between applications.
What kind of effects are there?
The package gst-plugins-base mostly provides elements to convert or adapt properties such as number of channels, formats (float/int and bit-depths) and sampling rate.
In gst-plugins-good is the audiofx plugin, that brings audiodynamic, audioamplify, audioinvert, and audiopanorama elements. There is also a peak-level meter.
In gst-plugins-bad is the equalizer plugin with several equalizer variants and a spectrum analyzer. Currently, the LADSPA wrapper is also there. LADSPA is the Linux Audio Developers Simple Plugin API. On my Linux system, I have 231 elements installed, and the GStreamer LADSPA plugin makes those available for use in GStreamer.
The code repository of the Buzztard project brings more elements. First, there is a gst-buzztard module which provides some experimental GStreamer extensions, a simple software synthesizer (simsyn), and an effect called audiodelay. Second, one can install the bml, gstbml modules. This provides a wrapper like LADSPA, but this time for so called “buzz machines.” These are win32/x86 DLLs of a discontinued freeware music composer called “buzz.” Buzz comes with more than 500 free sound generators and effects. The gstbml wrapper makes most of them available as GStreamer elements. Please note that these modules are a bit more difficult to set up. If you have problems, help is just one click away.
Lets get practical! As a shameless act of self-advertisement, I recommend you to install buzztard from CVS if you want to try the examples yourself. The example also needs an up-to-date GStreamer from CVS. For the not-so-fearless, I have some screenshots:
Start the graphical editor “bt-edit.” The first tab shows the machine view. Here, you can create elements and link them together. Buzztard automatically adds conversion elements on-the-fly. To create an element, right-click the background and select an element from the context menu. For the example, choose alsasrc. Not all the elements in the list will work perfectly in a live situation at this time. In your soundcard mixer, enable recording from the microphone or provide some music on line-in. Next, add audiodelay and equalizer-10band elements. Then, wire them together by pointing on the source element (alsasrc) and dragging a wire to the target (audiodelay) while holding the shift key. The wire should become green when the mouse is over a valid target. Releasing the mouse creates the link. Repeat the procedure for audiodelay -> equalizer and equalizer -> master. The wires have a triangle in the middle showing the direction. A left click onto it allows you to change the volume of the link. A right click shows a context menu for disconnecting the elements and for showing an “analyzer dialog.” Open that option as needed for the link between equalizer and master. Finally, double-click the equalizer and the audiodelay to open their realtime settings windows. Then, you should see something like this:
Lets give it a try! Activate the loop in the toolbar and press play. You can tweak the settings of the effects while making some noise. The parameter-changes are effective right away. In the “analyzer window,” you can see the frequency spectrum of the sound and the volume level. Tweaking the knobs is nice to try out settings, but that is not all there is. Buzztard will be a music composer, and in the future, it will be able to record those parameter changes into patterns from which one can build a song. Currently, recording is not yet well-supported by the user interface.
As shown above, the functionality is there. Adding and porting more effects will be a matter of time and manpower. The LADSPA wrapper already provides access to numerous plugins. Still there is room to improve the wrapper. The GStreamer API is richer than the LADSPA API. Thus it might make sense to add more metadata to known plugins, to improve their usefulness for GStreamer applications. More effects and features are planned as a Google Summer of Code proposal. Unfortunately, GStreamer has not been accepted by Google as a project. Nonetheless, we will work on the effects.
An important point for me is to continue discussing the Buzztard GStreamer extensions with the community and make them part of GStreamer once they are ready. To give an example: we currently work on a unified preset handling interface that works across applications. If your interested or have questions, feel free to ask in GStreamer IRC.
About the author
Stefan Kost works on the multimedia software used on the Nokia Internet Tablets. In his spare time he programs the Buzztard music composer and contributes to GStreamer. From time to time he composes music and writes lyrics for his band ekso:r.