[linux-audio-dev] plugin GUIs .. Mediastation, LADSPA ?
sbenno at gardena.net
Tue Nov 18 20:35:00 EST 2003
Disclaimer: I have not read the entire GUI thread so please don't flame
me (but correct me)
if I say nonsense.
Having written a few engines (midi player, audio player) and the
corresponding GUIs of
the upcoming Lionstracs Mediastation keyboard, we have faced the same
problem old problem:
we wanted the GUIs decoupled from the engines, perhaps allowing
controlling the engine
by more than one GUI at the same time where all GUIs automagically
when the engine (or other GUIs) change its parameters.
I decided to adopt the simplest possible protocol perhaps it is a bit
to the most elegant solution but I think the simplicity of my model
cannot be beaten that easily.
I'd like you folks to read the brief explanation of my protocol and make
comments about it,
eg. if an even more simple solution exists, or if it has serious flaws
(I don't think so :-) )
Let's explain a simple scenario:
I midi player which consists of a GUI-less engine and a GUI.
It's basically a client/server system: GUIs are clients, engines are
The midi player accepts commands like
start, stop, seek, get_engine_status
the method I use to communicate between the GUIs is SYSV message queues
because they can be multiclient but the API is abstracted so the
model can be chosen arbitrarily.
the server does:
// opens a message port in server mode (creates the port)
// key is needed to identify the port by clients that
// will connect to
int mcmd_open_server_port(int key, mcmd_info_t *mcmd_info);
key is a machine wide unique identifier.
in my implementation it's just the SYSV IPC message queue key,
but this is not mandatory, the API can easily be adapted to use
strings (eg for TCP/IP support which I will add soon).
now within the main loop the server does:
// receives a message. If no message is pending in the queue
// wait until message was received
int mcmd_receive(mcmd_info_t *mcmd_info, void *buffer, int buflen)
and then based on the type of message received it performs some action
and sends back a response to the client which can be a simple ACK or
a message containing some payload
it does so using this function:
int mcmd_send(mcmd_info_t *mcmd_info, void *buffer, int buflen, int
The destination is an unique identifier of the client that sent the message
So basically each server and client do have unique identifiers
which in my implementation are mapped to SYSV IPC struct msgbuf.mtype
(but this is only an implementation detail, as said the communication
layer can be
any form of IPC)
I have the mcmd_receive_nowait() call too which can be directly placed
within a high
priority thread that does other stuff too (eg a midi/audio playing thread).
SYSV msg queues seems quite fast a couple of usecs (or dozen of usecs)
per call so
perhaps one could probably even put msgrcv() calls into an audio
processing loop that does
process audio fragments of 1-2msec. Not sure if there is some locking
that might cause stalls.
If you are paranoid you could let an audio player accept external
commands via lower priority
thread and then pass these commands to audio thread via lock-free FIFO,
or even better
use lock-free fifos as the transport method of the mcmd_ API.
So far so good, let's continue with the MIDI player scenario:
The GUI user wants to load a midi file.
the command sent to the server is load_file file.mid
the server executes it (loads the file.mid) and then responds
with an ACK. (with a status code so that the GUI knows if loading succeded).
Same for start,stop, change volume of midi channel, etc commands.
Now let's see how the GUI updates itself: we want the time display to
and midi volume sliders update automagically.
Basically the GUI has a timer callback that is called 10-20 times/sec.
It sends a GET_ENGINE_STATUS message to the server
and the server responds with a ENGINE_STATUS message.
this message (you can define your own arbitrary message structs because the
protocol does not know anything about the payload of messages.
This engine status message should contain the values that should
exported to the GUI
in the midi player's case the current position (midi ticks), current
values of midi volumes,
muted channels and so on.
If the payload gets too big then export flags that signal that some
parameters in the engine
changes and then let the GUI request an additional info packet that
contains the values
The guideline should be to minimize the number of messages exchanged so
specific case I have chosen to put all parameters needed in single
Of course the GUI should do something like this to avoid unnecessary
of elements (which could lead to a slow, flickering GUI)
if(curr_volume != old_volume) update_volume_slider(curr_volume);
In my example midi player you see the volume sliders move when the there
changes in the player engine because the GUI is constantly fetching
If you attach a second GUI to the engine both GUIs update perfectly in sync.
If you move a slider on the first GUI it will move in synth on the
second GUI too.
(because GUI1 moves the slider, sends a set_volume message to the engine
the next gui update cycle of GUI2 this value is fetched by GUI2 and the
slider is moved)
Perhaps you might say this protocol is suboptimal because it sends 10-20
even if the values do not change so it would be better to let the server
wake up the clients
waiting for data.
Yes it would probably be more elegant but would make it more complex and
gain would be almost zero. (IPC message queues are capable of sending
tens/hundreds of thousands
of msgs/sec so 10-20/sec used by my API is a really negligible when
measuring the overall performance.
Plus think about it in the case of a audio/midi player gui the song
pointer position must be updated
frequently (at least 10-20 times/sec so that it updates smoothly on
screen) so even if in the case
where the engine wakes up the GUI the amount of messages that pass per
time unit are the same.
We will use the same API for LinuxSampler too because in that case we
want a fully detached GUI
and multiple ways to control it (who knows if LinuxSampler will some day
run in expanders that
are made of an embedded x86 with a few buttons and a small LCD ? :-) ).
I'm currently adding TCP/IP support too via a proxy module which allows
you to run the GUIs
on remote hosts and manage multiple engines (the headless midi player,
sampler etc) through
a single TCP port (so that in case of firewalled hosts you need to
open/forward only a single TCP port
even if you run multiple engines on the host).
The protocol will be published soon (probably via LinuxSampler CVS,
I'll keep you posted when it's ready).
The question is: would it make sense to use such a protocol for LADSPA ?
For example LADSPA could contain a definition that tells the host what
kind of parameters it must export to the GUI
via the message passing API.
At this the GUI can be implemented using any kind of GUI toolkit and the
above message passing API.
The GUI would run in a separate process so there would be no conflicts
with the host
and you get GUI automation for free: just change the internal values in
the host and the GUI will automatically update
In conclusion: do you see any drawbacks in using this stuff in LADSPA ?
Better solutions ?
If you ask me the GUI toolkit mixing is a kludge and keeping the GUI
stuff out of LADSPA by separating the two
things via a client/server model can improve code quality, because you
are not allowed to mix GUI stuff and engine stuff
in the same thread thus forcing the programmer to be a bit more disciplined.
Yes, it is a bit more work to write apps that way but the reward is
quite big, trust me.
I think without a client/server model writing the software for
Mediastation would be a big PITA.
Perhaps the protocol is suboptimal but perfectly suits our needs.
Steve Harris wrote:
>On Tue, Nov 18, 2003 at 02:50:37 -0500, Paul Davis wrote:
>>if we avoid the goal of having the host have some control over the
>>plugin GUI window, then this isn't necessary, and the design i
>>implemented last night will work without any special support from X or
>>toolkits. it does require a small library with what steve has termed
>>"toolkit specific hacks" - its not so much that as enumeration of
>>supported toolkits. however, the goal of the host having some control
>>over the plugin GUI window seems rather desirable.
>If the host has no control over the UI then I'm not quite sure what the
>point is. It saves on processes, but the're pretty cheap on UNIX anyway.
>We need some kind of IPC (ITC?) between the host and UI anyway, to handle
>serialisation of control changes.
More information about the linux-audio-dev