[music-dsp] Kullback-Leibler distance measure?

Ethan Duni eduni at ucsd.edu
Wed Jan 21 14:49:02 EST 2004


KL Distance measures the "distance" (in bits usually) between two
probability distributions. The interpretation is that, if one of the
distributions if the "true" distribution, the KL distance is the number of
extra bits you'd have to use if you coded the variable using the second
distribution. Note that it is non-symmetric, in that you will get different
answers if you swap the "true" distribution and the other one. It is simply
an analytic formula, so how you'd actually implement it in a computer
depends on what sort of description of the distributions you have available
in your implementation and how much accuracy you'll need. For example, it
can be approximated by stochastic integration. I have no idea how it would
come up in speech synthesis, but good info can be found in Thomas and
Cover's "Elements of Information Theory" or by googling for
"Kullback-Leibler Divergence" (this is the more common name for it).

Ethan

----- Original Message ----- 
From: "Mark Greenslade" <mark at speech.fsnet.co.uk>
To: <music-dsp at shoko.calarts.edu>
Sent: Wednesday, January 21, 2004 1:37 PM
Subject: [music-dsp] Kullback-Leibler distance measure?


Hi,

Does anyone have any idea how to do a kullback-leibler distance measure?

apparently, It has been used to measure spectral discontinuities across
formant boundaries in unit selection for speech synthesis but I can't find
code or info.

Any help much appreciated.

Thanks

Mark




More information about the music-dsp mailing list