[music-dsp] Getting really serious with sound models

[Aaron Oxford]

Hi all,

Long intro to this post but I have to give some credit around the 
place and set the scene, so bear with me. :-)

HUGE thank-you to everyone who answered any of my previous posts 
about wavelet analysis (or indeed any of my posts dating right back 
to around Y2K). I'm very *very* happy with where I'm at right now. 
Big ups to everyone, even just for putting up with me. :-)

Have a close listen to 
http://downloads.sourceforge.net/buzz-like/056.mp3. It would please 
me and my project ranking greatly ;-) and you can hear the successive 
approximations of the guitar riff with wavelets being played back and 
spectrally panned in real-time - a possibility I extend as open 
source to the community that was born of this list and its many 
wonderful members. BTW I would be happy to describe my algorithm for 
the MusicDSP archives, if anyone thinks anyone would ever be interested. :-p

Down to business now. Having got wavelet analysis going, I can 
clearly see (or rather hear) that what is missing from a *really* 
sweet pitch/time scaling algorithm, the holy grail of being able to 
say this *is* this instrument/sound in a digital model, is the 
ability to detect fundamental relative periodicity of harmonic 
bursts. I'm talking in wavelet terms here, because so far I'm not 
sure how else to put it.

I believe this has been referred to elsewhere as 'formants'. What I'm 
talking about is the little squigglies (harmonics, probably) that 
occur at the peaks of the big squigglies of most 'real' instruments. 
When you analyse the sound from a purely wavelets point of view, the 
small squigglies occur at a given *time*, not with a given *period*. 
When you do a trivial pitch/time scale, you get artifacts because of 
this - you aren't placing the 'formants' at each wavecrest, you're 
simply placing them at (wrongly) scaled times.

One can envision a blade server capable of not only doing the days of 
wavelet analysis that I've been doing but also the cross-referencing 
needed to detect that a certain frequency occurs in bursts relative 
to another frequency (as always I'm amazed at my ears and pray to go 
blind long before going deaf), and perhaps one day I will explore 
this space. But for now I'm looking to go a bit lateral - somewhere 
new. Either that or take a shortcut to what I previously described LOL. :-)

What I'd like to know is where people have gone beyond wavelets, both 
in terms of analysing these 'formants' (many apologies if my 
terminology is off) and in terms of completely alternative ways of 
viewing sound - so far I know about AM, FM, Fourier, and wavelets - 
my exploration of the latter uses a far from orthogonal basis, and 
even that fascinates me in a small way. Wavelets are just one of the 
ways of looking at sound data, and they've brought me and my app a 
long way towards being able to handle all kinds of multimedia data in 
all kinds of mathematical frameworks, but of course they are also 
limited in various ways just like any other space will be.

So has anyone here ever looked into other interesting or cool ways to 
represent audio? I thought about FM, but I can't see that it would be 
tractable for any useful purpose than making chirps and burps. How 
does one make an EQ or cross-delay in FM land? I'm probably just 
showing my lack of knowledge there, and even having said that people 
have asked the same thing about wavelets and e.g. compression, and it 
hasn't stopped me and my imagination from making godawful noise to 
this day. :-)

Hoping this leads to some (more) very interesting discussions,

Aaron.
---------------------------------------------------------------------------------
Aaron Oxford   -   aaron+hardwarehookups .com .au
Director, Innovative Computer Solutions (Aust) Pty. Ltd.
49 Maitland Rd, Mayfield, NSW 2304 Australia
http://www.ic-solutions.com.au
Developer, SourceForge project VioLet Composer
http://sourceforge.net/projects/buzz-like