[music-dsp] 32 bit floats >>> 24 bit integer
r_obert at hotmail.com
Sun Jan 22 17:09:53 EST 2006
> -----Original Message-----
> From: music-dsp-bounces at ceait.calarts.edu
> [mailto:music-dsp-bounces at ceait.calarts.edu] On Behalf Of Nigel Redmon
> Sent: Sunday, January 22, 2006 3:02 PM
> To: music-dsp
> Subject: Re: [music-dsp] 32 bit floats >>> 24 bit integer
> Despite these quirks (truncation in floating point versus
> integer and various types of rounding possibilities) that
> make it more difficult to directly compare truncation versus
> rounding, your assertion that rounding isn't a significant
> improvement over truncation is correct-- they are simply
> different methods of spreading the conversion error.
I disagree. Clearly the type of truncation that goes from something like
5.9 to 5.0 is going to produce a statistically larger magnitude of error
over time then rounding from 5.9 to 6.0. Anytime the truncation exceeds
0.5, it is worse. Plus it's always biased in one direction.
( Or at least this is how I see it ).
> You'll find that in practice, people use whichever gets them
> the conversion the fastest. (If dither first, you still have
I pretty much always use rounding. Yes, truncation will be a small amount
quicker than rounding. But if the small amount of extra time spent rounding
dooms you, you're in sorry shape anyhow ( in terms of CPU utilitzation ).
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.14.21/236 - Release Date: 1/20/2006
More information about the music-dsp