SIMD/Uses/Conversion

From MozillaWiki
< SIMD
Revision as of 19:23, 12 April 2014 by Maikmerten (talk | contribs) (Initial discussion on float/int conversion)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

In multimedia coding, converting data types is not uncommon. For instance, in JPEG and MPEG the reference DCT is operating on double precision intermediates, but the resulting coefficients are stored as integer values. Converting floating point values to integer types looks innocent enough written in C:

int i;
double d = 0.99;
i = (int) d;

The resulting value for a is 0, as casting in C will truncate any positions after the decimal point -- no rounding will occur. The same is true for JavaScript. (In JavaScript, the bitwise operators such as | will trigger an integer conversion.)

To get proper rounding, following approach is often employed:

if(d > 0) {
   i = (int) d + 0.5;
} else {
   i = (int) d - 0.5;
}

often written in a more compact fashion using a ternary operator:

i = (int) d > 0 ? (d + 0.5) : (d - 0.5);