51
edits
Changes
Initial discussion on float/int conversion
In multimedia coding, converting data types is not uncommon. For instance, in JPEG and MPEG the reference [[SIMD/Uses/DCT |DCT]] is operating on double precision intermediates, but the resulting coefficients are stored as integer values. Converting floating point values to integer types looks innocent enough written in C:
int i;
double d = 0.99;
i = (int) d;
The resulting value for a is 0, as casting in C will truncate any positions after the decimal point -- no rounding will occur. The same is true for JavaScript. (In JavaScript, the bitwise operators such as | will trigger an integer conversion.)
To get proper rounding, following approach is often employed:
if(d > 0) {
i = (int) d + 0.5;
} else {
i = (int) d - 0.5;
}
often written in a more compact fashion using a ternary operator:
i = (int) d > 0 ? (d + 0.5) : (d - 0.5);
int i;
double d = 0.99;
i = (int) d;
The resulting value for a is 0, as casting in C will truncate any positions after the decimal point -- no rounding will occur. The same is true for JavaScript. (In JavaScript, the bitwise operators such as | will trigger an integer conversion.)
To get proper rounding, following approach is often employed:
if(d > 0) {
i = (int) d + 0.5;
} else {
i = (int) d - 0.5;
}
often written in a more compact fashion using a ternary operator:
i = (int) d > 0 ? (d + 0.5) : (d - 0.5);
