Opened 4 years ago

Closed 4 years ago

#2216 closed defect (fixed)

Casting from larger to smaller integer type

Reported by: Dimitar Misev Owned by: Dimitar Misev
Priority: major Milestone: 10.0
Component: catalogmgr Version: 9.8
Keywords: Cc: Peter Baumann, Vlad Merticariu
Complexity: Medium

Description

When casting if the new type is smaller than the value's type, i.e. not all values can
be represented by it, then standard C++ casting will typically lead to
strange results due to wrap around for unsigned and implementation-defined
behavior for a signed types. For example, casting int 1234 to char in C++
will result in 210, even though the possible range would be 0 - 255.

Rasdaman implements a more reasonable cast behavior in this case: if the
value is larger than the maximum value representable by the new type, then
the result is the maximum value (e.g. 255 in the previous example);
analogously, if the value is smaller than the minimum possible value, then
the result is the minimum value.

This is implemented only on integer types and entails a small performance
penalty in comparison to raw C++ as up to two comparisons per cell (with
the maximum and minimum) may be necessary when casting.

This behavior was implemented only partially for char and short when casting from unsigned, and should be done uniformly for all integer types.

Change History (1)

comment:1 by Dimitar Misev, 4 years ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.