Opened 7 years ago

Closed 5 years ago

#1619 closed defect (invalid)

subspace queries with normal vectors that have 0 as the last component sometimes return incorrect results

Reported by: bbell Owned by: bbell
Priority: major Milestone: Future
Component: undecided Version: development
Keywords: Cc: Dimitar Misev
Complexity: Medium

Description

the following query is an example:

select encode(subspace(c, (0 10 0, 70 90 0, 70 90 2)),"csv") from test_grey3d as c

The unit normal vector to the subspace defined here is approximately (-.75, .65, 0), and the LU solver fails for most result values, even though sdom(test_grey3d) = [0:99,0:99,0:4]. The error is occurring because a division by 0 happens in the method qtfindsection.cc::solveLU(), and due to floating point arithmetic errors, this should be treated as a separate case, the resolution of which would result in speeding up such queries.

Change History (1)

comment:1 by Dimitar Misev, 5 years ago

Resolution: invalid
Status: newclosed
Note: See TracTickets for help on using tickets.