Condense expressions with where clause fail

[Dirk Daems]

I am having trouble adding a where clause to a condense operation.

This works (with expected output):

for a in (mean_summer_airtemp)
  return condense +
  over $x x(500:505),
       $y y(500:505)
  using a[x:"CRS:1"($x),y:"CRS:1"($y)]

But this doesn’t (only the where clause was added):

for a in (mean_summer_airtemp)
  return condense +
  over $x x(500:505),
       $y y(500:505)
  where (a[x:"CRS:1"($x),y:"CRS:1"($y)] > 0)
  using a[x:"CRS:1"($x),y:"CRS:1"($y)]

Error in the Petascope log file:

[16:38:17] ERROR PetascopeInterface@436: Error stack trace:
RuntimeError: Runtime error while processing request: Error translating parsed abstract WCPS query to XML format.
	at petascope.PetascopeInterface.doGet(PetascopeInterface.java:391)
	at petascope.PetascopeInterface.doPost(PetascopeInterface.java:214)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:679)
Caused by: InternalComponentError: Error translating parsed abstract WCPS query to XML format.
	at petascope.util.ras.RasUtil.abstractWCPStoXML(RasUtil.java:232)
	at petascope.PetascopeInterface.doGet(PetascopeInterface.java:320)
	... 15 more
Caused by: java.lang.NullPointerException
	at petascope.wcps.grammar.ProcessingExpr.toXML(ProcessingExpr.java:40)
	at petascope.wcps.grammar.ReturnClause.toXML(ReturnClause.java:40)
	at petascope.wcps.grammar.WCPSRequest.toXML(WCPSRequest.java:71)
	at petascope.util.ras.RasUtil.abstractWCPStoXML(RasUtil.java:229)
	... 16 more
[16:38:17] DEBUG PetascopeInterface@487: Done marshalling Error Report.

[Piero Campalani]

The first problem is that "(a[x:"CRS:1"($x),y:"CRS:1"($y)] == 0)" is apparently not recognized as a boolean expression:

DEBUG [17:10:35] WcpsServlet@163: WCPS: received the following  query via a 'query' parameter in a POST request:
 DEBUG [17:10:35] WcpsServlet@165: for a in (mean_summer_airtemp)
  return condense +
    over $x x(500:505),
         $y y(500:505)
    where (a[x:"CRS:1"($x),y:"CRS:1"($y)] = 0)
    using a[x:"CRS:1"($x),y:"CRS:1"($y)]
 TRACE [17:10:35] RasUtil@219: Parsing abstract WCPS query...
line 3:4 no viable alternative at input 'over'

Otherwise, simply try:

for a in (mean_summer_airtemp)
   return (a[x:"CRS:1"(100),y:"CRS:1"(300)] = 0)

Looking at the boolean expressions' handling in petascope.wcps.grammar.wcps.g, I cannot find a clear correspondence to the WCPS standard grammar definition.

Secondly, replacing it with a direct TRUE boolean constant, while the WCPS parser recognizes it, it is not properly translate to RasQL:

Error evaluating rasdaman query: 'select condense + over i_i in [500:505, 500:505]TRUE using (a) [i_i[0],i_i[1]] from mean_summer_airtemp AS a

There is no WCPS system test for the WHERE clause in a general condense operation.

[Dimitar Misev]

Submitted patch to fix the translation of the WHERE to rasql. There's the same issue as in #372, so the query actually would look like this:

for c in (mr)
  return condense +
  over $x x(0:100),
       $y y(0:100)
  where (max(c[x($x:$x),y($y:$y)]) < 100)
  using c[x($x),y($y)]

[Dimitar Misev]

Fixed in changeset:2a07e3b , for the other we have #372.

[Dimitar Misev]

This wasn't completely fixed, as it's translating to

select condense + over i_i in [0:100, 0:100] where (max_cells(  c[0:0,0:0]  ))<(100) using (c) [i_i[0],i_i[1]] from mr AS c

Fixing ticket #335 has fixed it, as now it translates to

select condense + over i_i in [0:100, 0:100] where (max_cells(  c[i_i[0]:i_i[0],i_i[1]:i_i[1]]  ))<(100) using (c) [i_i[0],i_i[1]] from mr AS c

[Dirk Daems]

Which version fixed this issue? I still get the same error message using 9.0-pre2.

[Dimitar Misev]

The original issue has been fixed since 6 months ago, query in the systemtest that tests it is source:systemtest/testcases_services/test_wcps/queries/61-general_condense_where-t371.test and the expected result is source:systemtest/testcases_services/test_wcps/oracle/61-general_condense_where-t371.test.oracle

It's not possible to have again the exception as that tests passes since long time ago. Is it exactly the query above that fails for you?

[Dimitar Misev]

As I've written in comment:2 there's a bug in #372 which doesn't recognize point query as a scalar, so you need to use something like max/min in the using clause.

So this ticket is fixed, the issue you have is because of ticket #372, and workaround is in comment:2