WCST-import tried to import empty coverage when "subset_correction" is false with geo-referenced image

[Bang Pham Huu]

I've checked with current wcst-import from newest rasdaman code. It has a problem with importing georeferenced image if "subset_correction": false. Correctly as Alex's temporary solution it should be true and if set to false then when user run wcst_import.sh there should be an error.

However, even when I set "subset_correction": false and import file, wcst-import still run and create an empty collection in Petascope (it has GML metadata but no data).

 <limits>
          <GridEnvelope>
            <low>0 0</low>
            <high>0 0</high>
          </GridEnvelope>
        </limits>

Here is what wcst-import is running (it should stop when it has tried first time).

 '/home/rasdaman/rasdaman1/rasdaman/applications/wcst_import/wcst_import.py' /home/rasdaman/ingredient.txt


Running
Progress: [------------------------------] 0/1 0.00% 
Exception thrown when trying to insert slice: 
{Axis Subset: Lat(20.0,80.0) Long(-110.0,-60.0) 
Data Provider: file:///home/rasdaman/test_1.tif}
Retrying, you can safely ignore the warning for now. Tried 1 times.

Progress: [------------------------------] 0/1 0.00% 
Failed to insert slice. Attempted 1 times.
Skipped slice {Axis Subset: Lat(20.0,80.0) Long(-110.0,-60.0) 
Data Provider: file:///home/rasdaman/test_1.tif}

Progress: [##############################] 1/1 100.00% Done.

The problem is solved when "subset_correction": true.

This should be fixed to avoid misunderstanding.

[Bang Pham Huu]

Can we do this just by checking the coordinates of coverage if it is float and user use "subset_correction": false then it will should raise an error? From my views, your configuration only have effect when user want to import with coordinates in floating points like (2.5, 3.2, 4.512, 6.2324, 123.434324234). I need your confirm before I could do anything.

[Bang Pham Huu]

@Alex: I think it is best to remove this configuration outside of recipe if my assumption is true (it only has good effect with file with coordinates is float). I suppose if this is set to "true" then you will make a correction to make it suitable to import by wcst-import. So we can avoid this by let user don't need to take care about this by my idea is checking the coordinate is float or integer (rounded number). I would happy when can get your decision.

[Alex Dumitru]

The subset_correction feature is needed when there is a discrepancy between the precision of gdal and the precision of petascope. Take for example this coordinate 0.33333333, this would be represented in gdal as 0.3333333374353 for example due to the loss of precision in floating point values. Petascope uses BigDecimal that keeps the precision up to 50 decimals.

When working with arithmetic operations, the difference between the two leads to inconsistencies that cannot be detected automatically. WCST Import tries to help with that by giving the users the option of trying to ingest the coverage by adding a small number (⇐ offset_vector / 2) to the coordinates in order to correctly align the subsets. However this is not guaranteed to work, for example if your extent is close to the next geopixel by less than ½ of the offset vector. In this cases it is left to the user to deal with the incosistency (e.g. by manually creating the coverage)

In conclusion, this is just a helper for certain edge cases that cannot be detected automatically. It does *not* work for any floating point coordinate and should not be enabled by default.

[Dimitar Misev]

Hey, could you add this text to the wcst_import guide?

[Alex Dumitru]

Good catch, I will add it on the page, thanks!

[Bang Pham Huu]

very detail and could give me the inside that is what I want to hear from you. Many thanks, yes, it should be added in WCST_import page (I think this page is really important (so the more information, the less questions like this but more advanced questions from user when they know how to use correctly) due to the wcst_import is the key tool when using Rasdaman to import data).

[Alex Dumitru]

I added it there under a FAQ section, feel free to add other things that seem to not be clear from the documentation.