Opened 5 years ago

Closed 5 years ago

#2008 closed defect (fixed)

wcst_import GDAL general recipe handles an exceptional case when file exists in validating phase but not exist later in processing phase

Reported by: Bang Pham Huu Owned by: bph
Priority: major Milestone: 9.8
Component: wcst_import Version: 9.7
Keywords: Cc: Dimitar Misev, Vlad Merticariu
Complexity: Medium

Description

wcst_import before processing ingredient file, it checks if files from input file paths exist and can be opened by GDAL in case of using GDAL recipe. However, in an extremely exceptional case, the file after this validating phase disappears and wcst_import will have a problem when it tries to get data from it for creating subsets.

If "skip": true, then wcst_import should ignore the exception when processing the input file instead of throwing exception.

An error has occured in the execution of the program. Error Message: `/eodata/Sentinel-2/MSI/L1C/2018/12/18/S2A_MSIL1C_20181218T140041_N0207_R067_T21LVC_20181218T153957.SAFE//GRANULE/L1C_T21LVC_A018225_20181218T140043/IMG_DATA/T21LVC_20181218T140041_B03.jp2' does not exist in the file system,
and is not recognised as a supported dataset name.

Stack Trace: Traceback (most recent call last):
  File "/opt/rasdaman/share/rasdaman/wcst_import/wcst_import.py", line 172, in main
    reg.run_recipe(session)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/master/recipe/recipe_registry.py", line 197, in run_recipe
    self.__run_recipe(session, recipe)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/master/recipe/recipe_registry.py", line 150, in __run_recipe
    recipe.describe()
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/recipe.py", line 168, in describe
    importer = self._get_importer()
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/recipe.py", line 647, in _get_importer
    self.importer = Importer(self.resumer, self._get_coverage(), self.options['wms_import'], self.options['scale_levels'],
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/recipe.py", line 561, in _get_coverage
    coverage = self._get_gdal_coverage(recipe_type)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/recipe.py", line 588, in _get_gdal_coverage
    self.options['import_order']).to_coverage()
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/abstract_to_coverage_converter.py", line 419, in to_coverage
    coverage_slices = self._create_coverage_slices(crs_axes)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/abstract_to_coverage_converter.py", line 373, in _create_coverage_slices
    coverage_slice = self._create_coverage_slice(file, crs_axes, evaluator_slice, axis_resolutions)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/abstract_to_coverage_converter.py", line 402, in _create_coverage_slice
    axis_subset = self._axis_subset(crs_axes[i], evaluator_slice, resolution)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/gdal_to_coverage_converter.py", line 124, in _axis_subset
    evaluator_slice)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/recipes/general_coverage/abstract_to_coverage_converter.py", line 98, in _user_axis
    min = self.sentence_evaluator.evaluate(user_axis.interval.low, evaluator_slice, user_axis.statements)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/master/evaluator/sentence_evaluator.py", line 66, in evaluate
    expression_result = evaluator.evaluate(expression, evaluator_slice)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/master/evaluator/gdal_expression_evaluator.py", line 60, in evaluate
    return self._resolve(expression, evaluator_slice.get_dataset())
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/master/evaluator/evaluator_slice.py", line 72, in get_dataset
    self.dataset = GDALGmlUtil(self.get_file().filepath)
  File "/rasv1/opt_rasdaman/rasdaman/share/rasdaman/wcst_import/util/gdal_util.py", line 48, in __init__
    self.gdal_dataset = gdal.Open(str(self.gdal_file_path).encode('utf8'))
  File "/usr/lib/python2.7/dist-packages/osgeo/gdal.py", line 1800, in Open
    return _gdal.Open(*args)
RuntimeError: `/eodata/Sentinel-2/MSI/L1C/2018/12/18/S2A_MSIL1C_20181218T140041_N0207_R067_T21LVC_20181218T153957.SAFE//GRANULE/L1C_T21LVC_A018225_20181218T140043/IMG_DATA/T21LVC_20181218T140041_B03.jp2' does not exist in the file system,
and is not recognised as a supported dataset name.


Change History (1)

comment:1 by Bang Pham Huu, 5 years ago

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.