Opened 2 years ago

Closed 2 years ago

#2592 closed defect (fixed)

wcst_import - all RAM occupied with large GRIB

Reported by: Bang Pham Huu Owned by: Bang Pham Huu
Priority: major Milestone: 10.1
Component: wcst_import Version: 10.0
Keywords: Cc: Dimitar Misev, Vlad Merticariu
Complexity: Medium

Description

With a large GRIB file with 2977 messages (4 bands), wcst_import analyzes messages up to message 1823th and it occupied all 18 GB of ram, then wcst_import is killed.

In grib_to_coverage_converter.py it has a for loop to iterate over all messages and this needs to enhance to not keep all the collected message:

        for i in range(1, self.dataset.messages + 1):
            ...
            band_name = grib_message["shortName"]
            if band_name == band.identifier:
                evaluated_messages.append(GRIBMessage(i, axes, grib_message))

            collected_messages.append(GRIBMessage(i, axes, grib_message))

Change History (2)

comment:1 by Bang Pham Huu, 2 years ago

Cc: Dimitar Misev Vlad Merticariu added

comment:2 by Bang Pham Huu, 2 years ago

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.