Test aligned tiling with 3D schemes

[Dimitar Misev]

It appears that aligned tiling doesn't quite work as expected, creating more tiles than expected when using a 3D scheme.

[Dimitar Misev]

I think this has nothing to do with dimension, but with the tile size. If it is not specified, by default it is

"tileSize": "131072"

which takes precedence and restricts the tiling domain given. So for a 1000x1000x1 floating point array with default tile size we get this:

 "baseType": "marray <float>",
 "tileNo": "16",
 "totalSize": "4000000",
 "tiling": {
	"tilingScheme": "aligned",
	"tileSize": "131072",
	"tileConfiguration": "[0:999,0:999,0:0]"",
	"tileDomains":
	[
		"[320:639,960:999,0:0]",
		"[640:959,0:319,0:0]",
		"[640:959,320:639,0:0]",
		"[640:959,640:959,0:0]",
		"[640:959,960:999,0:0]",
		"[960:999,0:319,0:0]",
		"[960:999,320:639,0:0]",
		"[960:999,640:959,0:0]",
		"[960:999,960:999,0:0]",
		"[320:639,640:959,0:0]",
		"[320:639,320:639,0:0]",
		"[320:639,0:319,0:0]",
		"[0:319,960:999,0:0]",
		"[0:319,640:959,0:0]",
		"[0:319,320:639,0:0]",
		"[0:319,0:319,0:0]"
	]
 },

This tileSize is misleading, because it's not really size as in bytes, but number of cells covered by the domain.. as such we have no way to specify a default limit of 4MB for example that is independent of the cell type. We could set a default limit of 2^20 which would translate to 1MB tiles for char arrays, 2MB for short, 4MB for floats, etc.

[Dimitar Misev]

I'm guessing wcst_import.sh specifies by default the tiling scheme domain, but not tile size?

[Alex Dumitru]

WCSTImport supports any tiling that rasdaman supports, it basically sends the string mentioned in the tiling parameter directly to rasdaman. Usually the parameter is set to something like "ALIGNED [0:1000, 0:1000, 10]"
We can indicate to our users that they need to specify the tile size as well or any other option if we come to the conclusion that this is the expected behavior in rasdaman.

[Dimitar Misev]

Best if it is "ALIGNED .. TILE SIZE 2097152" I guess.

I don't even get it where is the default 131072 coming from though (no references in the code), must be something dynamically calculated.

[Dimitar Misev]

So the ingredients specify by default

"tiling": "ALIGNED [0:500, 0:500]"

I'd suggest to change it to

"tiling": "ALIGNED [0:1023, 0:1023] TILE SIZE 4194304"

[Alex Dumitru]

I will do this as a first step to alleviate the problems. Thanks for investigating. I'm assuming no solution for 3D objects using ALIGNED tiling, right?

[Dimitar Misev]

Yes, and I have to figure out where is the 131072 coming from.

Then I'll look at making it work right for 3D objects, but this will have to be similar to the regular tiling where the full tiles are initialized as soon as any pixel is touched.