TIFF to GPU memory via cog3pio backend entrypoint

[weiji14]

Read TIFF data into GPU memory inside an xarray data structure via cog3pio's experimental CudaCogReader struct that uses nvTIFF as its backend. Based on a proof-of-concept I got working at weiji14/cog3pio#71. Would like it to live in cupy-xarray instead 😃

Need to install cog3pio with the 'cuda' feature flag enabled like so to get the:

MATURIN_PEP517_ARGS="--features cuda,pyo3" pip install  -v "cog3pio[cuda] @ git+https://github.com/weiji14/cog3pio.git@178a3ffb8163c97f7af9e71bc68b6545a4e8e192"

Notes:

• No direct-to-gpu is happening. As far as I can tell, compressed data still goes through CPU RAM, and is sent to the GPU to be decompressed with nvCOMP, which is all handled by nvTIFF
• Decoded output is a 1-D array, so need to do some reshaping to 3-D (Channels, Height, Width). May need to double check dim order for different TIFF planar configurations
• NaN data is not handled properly yet

TODO:

• Initial implementation
• Add more unit tests
• Refactor to use rasterix for coordinate labels?
• Document installation better (might need special handling for nvTIFF)
• Polish API documentation with proper intersphinx links
• Create tutorial showing example usage
• Remove circular dependency on cog3pio side requiring cupy-cuda13x

References:

• https://discourse.pangeo.io/t/decode-geotiff-to-gpu-memory

[jacobtomlinson]

How would this be handles on a multi-GPU system? You may want to load many tif files into a dask-cupy-xarray object where different chunks are on different GPUs. This API feels a little inflexible for this use case.