Better compression schemes for GeoTIFFs is great and all - I'm not going to complain about that; however, I'd like to add a little more context from a data engineering perspective especially for data in transit.
As we all know, the best way to speed up some processes is by avoiding the step entirely, and as much as better compression schemes for GeoTIFFs shrink transport sizes, avoiding transfer of rasters reduces the cost to zero. Internally tiled GeoTIFFs (Cloud Optimized GeoTIFFs[1] in particular) allow portions of GeoTIFFs to be read using range reads instead. This can dramatically reduce transfer costs when only sections of the raster are required and pair well when used with vector data for look ups. In the specific case used by the author, omitting water-only tiles would potentially cut transfer costs by half.
Additionally, if using multi-spectral imagery, band re-ordering can be an important subsequent optimization if certain indices are known in advance to be used dis-proportionally more. There's a lot of fun to be had in optimizing the heck out of imagery data and this barely scratches the surface.
Ah, the "My Workstation" guy. I wonder if that text could be factored out and linked to in a sidebar or something (it's in every article I've seen from him, here).
As we all know, the best way to speed up some processes is by avoiding the step entirely, and as much as better compression schemes for GeoTIFFs shrink transport sizes, avoiding transfer of rasters reduces the cost to zero. Internally tiled GeoTIFFs (Cloud Optimized GeoTIFFs[1] in particular) allow portions of GeoTIFFs to be read using range reads instead. This can dramatically reduce transfer costs when only sections of the raster are required and pair well when used with vector data for look ups. In the specific case used by the author, omitting water-only tiles would potentially cut transfer costs by half.
Additionally, if using multi-spectral imagery, band re-ordering can be an important subsequent optimization if certain indices are known in advance to be used dis-proportionally more. There's a lot of fun to be had in optimizing the heck out of imagery data and this barely scratches the surface.
1. https://cogeo.org/