What is the bug or the crash?
In remote sensing, surface reflectance data (values between 0 and 1) is usually stored as integers with values between 0 and 10000 to save disk space. When setting a proper scale factor (i.e. 1/10000) in the raster, QGIS will on-the-fly apply the scale factor and visualize the data correctly between 0 and 1.
Everything works fine so far, but QGIS calculates and stores very large histograms with 10 mio bins for each band, consuming ~50 mb disk space per band. This is not optimal, but probably tolerable for most users. The main problem is, that this is also slowing down the visualization of the raster itself.
Steps to reproduce the issue
- Create a random raster with integer values between 0 and 10000.
- Use GDAL to set the band scale 1/10000.
- Visualize the raster in QGIS.
- Close QGIS.
- Look into the GDAL PAM *aux.xml file associated with the raster and find the histogram with 10 mio bins, e.g.
Versions
QGIS version
3.34.11-Prizren
QGIS code revision
2904bce
Qt version
5.15.13
Python version
3.12.6
GDAL/OGR version
3.9.2
PROJ version
9.4.0
EPSG Registry database version
v11.004 (2024-02-24)
GEOS version
3.12.2-CAPI-1.18.2
SQLite version
3.45.1
PDAL version
2.6.3
PostgreSQL client version
16.2
SpatiaLite version
5.1.0
QWT version
6.2.0
QScintilla2 version
2.14.1
OS version
Windows 10 Version 2009
Active Python plugins
BitFlagRenderer
0.5.20230630T235901.master
DataPlotly
4.1.0
ee_plugin
0.0.6
enmapboxplugin
3.15.1
geetimeseriesexplorer
2.0
qgis_resource_sharing
1.1.0-beta1
qgis_stac
1.1.2
quick_map_services
0.19.34
rasterdataplotting
1.6.3
rastertimeseriesmanager
1.8
zoomview
1.1.2
processing
2.12.99
Supported QGIS version
New profile
Additional context
The issue is related to a bug in the EnMAP-Box plugin, which I'm developing: EnMAP-Box/enmap-box#1028
We will implement a workaround for it on the Python side (storing precalculated histograms inside the raster).
We also report this issue here, because it needs to be handled on the C++ side as well.
What is the bug or the crash?
In remote sensing, surface reflectance data (values between 0 and 1) is usually stored as integers with values between 0 and 10000 to save disk space. When setting a proper scale factor (i.e. 1/10000) in the raster, QGIS will on-the-fly apply the scale factor and visualize the data correctly between 0 and 1.
Everything works fine so far, but QGIS calculates and stores very large histograms with 10 mio bins for each band, consuming ~50 mb disk space per band. This is not optimal, but probably tolerable for most users. The main problem is, that this is also slowing down the visualization of the raster itself.
Steps to reproduce the issue
Versions
QGIS version
3.34.11-Prizren
QGIS code revision
2904bce
Qt version
5.15.13
Python version
3.12.6
GDAL/OGR version
3.9.2
PROJ version
9.4.0
EPSG Registry database version
v11.004 (2024-02-24)
GEOS version
3.12.2-CAPI-1.18.2
SQLite version
3.45.1
PDAL version
2.6.3
PostgreSQL client version
16.2
SpatiaLite version
5.1.0
QWT version
6.2.0
QScintilla2 version
2.14.1
OS version
Windows 10 Version 2009
Active Python plugins
BitFlagRenderer
0.5.20230630T235901.master
DataPlotly
4.1.0
ee_plugin
0.0.6
enmapboxplugin
3.15.1
geetimeseriesexplorer
2.0
qgis_resource_sharing
1.1.0-beta1
qgis_stac
1.1.2
quick_map_services
0.19.34
rasterdataplotting
1.6.3
rastertimeseriesmanager
1.8
zoomview
1.1.2
processing
2.12.99
Supported QGIS version
New profile
Additional context
The issue is related to a bug in the EnMAP-Box plugin, which I'm developing: EnMAP-Box/enmap-box#1028
We will implement a workaround for it on the Python side (storing precalculated histograms inside the raster).
We also report this issue here, because it needs to be handled on the C++ side as well.