2

I want to merge 2 large raster files (7GB+8GB) I got error messages, with a final

Unable to allocate 14.2 GiB for an array with shape (53701, 35412) and data type int64

I already manage to generate a 13GB file with the merge command.

Is 14.2 GB an absolute limit (for merging) of the QGIS software? Is thère another way to merge 2 large raster file ?

(My computer has 8GB RAM + 1TB Disk)

GDAL command and log:

gdal_merge.bat -a_nodata 0.0 -ot Float32 -of GTiff -o "C:/Users/Asus/Desktop/FALL test.tif" --optfile C:/Users/Asus/AppData/Local/Temp/processing_PdBtfS/4ffcc760e4124a3f9c847586ff134e2e/mergeInputFiles.txt

Sortie de commande GDAL :
Traceback (most recent call last): 
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Scripts\gdal_merge.py", line 12, in <module>
sys.exit(main(sys.argv))
^^^^^^^^^^^^^^
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 657, in main
return gdal_merge(argv)
^^^^^^^^^^^^^^^^ 
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 397, in gdal_merge
return _gdal_merge(argv=argv)
^^^^^^^^^^^^^^^^^^^^^^ 
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 642, in _gdal_merge
fi.copy_into(t_fh, band, band, nodata, verbose)
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 348, in copy_into
return raster_copy( 
^^^^^^^^^^^^ 
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 95, in raster_copy
return raster_copy_with_mask( 
^^^^^^^^^^^^^^^^^^^^^^
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\osgeo_utils\gdal_merge.py", line 190, in raster_copy_with_mask
to_write = np.choose(mask_test, (data_src, data_dst))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\numpy\core\fromnumeric.py", line 416, in choose
return _wrapfunc(a, 'choose', choices, out=out, mode=mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
File "C:\PROGRA~1\QGIS34~1.2\apps\Python312\Lib\site-packages\numpy\core\fromnumeric.py", line 59, in _wrapfunc
return bound(*args, **kwds)
^^^^^^^^^^^^^^^^^^^^ 
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 14.2 GiB for an array with shape (53701, 35412) and data type int64
Le processus a renvoyé le code erreur 1
Improve this question
2
  • you will need more physical RAM (14.2GB +operating system (4GB). 18.2 you would be getting towards 24GB required - 32GB is recommended from experience. Commented May 13 at 20:22
  • 2
    At least create the output tiff as tiled so it can be written into file as smaller chunks. And probably it would be good to apply lossless compression like LZW or deflate as well. GDAL is a 24 years old program and it does not need huge amounts of memory if used in a right way. That time typical PC had 64 MB of memory. But it would be better to use gdalbuildvrt gdalbuildvrt output.vrt *.tif and then gdal_translate -co tiled=yes -co compress=LZW output.vrt output.tif instead of gdal_merge. Commented May 13 at 21:14

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.