Hi,
We have 21 separate ecw files that span a total of ~8200km² we want to merge them into a single ecw using descartes.The files are in EPSG:3395 (WGS 84 World Mecator)
we have...Batch converted them to uncompressed iTiff64 (using average pixel size of 0.15m) The total size being 551GB. Amazingly despite the size this all references into a dgn extremely quickly and load times are next to nothing!
Now we are in the process of Merging to a single iTiff64 using the descartes 'merge' tool. With the intention of then re projecting to another coordinate system and merging out to ecw. But this process is taking for ever.
After two weeks the resultant iTiff64 is only 32GB and from experience when using no compression the resultant merged file will be approximately the sum of the source iTiff's. Meaning it won't complete for another ~120 days!
'Resource Monitor' is reporting a write speed of 25mB/s (which should give us a better result) memory usage is only 2.3GB the source files are on one dedicated local drive and the destination file is on another.
I have had success in the past with areas as large as 50km² within reasonable time.Is this just too big for Descartes?
Mike
Hi Mike,
If I understand correctly, you have 21 iTIFF64 uncompressed files created from 21 ECW files and you are trying to merge those 21 iTIFF64 file into one big uncompressed iTIFF64 file, is that correct?
Does the iTIFF64 images to merge are reprojected? If so that might be the culprit, since reprojection involve further complex mathematical operation which can slow the whole thing down.
If so I would suggest that you avoid doing a reprojection during the merge and use instead reprojection during the a Save As operation (not sure it would be faster, but it is worth a try).
Thanks,
Mathieu
Hi Mathieu,
No we are merging without reprojection for the reason you mention.
Unknown said:use instead reprojection during the a Save As operation
I didn't know save as actually reprojected the data. I had assumed a change in the GCS through 'save as' would correct the projection definition but not reproject the data. I thought I would need to do another 'merge' operation after reprojecting the one 'BIG' itiff64.
Reprojection usually involves a rotation. Wouldn't It need to be resampled ('merged'?) to create the pixels to be square to their intended projection grid?
Have you tried using the Corridor command in Descartes to create your new image, which would contain both existing images in the new output image?
For the performance issue you might try cleaning up the C:\Users\"Login Name"\Appdata\Local\Temp folder. We have seen an issue where users would get a Memory error when a process like Merge would run a long time. It may not make it run faster, but may prevent you from getting an error and failing.
Yes you are right, just setting the GCS doesn't reproject the data, but instead just tell MicroStation which GCS to embed in the raster file.
But when Descartes is loaded if you select yes for the Resample field on the Save options dialog another option will show asking you if you want to reproject. If you specify yes, the raster will be reprojected during the Save As operation to the GCS specify and yes, the pixel will be square (i.e. : a resample operation will occurs between the original raster grid and the new raster grid).
Ok so no reprojection is involved here. I'll do some test to see what the problem is. By the way, did you notice any speed decrease during the merge operation (i.e. : the first hour you got a couple of gigabytes of data created then it slows almost down to almost nothing)?
Mathieu.
Yes the initial rate of file size increase is rapid then after a day or so it slows to a crawl even though microsoft resource monitor reports a constant rate of ~ 20-25Mb/s.
I should mention I am doing this in Bentley map enterprise. Also I am running another instance of bentley map enterprise on the same machine (we can't afford to have the licence tied up for days just to complete one task).