setup/configuration for big images

[sbesson]

Hi Anatole,

you simply need to create a local empty file with the correct naming pattern, e.g. on Linux/OSX using the touch command:

Code: Select all

touch image&sizeX=12000&sizeY=3000&sizeZ=400&sizeC=3.fake


See https://www.openmicroscopy.org/site/sup ... mages.html for more information on these test images.

Sebastien

[achessel]

Fake images, interesting I did not know those existed.

Ok, Server settings:

Code: Select all

omero@Jules-Arthur:~/OMERO.server$ bin/omero admin jvmcfg
JVM Settings:
============
blitz=-Xmx7200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
indexer=-Xmx4800m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
pixeldata=-Xmx7200m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
repository=-Xmx4800m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions


for

Code: Select all

touch image&sizeX=12000&sizeY=3000&sizeZ=400&sizeC=3.fake

viewing/changing z of the image in insight takes ~10s


for

Code: Select all

touch image&sizeX=12000&sizeY=3000&sizeZ=1&sizeC=3.fake

viewing/changing z of the image in insight is almost instant (less than a second).

(similar but faster results for web vieiwer,)
(pyramid generation is fine, as expected)

So it's definitely faster than for tif files (which is I guess expected?) but the difference stays the same...


A.

[sbesson]

Thanks for the double testing,

we will be investigating the underlying reasons of this loading time difference for multi-Z large images, especially looking at the time spent when calling setId, initializing the rendering services...
At the moment, most of the team is focused on getting ready for the upcoming 5.2.0 release but we will keep you posted as soon as we have extra information.

Bet,
Sebastien

[achessel]

no problem, thanks

[achessel]

Friendly *bump*?

Just wondering if you had time/whether you'll have time to look into it...

Thanks

[jmoore]

Hi Anatole,

thanks for the bump. Unfortunately, 5.2.1 is still consuming most of our testing cycles. Most likely we won't have time to investigate further until we're back from holidays as a part of 5.2.2: https://trello.com/c/cO0va6Ni/21-big-images-testing

Sorry for not having better news.

All the best,
~Josh.

[pwalczysko]

Hi Anatole
I am just trying to move on this topic and was wondering if you would be happy to submit your 'big' image 12k*3k with 400 zslice and 3 colors. It's stored in plain tiff (output from fiji stiching). via our QA submission system ? There is a 2GB size limit, so alternatively you could upload it via FTP. Please let me know what is the best for you.
Thank you very much
All the best
Petr

[achessel]

Sorry, I didn't see your answer earlier...

The full image is more 100-200 Gb so it would have to be via ftp...
Alternatively, the same behaviour (difference between displaying a z-stack and a single z-plane for the same size in xyct) is also seen using 'fake' image (as said in that thread somewhere)

[pwalczysko]

And sorry Anatole, I did not see your answer So just to summarize my effort in the meantime:
- yes, I did repeat the slowing down with more z planes for opening in Web and Insight
- we are working on that, have logs and are trawling through them
- you can watch the progress on https://trac.openmicroscopy.org/ome/tic ... #comment:7
- if you want to be cc-ed on the ticket https://trac.openmicroscopy.org/ome/tic ... #comment:7 let me know and you will then get emails from any comment made there
- I have created a bigtiff image in Fiji and used it, it confirmed the behaviour of the fakes, but it is nowhere near your 100G
- I or another member of the team will give you ftp address, hopefully tomorrow

Thank you
All the best
Petr

[mlinkert]

Hi Anatole,

FTP information has been sent to you via private message. If you encounter any problems in uploading the file, please let us know.

Regards,
-Melissa