Images triggering very slow viewing in insight

[jacques2020]

Download was extremely fast but reported as completed. I don't know how to check. The file should appear with a size of 2147103116 B (md5 20110624-gpa-16-1322.tif -> MD5 (20110624-gpa-16-1322.tif) = f84fc352d5f498334ac9271aab2486df). Please let me know if something went wrong or if you need other similar files.

[Dominik]

Hi,

sorry, I didn't get the file. It might be too big for the QA upload. I try to find an alternative upload method.

Regards,
Dominik

[jacques2020]

Hi,

I will try to bzip2 it because it is likely very close to the limit (2gb according to the website)
I will let you know if I succeed.

Best

Jacques

[jacques2020]

Hi Dominik,

I redownloaded a bzip2-ed version (named 20110624-gpa-16-1322.tif.bz2). Size = 1185387022 B and hashcode MD5 (20110624-gpa-16-1322.tif.bz2) = 2742bf38f3daef10586f7e9add2fcbd4
Please let me know if you manage to get it this time.
Thank you for your help !

Best

Jacques

[Dominik]

Yes, got it, thanks for your effort! Now I just hope, that I can replicate the problem, so that I can do some debugging. Will keep you posted.

Regards,
Dominik

[Dominik]

Unfortunately the image works fine, imported it to a 5.2.2 test server. It was black after the import, had to adjust the rendering settings manually, but apart from that, the image renders as fast as other images (over LAN however have to test it via broadband later). Another interesting thing to know would be, if it performs better, when you re-import it. Have you tried that? Maybe the issue is related to the fact that is was originally imported with OMERO 4.

Regards,
Dominik

[jacques2020]

Hi Dominik,

With these new indications how to download original file, I made a try and it indeed solve the issue. Now, I of course have numerous file in the situation. How can I batch the process. If I write a python macro, I guess it uses insight and is plagued by the bug. Is matlab also not an option. What is the extend of the issue on original file download ? This could be related to a script that would convert pre omero 5 Images to omero 5 ones to also save space (My post viewtopic.php?f=4&t=7487 and Task #12176).
Thank you for this final information

Best

Jacques

[wmoore]

Hi Jacques,

I have made a prototype Python script that downloads an image and re-imports it.

https://gist.github.com/will-moore/51977ef420ca9f31d28a

If you run this against an image imported with OMERO 4 and archived, it will download the archived image and re-import it using the CLI into the same Dataset as the original image.

NB: you have to have "omero" on your PATH, so that the python script can do:

Code: Select all

$ omero import -d 4852 image.tiff


If you can test this and it works, then it could be extended to work with multiple images and run as a server-side OMERO script.
Also you could add code to take any annotations on the original image and move them to the re-imported image.
This would allow you to run the script, then check the reimported images are OK before deleting them in the client.

Let us know if you want to take this option further...

Cheers,

Will.

[jacques2020]

Hi Will,

It works great! I modified it to use keep alive since it is mandatory with large files as mine + not hard code the user/pwd and loop. (see https://gist.github.com/ee863e83c3e2b663d68f.git). In our use of omero, we don't attach annotations to Images, so I should be fine on that side. I will rather use a "REIMPORTED" annotation to identify old images once successfully re-imported. Since I am not the only user and will let people fix their pictures, I guess I will make it a script. To do so, there are still three aspects unclear to me:
- how to reliably know if an images was imported with omero 4 ? (to safeguard against user mistakes)
- will omero (program in shell) and python library be by default into the PATH and PYTHONPATH if running an omero web script ? If not, what should I do to get them in ?
- Could it be run using insight client (then, how to ensure that it runs on the server ? Or insight embed python runtime and omero command line ?)
Thank you so much for your help

Best

Jacques

[jmoore]

Hi Jacques,

- how to reliably know if an images was imported with omero 4 ? (to safeguard against user mistakes)

There's an OMERO cmd you can use ManageImageBinariesResponse. This is equivalent though to checking whether or not an image is contained in a fileset:

Code: Select all

select i from Image i where i.fileset is null

- will omero (program in shell) and python library be by default into the PATH and PYTHONPATH if running an omero web script ? If not, what should I do to get them in ?

Everything should be present since it will all be configured by the `bin/omero` command. If this doesn't match your experience, let us know. Note: it is also possible to not use a subprocess and instead use the `client` object directly if you run into any issues with starting a new Python interpreter.

- Could it be run using insight client (then, how to ensure that it runs on the server ? Or insight embed python runtime and omero command line ?)

If your script is installed as a Users Script, then yes, it can be run via Insight as well.

Cheers,
~Josh

P.S. it's great that this is working for you, but I still have some concern that this shouldn't be necessary. The underlying slowness is certainly not intended and should be considered a bug, even if we have not yet reproduced.