We're Hiring!

Inplace Import of CV7000S

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

Inplace Import of CV7000S

Postby gudlap » Tue Mar 12, 2019 2:35 pm

Sebastien -

This is a follow-up to an earlier thread. Furthermore, per Sebastien's suggestion, I'm moving this PM to public forum for future reference.

Looks like the latest version of Bioformats (6.0) started supporting import datasets acquired on Yokogawa CellVoyager CV7000/CV7000S. Thanks to all the OMERO/Bioformats developers for implementing the importer for this High-throughput microscope.

Is there a way to test this new functionality in OMERO.Server for our datasets from CV7000S?

We've tired the latest OMERO.Server docker image (release 5.4.10) from docker hub. But based on the import logs, it looks like the latest OMERO.Server (rel. 5.4.10) is using Bioformats 5.9.2.

My question(s):
1) Could we just upgrade ONLY the "bioformats" component on the OMERO.Server 5.4.10 to the latest version to test out the functionality? Looks like the user in this thread managed to test OMERO.Server 5.4.9 with Bioformats 6.0. Any pointers?

2) Is there a docker build for OMERO.Server (version 5.5.0) that is in "Alpha/Beta/RC" phase that we can use to test out the functionality?

3) We have also noticed some (unexpected??) behavior while testing inplace import feature. Specifically, if we use the following command to do a inplace import of a "source" folder OMERO.Server does NOT complain/check for "duplication" of the source folder. That is, we can import the "source" folder multiple times. What are we doing wrong? Is this somewhat related to thread?

Here is the relevant part of the code (dockerized OMERO.Server) for inplace import that we used.

Code: Select all
# Environment variable have been substituted with "dummy" values.
OMERO="/opt/omero/server/OMERO.server/bin/omero"
PREFIX="omero-5_4_10"
OMERO_PI_LAB="G1"
OMERO_PI_NAME="PI ABC"
OMERO_PI_USERNAME="piuname"
OMERO_USER_NAME="USER ABC"
OMERO_USER_USERNAME="useruname"
OMERO_EXPERIMENT_NAME="Exp_031"
IMPORT_PATH="/OMERO/inplace/PI_ABC/User_ABC/Exp_031_20171206_114408/"
USER_PATH="User_ABC"

docker exec $PREFIX-server $OMERO import -s localhost -p 4064 -g $OMERO_PI_LAB -u $OMERO_USER_USERNAME -w $OMERO_USER_USERNAME --skip all -- --no-stats  -- --auto-close -- --transfer=ln_s $IMPORT_PATH -- -T "regex:+name:^.*${USERPATH}/(?<Container1>[^/]*)/.*"


Thank you.

Regards,

Prabhakar (Reddy) Gudla

High-throughput Imaging Facility
Laboratory of Receptor Biology and Gene Expression
Center for Cancer Research, National Cancer Institute
National Institutes of Health, Rockville, MD USA
gudlap
 
Posts: 5
Joined: Sun Apr 29, 2018 3:58 pm

Re: Inplace Import of CV7000S

Postby sbesson » Wed Mar 13, 2019 2:56 pm

Hi Prabhakar,

This is a follow-up to an earlier thread. Furthermore, per Sebastien's suggestion, I'm moving this PM to public forum for future reference.


Thanks for doing so.

1) Could we just upgrade ONLY the "bioformats" component on the OMERO.Server 5.4.10 to the latest version to test out the functionality? Looks like the user in this thread managed to test OMERO.Server 5.4.9 with Bioformats 6.0. Any pointers?


Upgrading only Bio-Formats in an OMERO 5.4 server is certainly a possibility and we have used this internally for pre-testing pyramids. The generic way to perform such an upgrade is to replace all the Bio-Formats JARs under `lib/server` and `lib/client` if you are using the server for importing. In the case of Bio-Formats 6 there have been multiple components updated that would need to be substitued. Would a modified Docker image be sufficient or useful or are you working against a non-containerized environment?
Obviously, modifying a production environment has all types of caveats in terms of testing and support.

2) Is there a docker build for OMERO.Server (version 5.5.0) that is in "Alpha/Beta/RC" phase that we can use to test out the functionality?


More generally, you can see more about the roadmap of OMERO.server 5.5.0 here. We are working through various features via development milestones. Upgrading Bio-Formats is one of the latter milestones at the moment. Once this is done, we can certainly make pre-release Docker images of the server and web deployment availabe.

3) We have also noticed some (unexpected??) behavior while testing inplace import feature. Specifically, if we use the following command to do a inplace import of a "source" folder OMERO.Server does NOT complain/check for "duplication" of the source folder. That is, we can import the "source" folder multiple times. What are we doing wrong? Is this somewhat related to thread?


You might want to try the `excludePath` option comparing the path on disk to what is stored in the DB. e.g.

Code: Select all
bin/omero import --exclude=clientPath


This is used in IDR import configuration files to prevent multiple in-place imports.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm

Re: Inplace Import of CV7000S

Postby gudlap » Wed Mar 13, 2019 6:05 pm

Greetings Sebastien -

The generic way to perform such an upgrade is to replace all the Bio-Formats JARs under `lib/server` and `lib/client` if you are using the server for importing.


I can try that. Is there list of jar files that I should focus on? Sorry, never done this before.

Would a modified Docker image be sufficient or useful or are you working against a non-containerized environment?


That would be great. We are testing the functionality using Docker images in a non-production environment.

The "--exclude=clientpath" worked like a charm. I glossed over this section in the documentation :)

Best,

Prabhakar (Reddy) Gudla
gudlap
 
Posts: 5
Joined: Sun Apr 29, 2018 3:58 pm

Re: Inplace Import of CV7000S

Postby sbesson » Fri Mar 15, 2019 2:04 pm

Hi,

I can try that. Is there list of jar files that I should focus on? Sorry, never done this before.


No worries. Minimally the three files you want to replace under both locations are `formats-gpl.jar`, `formats-bsd.jar` and `formats-api.jar`. After replacing these files by the versions 6.0.0, restarting your server should be sufficient to take into account the new library.

We are testing the functionality using Docker images in a non-production environment


You might want to take a look at https://doi.org/10.5281/zenodo.2594964. This is a work in-progress docker compose service. It includes a modified version of the `omero-server` Docker image which updates all the Bio-Formats JARs before starting the server. Feedback and comments welcome.

The "--exclude=clientpath" worked like a charm. I glossed over this section in the documentation :)


Great, thanks for letting us know.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm


Return to User Discussion

Who is online

Users browsing this forum: No registered users and 1 guest