We're Hiring!

Script to import images similar to "in-place"

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

Script to import images similar to "in-place"

Postby austinMLB » Mon Nov 05, 2018 4:42 pm

With help from this forum, I have a script that creates some images and then imports them into my Omero instance. My python for the import looks something like this:

#get my ManagedRepository
repo = connection.c.getManagedRepository()
#using fileset and settings that I created earlier
proc = repo.importFileset(fileset, settings)

All of this works fine, but results in a copy of all of my images. In my particular case, I can directly place my new images in a safe location (in the Managed Repository, if helpful). Is there an easy way to convert my code to something similar to the "in-place" import so that it just makes a hard-link instead of copying the files?

Thanks for any guidance
Michael
austinMLB
 
Posts: 19
Joined: Wed Jul 25, 2018 3:26 pm

Re: Script to import images similar to "in-place"

Postby jmoore » Tue Nov 06, 2018 9:48 am

Hi Michael,

Your code will currently have something that looks roughly like UploadFileTransfer.transfer. This is the default behavior of the CLI when you use `bin/omero import`. To get the in-place behavior (`bin/omero import --transfer=ln`), you need to modify that logic to match AbstractExecFileTransfer.transfer, where the "exec" method is implemented either via `ln` or `ln -s`. Eventually these same classes should exit in Python for exactly your use case, but at the moment, you will need to copy the logic.

If we can help with that, let us know. It might be easiest if you can send us a link to your code on GitHub (or similar).

All the best,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Script to import images similar to "in-place"

Postby austinMLB » Tue Nov 13, 2018 2:18 am

Thanks, Josh. That was very helpful. I think below is the general approach. If it seems wrong to you or anyone, any feedback would be appreciated.

From the importFileset() call on the ManagedRepostiory, I had a “proc” and then, for each image, a RawFileStorePrx:
Code: Select all
proc = mrepo.importFileset(fileset, settings)
rfs = proc.getUploader(i)

To do the in-place, from “rfs”, I can get a pointer to the OriginalFile,
Code: Select all
o_file = omero_gateway.getObject(“OriginalFile”, rfs.getFileId())

From that OriginalFile, I can construct the full path to where the file should be and add a link there.

Thanks again,
Michael
austinMLB
 
Posts: 19
Joined: Wed Jul 25, 2018 3:26 pm

Re: Script to import images similar to "in-place"

Postby jmoore » Tue Nov 13, 2018 10:27 am

Hi Michael,

austinMLB wrote:That was very helpful.


Glad to hear it.

If it seems wrong to you or anyone, any feedback would be appreciated.

From the importFileset() call on the ManagedRepostiory, I had a “proc” and then, for each image, a RawFileStorePrx:
...
To do the in-place, from “rfs”, I can get a pointer to the OriginalFile,
Code: Select all
o_file = omero_gateway.getObject(“OriginalFile”, rfs.getFileId())

From that OriginalFile, I can construct the full path to where the file should be and add a link there.


Sounds like the general strategy. Probably the only thing missing is the "handshake" to robustly ensure that the server and client are looking at the same file, which is done in the transfer implementation using a UUID so that the client can't trick the server. If you trust both sides of the equation, then omitting that is likely fine for the moment.

All the best,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany


Return to User Discussion

Who is online

Users browsing this forum: Google [Bot] and 0 guests

cron