Script to import images similar to "in-place"

[austinMLB]

With help from this forum, I have a script that creates some images and then imports them into my Omero instance. My python for the import looks something like this:

#get my ManagedRepository
repo = connection.c.getManagedRepository()
#using fileset and settings that I created earlier
proc = repo.importFileset(fileset, settings)

All of this works fine, but results in a copy of all of my images. In my particular case, I can directly place my new images in a safe location (in the Managed Repository, if helpful). Is there an easy way to convert my code to something similar to the "in-place" import so that it just makes a hard-link instead of copying the files?

Thanks for any guidance
Michael

[jmoore]

Hi Michael,

Your code will currently have something that looks roughly like UploadFileTransfer.transfer. This is the default behavior of the CLI when you use `bin/omero import`. To get the in-place behavior (`bin/omero import --transfer=ln`), you need to modify that logic to match AbstractExecFileTransfer.transfer, where the "exec" method is implemented either via `ln` or `ln -s`. Eventually these same classes should exit in Python for exactly your use case, but at the moment, you will need to copy the logic.

If we can help with that, let us know. It might be easiest if you can send us a link to your code on GitHub (or similar).

All the best,
~Josh

[austinMLB]

Thanks, Josh. That was very helpful. I think below is the general approach. If it seems wrong to you or anyone, any feedback would be appreciated.

From the importFileset() call on the ManagedRepostiory, I had a “proc” and then, for each image, a RawFileStorePrx:

Code: Select all

proc = mrepo.importFileset(fileset, settings)
rfs = proc.getUploader(i)


To do the in-place, from “rfs”, I can get a pointer to the OriginalFile,

Code: Select all

o_file = omero_gateway.getObject(“OriginalFile”, rfs.getFileId())


From that OriginalFile, I can construct the full path to where the file should be and add a link there.

Thanks again,
Michael

[jmoore]

Hi Michael,

That was very helpful.

Glad to hear it.

If it seems wrong to you or anyone, any feedback would be appreciated.

From the importFileset() call on the ManagedRepostiory, I had a “proc” and then, for each image, a RawFileStorePrx:
...
To do the in-place, from “rfs”, I can get a pointer to the OriginalFile,

Code: Select all

o_file = omero_gateway.getObject(“OriginalFile”, rfs.getFileId())


From that OriginalFile, I can construct the full path to where the file should be and add a link there.

Sounds like the general strategy. Probably the only thing missing is the "handshake" to robustly ensure that the server and client are looking at the same file, which is done in the transfer implementation using a UUID so that the client can't trick the server. If you trust both sides of the equation, then omitting that is likely fine for the moment.

All the best,
~Josh