We're Hiring!

Need CLI script to download all images with a given tagID

General and open developer discussion about using OMERO APIs from C++, Java, Python, Matlab and more! Please new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

If you are having trouble with custom code, please provide a link to a public repository, ideally GitHub.

Need CLI script to download all images with a given tagID

Postby jstitlow » Thu Feb 08, 2018 5:25 pm

What is the best approach?
Thanks!
j
jstitlow
 
Posts: 18
Joined: Sat Feb 03, 2018 9:07 pm

Re: Need CLI script to download all images with a given tagI

Postby jmoore » Mon Feb 12, 2018 9:54 am

Hi j,

can you tell us a little more about what you'd like the final result to look like? The two best options from the CLI will be bash or pure python. A rough break down in bash:

My available test data:
Code: Select all
[jamoore@ome-c6100-1 dist]$ bin/omero -q hql "select a.id, count(i.id) from ImageAnnotationLink l join l.child a join l.parent i where a.class = ome.model.annotations.TagAnnotation group by a.id"
# | Col1  | Col2
---+-------+------
0 | 12786 | 1
1 | 12901 | 3
2 | 20259 | 2
3 | 12907 | 1
4 | 20260 | 1
5 | 12908 | 1
6 | 20253 | 1
7 | 20257 | 3
8 | 20258 | 3
(9 rows)


Inspecting the images:

Code: Select all
[jamoore@ome-c6100-1 dist]$ bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 12901"
# | Col1  | Col2                      | Col3  | Col4
---+-------+---------------------------+-------+-------
0 | 21905 | UMD001_ORO.svs [Series 3] | 1280  | 501
1 | 21904 | UMD001_ORO.svs [Series 2] | 416   | 482
2 | 21903 | UMD001_ORO.svs [Series 1] | 17317 | 11041
(3 rows)



Getting the images:

Code: Select all
[jamoore@ome-c6100-1 dist]$ for x in $(bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 12901" --style=plain | head -n 2 | cut -f2 -d,); do bin/omero -q export Image:$x -f- >/dev/null ; done


This however downloads as OME-TIFF which will have some limitations. If you are looking to get the original files themselves, you'll need to use `bin/omero fs ls Fileset:$FILESET_ID`.

Cheers,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Need CLI script to download all images with a given tagI

Postby jstitlow » Thu Jun 14, 2018 11:39 pm

Thanks Josh!

That works.

Using the script that exports the original file, how do you name the output file with the original filename?
jstitlow
 
Posts: 18
Joined: Sat Feb 03, 2018 9:07 pm

Re: Need CLI script to download all images with a given tagI

Postby jmoore » Fri Jun 15, 2018 8:21 am

jstitlow wrote:Thanks Josh!

That works.


Glad to hear it.

Using the script that exports the original file, how do you name the output file with the original filename?


Sorry, you lost me. Which script are you referring to?
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Need CLI script to download all images with a given tagI

Postby jstitlow » Fri Jun 15, 2018 12:00 pm

This script exports the images and gives them the name of the image ID:

Code: Select all
for x in $(/opt/OMERO.server-5.4.5-ice35-b83/bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 2113196" --style=plain | head -n 2 | cut -f2 -d,); do /opt/OMERO.server-5.4.5-ice35-b83/bin/omero -q export Image:$x -f- >$x.ome.tiff ; done


How to export the images and give them the name of the image filename instead of the image ID?

Also the script above exports as ome.tiff and you suggested an alternative script to export the image in its original file format. I get an error when running that script, though probably I am implementing it wrong...?

Code: Select all
for x in $(/opt/OMERO.server-5.4.5-ice35-b83/bin/omero -q hql "select i.id, i.name, p.sizeX, p.sizeY from ImageAnnotationLink l join l.child a join l.parent i join i.pixels p where a.id = 2113196" --style=plain | head -n 2 | cut -f2 -d,); do /opt/OMERO.server-5.4.5-ice35-b83/bin/omero fs ls Fileset:$FILESET_ID>$x ; done
usage: /opt/OMERO.server fs ls [-h] fileset
/opt/OMERO.server fs ls: error: argument fileset: invalid proxy value: 'Fileset:'
jstitlow
 
Posts: 18
Joined: Sat Feb 03, 2018 9:07 pm

Re: Need CLI script to download all images with a given tagI

Postby jmoore » Mon Jun 18, 2018 10:09 am

The BASH is obviously getting gnarly. Does the Python code from https://www.openmicroscopy.org/community/viewtopic.php?t=8521&p=19519#p19519 get you what you want then?

~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Need CLI script to download all images with a given tagI

Postby jstitlow » Mon Jun 18, 2018 12:15 pm

Yes.

Do you have a simple Python script to import data into OMERO? Would be helpful for naive users like me to have import/export Python scripts where we can just plugin a path/datasetID and upload/download.

The export script you reference above is perfect, would be handy to have something similar for import.

We have a nice BASH script for batch importing all files in a given directory:

Code: Select all
for i in $1; do
        /opt/OMERO.server-5.4.5-ice35-b83/bin/omero import ${i} -T Dataset:+name:$2\
        --group 'davisgroup'
done


.. which can be called with command line arguments for 1=local directory where the data are located; and 2=OMERO dataset name.

So how to convert that to Python?
Thanks!
j
jstitlow
 
Posts: 18
Joined: Sat Feb 03, 2018 9:07 pm

Re: Need CLI script to download all images with a given tagI

Postby jmoore » Tue Jun 19, 2018 12:21 pm

jstitlow wrote:Do you have a simple Python script to import data into OMERO? Would be helpful for naive users like me to have import/export Python scripts where we can just plugin a path/datasetID and upload/download.


We've and especially Will has been looking into supporting a native Python importer, but nothing at the moment is released. The most flexible option at the moment is to script the CLI:

Code: Select all
from omero.cli import cli_login
with cli_login() as cli:
    cli.invoke("import --help")


where in a loop or similar you could call each of the `bin/omero import` statements that you are currently calling from bash.

The export script you reference above is perfect, would be handy to have something similar for import.

We have a nice BASH script for batch importing all files in a given directory:

Code: Select all
for i in $1; do
        /opt/OMERO.server-5.4.5-ice35-b83/bin/omero import ${i} -T Dataset:+name:$2\
        --group 'davisgroup'
done


.. which can be called with command line arguments for 1=local directory where the data are located; and 2=OMERO dataset name.


Did you see any of the presentations on the --bulk option to bin/omero import during #ome2018? e.g. http://downloads.openmicroscopy.org/presentations/2018/Users-Meeting/Workshops/Metadata-Import/import.pdf

It's a somewhat different, more declarative supported alternative. There, you'd have input YML & CSV files with the sets of parameters you want in your loop, and is what's in use for the IDR (e.g. idr0042)

~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Need CLI script to download all images with a given tagI

Postby jstitlow » Tue Jun 19, 2018 8:10 pm

Thanks Josh,

I like the omero.cli approach.

Re: --bulk option for bin/omero import, will have a look at this for our screen datasets.

Currently trying to write basic batch input.py/output.py scripts for everyday use in our lab and for sharing data with collaborators.

Appreciate the help,
j
jstitlow
 
Posts: 18
Joined: Sat Feb 03, 2018 9:07 pm


Return to Developer Discussion

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron