We're Hiring!

Images & Link Data from IDR

Discussions relating to the Image Data Resource. Please look ask new questions at https://forum.image.sc/tags/idr
Please note:
Historical discussions about the Image Data Resource, https://idr.openmicroscopy.org/. Please look for and ask new questions at https://forum.image.sc/tags/idr

You can also submit queries via GitHub at https://github.com/idr-contrib/community

Images & Link Data from IDR

Postby DavidEgan » Mon Jan 29, 2018 1:48 pm

I would like to download all the images from an IDR study for High Content Imaging of Stem Cells workshop in Edinburgh in March. http://www.crm.ed.ac.uk/seminars/high-c ... -edinburgh

Can I get some help on how to do this? Is there an API that we can use?

Also I'd like to be able to download the web links to the all wells with the well ID e.g. https://idr.openmicroscopy.org/webclien ... ll-1313202

Thanks for your help.

David
DavidEgan
 
Posts: 6
Joined: Mon Jan 29, 2018 9:08 am

Re: Images & Link Data from IDR

Postby wmoore » Tue Jan 30, 2018 10:40 am

Hi David,

You can get the Wells and Images in a Plate using e.g.

https://idr.openmicroscopy.org/webgateway/plate/5960/0/

with plate ID and field Index 0.
This is what the webclient UI is using to build the plate view.

Docs for the /webgateway/ API are at https://docs.openmicroscopy.org/latest/ ... teway.html

We are moving to a newer api designed to be a more complete solution - see https://docs.openmicroscopy.org/latest/ ... n-api.html but we first need to make this publicly available for the IDR.

Do you need to download original images or can you use rendered images?
Currently the IDR doesn't allow download of original image data, but you can get rendered images using e.g.

https://idr.openmicroscopy.org/webgatew ... 05244/0/0/

where 3205244/0/0 refers to Image ID, Z-index and T-index.
This will render the image with the currently saved rendering settings.

You can get more info on each image using:

https://idr.openmicroscopy.org/webgatew ... a/3205244/

This will give you the number of channels, pixel type, rendering settings etc.
The 'min' and 'max' for each channel are the pixel range, the 'start' and 'end' are current rendering cutoff levels.
You can apply various rendering settings to the render_image URL (see the webgateway Docs link)

E.g. To render only the 5th channel, white, with start level:600 -> end level 6505, use:

https://idr.openmicroscopy.org/webgatew ... 505$FFFFFF

Hope that helps,
Regards,
Will.
User avatar
wmoore
Team Member
 
Posts: 674
Joined: Mon May 18, 2009 12:46 pm

Re: Images & Link Data from IDR

Postby DavidEgan » Thu Feb 15, 2018 10:12 am

Will,
Thanks a lot. I'll have a look at this and see how it works for me. For me rendered images are already useful.
David
DavidEgan
 
Posts: 6
Joined: Mon Jan 29, 2018 9:08 am

Re: Images & Link Data from IDR

Postby DavidEgan » Wed May 16, 2018 12:42 pm

The information you gave me above was really useful. I used it run a very cool workshop where we processed extracted features for a screen that is in the IDR and then we could link out directly to your browser. All the meta data came from the IDR too. Very nice.

We would love to be able to get the original images from the IDR. At the moment as you mentioned we can only get the rendered images. Are there any plans to allow people to get the original images. This will be in demand especially for Deep Learning.

D.
DavidEgan
 
Posts: 6
Joined: Mon Jan 29, 2018 9:08 am

Re: Images & Link Data from IDR

Postby jmoore » Thu May 17, 2018 10:03 am

Hi David,

DavidEgan wrote:The information you gave me above was really useful. I used it run a very cool workshop where we processed extracted features for a screen that is in the IDR and then we could link out directly to your browser. All the meta data came from the IDR too. Very nice.


Great to here it. If you have any links or recordings, we'd love to see them.

We would love to be able to get the original images from the IDR. At the moment as you mentioned we can only get the rendered images. Are there any plans to allow people to get the original images. This will be in demand especially for Deep Learning.


We understand completely. Our recent push has been a major upgrade of the IDR to make it compatible with OMERO 5.4. This is done as of last week. After the Users Meeting, we'll be investigating the type of access that you're referring to. If you're interested in the details early, this will be via https://github.com/glencoesoftware/omer ... xel-buffer or a similar service.

All the best,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Images & Link Data from IDR

Postby davidjlogan » Thu Jan 17, 2019 5:59 pm

Just curious about any updates to this. I like @DavidEgan's point of having IDR serve images for training deep learning models -- what is the best/suggested way to download or otherwise access all the raw images?

Thanks,
David L.
davidjlogan
 
Posts: 2
Joined: Tue Jan 15, 2019 6:20 pm

Re: Images & Link Data from IDR

Postby manics » Fri Jan 18, 2019 10:07 am

Hi David

We're still working on a good long term solution. Using the IDR as a data source for machine learning is something we're looking into ourselves.

For now if you want the raw data the best option is to use Aspera. We have a pre-configured Docker image:

https://github.com/IDR/aspera-client-docker

If you're familiar with Aspera you can also use the standard client directly.

As you're hopefully aware the raw data only contains a small portion of the metadata provided by the IDR- this is stored separately in OMERO and available via an API. A majority of the metadata can be queried using the public JSON API:

https://idr.openmicroscopy.org/about/api.html

If you want to make more complex queries, or to request the raw pixel values, you'll need to use the OMERO API:

https://docs.openmicroscopy.org/omero/5 ... index.html

Please let us know if you require a username/password for this.

Simon
User avatar
manics
Team Member
 
Posts: 261
Joined: Mon Oct 08, 2012 11:01 am
Location: Dundee

Re: Images & Link Data from IDR

Postby davidjlogan » Tue Jan 29, 2019 1:55 pm

A late thank you for the info, Simon. I'll let you know if I need further info. Cheers!
davidjlogan
 
Posts: 2
Joined: Tue Jan 15, 2019 6:20 pm


Return to IDR

Who is online

Users browsing this forum: No registered users and 0 guests