We're Hiring!

combining TIFF and CSV file with metadata

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

combining TIFF and CSV file with metadata

Postby Thorondor » Fri Apr 13, 2012 7:45 am

I want to combine tiffs with metadata listed in csv file. So one csv file includes data for a 96-well-plate e.g. including data like: WELL, CELLCOUNT, HASPB-GFP % CYTOPLASMA, HASPB-GFP % GOLGI, HASPB-GFP % MEMBRAN, HASPB-GFP SCORE CYTOPLASMA, YES-CHERRY % GOLGI...

so is there a way to add the corresponding meta data to each picture and then export the well plate as a OMERO-TIFF? Maybe Bio-Formats could help? I am kind of new in the field, so I try to figure out an easy way.
Any suggestions are welcome.
Thorondor
 
Posts: 58
Joined: Thu Apr 12, 2012 8:10 am

Re: combining TIFF and CSV file with metadata

Postby Thorondor » Mon Apr 16, 2012 7:18 am

To be a bit more precise, I have hundreds of tiff-files (x Plates each 96 wells) and for each plate I have a csv-file with following information: WELL , CELLCOUNT, HASPB-GFP % CYTOPLASMA, HASPB-GFP % GOLGI, HASPB-GFP % GOLGI (STRONG), HASPB-GFP % MEMBRAN, HASPB-GFP % INVALID, HASPB-GFP SCORE CYTOPLASMA, HASPB-GFP SCORE GOLGI......

So I want to add the data of one row to the corresponding well picture in a way that it is easily accessible later on. Any ideas are welcome.
Thorondor
 
Posts: 58
Joined: Thu Apr 12, 2012 8:10 am

Re: combining TIFF and CSV file with metadata

Postby jmoore » Mon Apr 16, 2012 7:27 am

Hi Gideon,

Thorondor wrote:so is there a way to add the corresponding meta data to each picture and then export the well plate as a OME-TIFF? Maybe Bio-Formats could help? I am kind of new in the field, so I try to figure out an easy way.


What's the target for the exported TIFF? Are you trying to get data into OMERO? Are you trying to share it as TIFF files with others? If the latter, then you will need to use structured annotations to attach your CSV data to the individual elements. If you are storing then is in OMERO, then likely your best candidate is to use OMERO.tables in order to attach all of the CSV data to the individual objects.

In either case, the best place to start is by making sure that you are using the screen-plate-well ("SPW") model which is how HCS data is stored in OME. Under Working with OME-XML there are several links such as http://www.openmicroscopy.org/site/support/file-formats/working-with-ome-xml/screen-plate-well/screen-plate-well-2010-04 which you might want to take a look at.

Hope that helps.
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: combining TIFF and CSV file with metadata

Postby Thorondor » Mon Apr 16, 2012 7:42 am

I am just trying to get data into Omero. Uploading the tiff files into omero is no problem. But adding the meta data is a problem. I want to add the data so it is easily accessible as key-value pair, for later image processing, not just for storing the data in omero.
So I don't want to just attach the table. I'll have a closer look at omerotables, maybe it does what I want.
Thanks
Thorondor
 
Posts: 58
Joined: Thu Apr 12, 2012 8:10 am

Re: combining TIFF and CSV file with metadata

Postby jmoore » Mon Apr 16, 2012 8:10 am

Before you dive into OMERO.tables, though, it will be important (or at least far more valuable) if your data is not just disparate TIFFs, but is arranged into actual plates.

There are at least three ways to do this. First, using bioformats' tiffcomment command, you can add the SPW metadata to each file. You can use the API to add the SPW objects directly to OMERO. Or you can use a script to group the TIFFs as they sit now into plates.

It's really a question of how much metadata you have available yourself and/or want attached to your data, and which programming environment (XML, Python, Java) you are most comfortable with. In general, for long term storage, we'd encourage capturing all of the original layout metadata. If, however, you just want to get started quickly, the grouping script might be the way to go. (I'll let Will explain more)

Cheers,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: combining TIFF and CSV file with metadata

Postby wmoore » Mon Apr 16, 2012 8:41 am

Hi,

Here's a quick demo of how to get you data into a Plate format once you've imported it into OMERO.

http://cvs.openmicroscopy.org.uk/snapsh ... -4.3.2.mov

If your tiffs also represent multiple planes of the same image, then you might also want to combine these FIRST: see http://cvs.openmicroscopy.org.uk/snapsh ... Images.mov

There's a brief intro to these and other scripts here http://www.openmicroscopy.org/site/supp ... il-scripts

Any idea how you want to parse your csv and populate Tables (or add annotations in another way)?
E.g. Java, Python?
Do you want this code 'server-side' or client-side? etc.

E.g. have a look at code samples in Python http://trac.openmicroscopy.org.uk/ome/w ... MEROtables or Java http://trac.openmicroscopy.org.uk/ome/w ... MEROtables.

Will.
User avatar
wmoore
Team Member
 
Posts: 674
Joined: Mon May 18, 2009 12:46 pm

Re: combining TIFF and CSV file with metadata

Postby Thorondor » Tue Apr 17, 2012 1:40 pm

ok, for now I installed hdf5, numpy, pytables...
but when i run python first.py (http://trac.openmicroscopy.org.uk/ome/b ... s/first.py)
I just get: ImportError: No module named omero, do I have to set any PATHS or restart a service, I try it on the machine where the omero.server is running.

setting the pythonpath, helps.
now I get:
File "/omero/OMERO.server-Beta-4.3.4/lib/python/omero/__init__.py", line 28, in client
File "/omero/OMERO.server-Beta-4.3.4/lib/python/omero/clients.py", line 128, in __init__
File "/omero/OMERO.server-Beta-4.3.4/lib/python/omero/clients.py", line 235, in _initData
omero.ClientError: No host specified. Use omero.client(HOSTNAME), ICE_CONFIG, or similar.
Thorondor
 
Posts: 58
Joined: Thu Apr 12, 2012 8:10 am

Re: combining TIFF and CSV file with metadata

Postby Thorondor » Wed Apr 18, 2012 7:55 am

ok, so I can finally import omero in python, so omero.tables should work, but were to start?

these two links would hepl:
Creating a Measurement Table: ome.git/examples/OmeroTables/creating.py (see "Going forward" below)
Querying a Table: ome.git/examples/OmeroTables/creating.py (See "Going forward" below)

but for some reason they are not working. Where else can I get more information about omero.tables

I think the code will be in java or python whatever works. The could should be server side, fro the moment, since I mostly need it for testing issues.

thanks so far!
Thorondor
 
Posts: 58
Joined: Thu Apr 12, 2012 8:10 am

Re: combining TIFF and CSV file with metadata

Postby wmoore » Wed Apr 18, 2012 1:05 pm

If you can import omero in python and you have tables installed OK, then you should be good to follow example code here
http://github.com/openmicroscopy/openmi ... /Tables.py

Remove the
Code: Select all
from Connect_To_OMERO import USERNAME, PASSWORD, HOST, PORT


and replace with E.g.
Code: Select all
USERNAME = "bob"
PASSWORD = "somethingSecret"
HOST = "localhost"
PORT = 4064


For the columns of your table, see
http://hudson.openmicroscopy.org.uk/job ... olumn.html

Choose the appropriate type. E.g. WellColumn if you want to refer to wells by ID and you'd attach the table to the Plate.

Hope that helps,

Will
User avatar
wmoore
Team Member
 
Posts: 674
Joined: Mon May 18, 2009 12:46 pm

Re: combining TIFF and CSV file with metadata

Postby Thorondor » Thu Apr 19, 2012 7:52 am

hmmm, when starting
python tables.py
i get following:
No handlers could be found for logger "blitz_gateway"
Traceback (most recent call last):
File "tables.py", line 50, in ?
table = conn.c.sf.sharedResources().newTable(repositoryId, tablename)
File "/omero/OMERO.server-Beta-4.3.4/lib/python/omero/clients.py", line 886, in __getattr__
File "/omero/OMERO.server-Beta-4.3.4/lib/python/omero/clients.py", line 351, in getSession
omero.ClientError: No session available

do I need to update omero, or what seems to be wrong here? Sorry that I cant handle it on my own. :|
Thorondor
 
Posts: 58
Joined: Thu Apr 12, 2012 8:10 am

Next

Return to User Discussion

Who is online

Users browsing this forum: Majestic-12 [Bot] and 1 guest