We're Hiring!

Errors in OMERO.import - bug in Olympus SIS TIFF?

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

Errors in OMERO.import - bug in Olympus SIS TIFF?

Postby Vanhecke » Wed Feb 01, 2012 2:31 pm

Hi all,

As described here (viewtopic.php?f=4&t=1029), I am interested in implementing OMERA for our microscopy lab, which involves about 20 scientists and several methods (we have a confocal, several electron microscopes, a SAXS, light scattering, and an atomic force microscope). I downloaded the OMERO virtual appliance from the website and installed it in virtualbox. After some initial hitches, this now works flawlessly on my Ubuntu 11.10 (64-bit) system.

As I mentioned before, transmission electron microscopy is an important issue here. Each TEM is equipped with a Morada digital camera (Olympus/SIS) and recording is done over the AnalySIS 5.0 ("iTEM") software of SIS. This is a commercially available package, and is fairly common in microscopy labs. Some people have been using the Structured Archive (STAR) database of AnalySIS 5, but we feld we needed a database with a wider perspective (than TEM), and therefore I am now trying out OMERO.

"I" and "We" are users (mostly biologists and chemists), not programmers. Please be gentle...

Now, I am encountering problems when I try to import our images into OMERO. Basically, two types of errors:
1. Some images do not get imported
2. Those that get imported, have wrong/incorrect metadata


1. Some Images do not get imported
-------------------------------------------

I recorded this series of images on a FEI Morgagni TEM equipped with a Morada camera using AnalySIS 5 software. In short: there is no feature of interest, I just wanted to test if the metadata (pixelsize, magnification, ...) is stored properly. The data is saved in 16-bit Olympus SIS TIFF format.
I imported the data using OMERO.import. Interestingly, some images get imported without a problem, while other fail to get imported. The error happens during the "saving to DB" step. After 3 such failures, OMERO.import "hangs", and produces a new warning window (header says warning). Inside the window, there is no text whatsoever. I have repeated the procedure, with exactly the same result.

I attached a screenshot of OMERO.import and a screenshot of the warning. the Debug text and the output text are below.
OMERO.import - warning.png
warning - no text
OMERO.import - warning.png (24.59 KiB) Viewed 5622 times

import.OMERO failed.png
Screenshot of OMERO.import with fails and pending
import.OMERO failed.png (180.2 KiB) Viewed 5622 times


DEBUG TEXT:
This is essentially what I see as a recurrent error. The entire debug text is attached in the zip file.
java.lang.RuntimeException: omero.ValidationException
serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.core.LogicalChannel]; SQL [insert into logicalchannel (contrastMethod, creation_id, external_id, group_id, owner_id, permissions, update_id, detectorSettings, emissionWave, excitationWave, filterSet, fluor, illumination, lightPath, lightSourceSettings, "mode", name, ndFilter, otf, photometricInterpretation, pinHoleSize, pockelCellSetting, samplesPerPixel, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; nested exception is org.hibernate.exception.DataException: could not insert: [ome.model.core.LogicalChannel]
at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:642)
at org.springframework.orm.hibernate3.HibernateAccessor.convertHibernateAccessException(HibernateAccessor.java:412)
at org.springframework.orm.hibernate3.HibernateInterceptor.invoke(HibernateInterceptor.java:117)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:108)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:231)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:116)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy65.doWork(Unknown Source)
at ome.services.util.Executor$Impl.execute(Executor.java:378)
at ome.services.throttling.Adapter.run(Adapter.java:51)
at ome.services.throttling.InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.java:89)
at ome.services.blitz.impl.AbstractAmdServant.runnableCall(AbstractAmdServant.java:150)
at ome.services.blitz.impl.MetadataStoreI.saveToDB_async(MetadataStoreI.java:190)
at omero.api._MetadataStoreTie.saveToDB_async(_MetadataStoreTie.java:85)
at omero.api._MetadataStoreDisp.___saveToDB(_MetadataStoreDisp.java:218)
at omero.api._MetadataStoreDisp.__dispatch(_MetadataStoreDisp.java:366)
at IceInternal.Incoming.invoke(Incoming.java:159)
at Ice.ConnectionI.invokeAll(ConnectionI.java:2037)
at Ice.ConnectionI.message(ConnectionI.java:972)
at IceInternal.ThreadPool.run(ThreadPool.java:577)
at IceInternal.ThreadPool.access$100(ThreadPool.java:12)
at IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:971)



From this I understand that there is some error in the extraction from the file or writing to the database of metdata information. This is also exactly what happens if the "sucessfully" uploaded images are opened in OMERO.insight:


2. Metadata is not read properly
---------------------------------------

Some metadata is read properly from the TIFF file and written to the database (ImageWidth (pixels), Software (AnalySIS), ...) but other (the crucial information, I think) is not saved properly: pixel size and magnification. After the file was imported in OMERO, I cannot scale it in any way. From what I understand of the error messages, there is a link here.

Given the relative popularity of AnalySIS in the electron microscopy world, I would be surprised if I am alone with this error. I thought an original TIFF file might be helpful; you can find it here:

http://dl.dropbox.com/u/6148766/ELN-ARM_00049_38.tif

(temporary link, made 1.02.2012)

Finding a solution: opening with LOCI
--------------------------------------------

I have tried to load these files with LOCI (in Fiji). The LOCI website gives for the Olympus SIS TIFF format the value 3 (good) for "Pixels". The Metadata I got there was:
BitsPerPixel 16
DimensionOrder XYCZT
IsInterleaved false
IsRGB false
LittleEndian true
PixelType uint16
SizeC 1
SizeT 1
SizeX 2970
SizeY 2100
SizeZ 1
Artist Supervisor
BitsPerSample 16
Compression Uncompressed
ImageLength 2100
ImageWidth 2970
Instrument Make Olympus Soft Imaging Solutions
Location /home/dimitri/Documents/pictures/mag morgagni/ELN-ARM_00049_38.tif
MetaDataPhotometricInterpretation Monochrome
MetaMorph no
NewSubfileType 0
NumberOfChannels 1
PhotometricInterpretation BlackIsZero
PlanarConfiguration Chunky
ResolutionUnit None
SamplesPerPixel 1
Software analySIS 5.0
XResolution 200
YResolution 200


Basically: also here the crucial info (pixelsize, magnification,...) is missing or wrong (XResolution = 200 is wrong). The correct Resolution is: 1 pixel=23.84 nm, or the entire image (2970 pixels) is 70.81 micron wide.


In short: I think I hit a bug in the Olympus SIS TIFF reading routines. I also noticed that this format is fairly new to OMERO (added June 2011). As this format is well spread in TEM labs and the bug is currently preventing such TEM users from using OMERO, I am hoping that a solution can be found. Any help is welcome (remember: we are biologists...)!

Kind greetings,


Dimitri Vanhecke
Adolphe Merkle Institute
University of Fribourg
Switzerland
Attachments
debug and output.zip
(6.13 KiB) Downloaded 160 times
Vanhecke
 
Posts: 15
Joined: Wed Jan 18, 2012 8:03 am

Update: Errors in OMERO.import - bug in Olympus SIS TIFF?

Postby Vanhecke » Thu Feb 02, 2012 1:19 pm

I do not know if this is of any importance, but I tried to upload 8-bit images (scaled down during exporting from the Olympus SIS software "AnalySIS"). These images all (100%) fail to be imported. In comparison: some of the 16-bit images I was talking about here above get imported (but not properly).


Dimitri
Vanhecke
 
Posts: 15
Joined: Wed Jan 18, 2012 8:03 am

Re: Errors in OMERO.import - bug in Olympus SIS TIFF?

Postby mlinkert » Mon Feb 06, 2012 7:48 pm

Hi Dimitri,

Thank you very much for your detailed report. You have indeed encountered a bug in OMERO's/Bio-Formats' SIS TIFF support.

I have filed a ticket for this problem on our issue tracking system:

http://trac.openmicroscopy.org.uk/ome/ticket/8009

If you wish to receive an automated email each time we update the status of the ticket, please let me know and I will be sure to CC you. I expect that we will have this problem sorted out in time for the next release of OMERO/Bio-Formats.

Regards,
-Melissa
User avatar
mlinkert
Team Member
 
Posts: 353
Joined: Fri May 29, 2009 2:12 pm
Location: Southwest Wisconsin


Return to User Discussion

Who is online

Users browsing this forum: No registered users and 1 guest