We're Hiring!

PlaneID support for Columbus imports

Historical discussions about the Bio-Formats library. Please look for and ask new questions at https://forum.image.sc/tags/bio-formats
Please note:
Historical discussions about the Bio-Formats library. Please look for and ask new questions at https://forum.image.sc/tags/bio-formats

If you are having trouble with image files, there is information about reporting bugs in the Bio-Formats documentation. Please send us the data and let us know what version of Bio-Formats you are using. For issues with your code, please provide a link to a public repository, ideally GitHub.

Re: PlaneID support for Columbus imports

Postby dgault » Wed Sep 12, 2018 9:30 am

This looks like a version mismatch with the other Bio-Formats jars, you will have to keep the core Bio-Formats jars to at least the same minor version. In this case if you have a 5.9 version of formats-gpl then upgrading formats-api and formats-bsd to a 5.9 version to match should prevent these errors.
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: PlaneID support for Columbus imports

Postby etienne.dumoulin » Wed Sep 12, 2018 2:46 pm

Hi David,

The link you sent previously are for bio-formats 5.10 jars today. I don't think you keep previous build online on Jenkins. Do you have them somewhere else?
etienne.dumoulin
 
Posts: 22
Joined: Tue Aug 14, 2018 3:43 pm

Re: PlaneID support for Columbus imports

Postby dgault » Thu Sep 13, 2018 8:32 am

Sure, you can retrieve the 5.9 components at https://web-proxy.openmicroscopy.org/we ... omponents/

Notably the 3 core components are the ones you will need:
formats-api
formats-bsd
formats-gpl
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: PlaneID support for Columbus imports

Postby frankgu968 » Thu Feb 28, 2019 2:16 am

Hi David,

I am currently working with Etienne and Oren (from the previous posts in this thread). I have tried swapping the library versions to the newer 5.9.2 and 6.0.0 but still can't seem to have the PlaneID of the Columbus data imported. The imports are still single z-plane. I noticed similarity between the Columbus data and the PE Operetta format. After some modification to the xml file, I was able to make the Operetta reader ingest the columbus data and correctly import the filesets. Unfortunately, depending on the version of Bio Formats (5.9.2 or 6.0.0), I get some funky errors in from the ImportLibrary saying that there is a "checksum mismatch".

What would you suggest I try to get the z-plane imported from a columbus dataset?

Thanks,
Frank Gu
frankgu968
 
Posts: 16
Joined: Fri Nov 23, 2018 5:31 pm

Re: PlaneID support for Columbus imports

Postby dgault » Thu Feb 28, 2019 3:24 pm

Hi Frank,

Do you have the stack trace for the error you are seeing with Bio-Formats 6.0.0?
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: PlaneID support for Columbus imports

Postby frankgu968 » Thu Feb 28, 2019 4:29 pm

Hi David,

With the 5.9.3 library:
Code: Select all
2019-02-28 02:52:43,704 1707658    [2-thread-1] ERROR        ome.formats.importer.ImportLibrary - Error on import: A checksum mismatch has occurred.
omero.ChecksumValidationException: null
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_191]
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_191]
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_191]
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_191]
   at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_191]
   at IceInternal.BasicStream.createUserException(BasicStream.java:2779) ~[ice.jar:na]
   at IceInternal.BasicStream.access$300(BasicStream.java:14) ~[ice.jar:na]
   at IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3298) ~[ice.jar:na]
   at IceInternal.BasicStream.throwException(BasicStream.java:2291) ~[ice.jar:na]
   at IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399) ~[ice.jar:na]
   at omero.grid.ImportProcessPrxHelper.end_verifyUpload(ImportProcessPrxHelper.java:1616) ~[blitz.jar:na]
   at omero.grid.ImportProcessPrxHelper.verifyUpload(ImportProcessPrxHelper.java:1495) ~[blitz.jar:na]
   at omero.grid.ImportProcessPrxHelper.verifyUpload(ImportProcessPrxHelper.java:1482) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:607) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary$1.call(ImportLibrary.java:306) [blitz.jar:na]
   at ome.formats.importer.ImportLibrary$1.call(ImportLibrary.java:286) [blitz.jar:na]
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_191]
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_191]
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_191]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_191]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_191]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]
2019-02-28 02:52:43,704 1707658    [2-thread-1] INFO         ome.formats.importer.ImportLibrary - Exiting on error


With the 6.0.0 library:
Code: Select all
2019-02-28 16:27:50,596 168803     [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2019-02-28 16:27:50,612 168819     [2-thread-1] DEBUG   ome.services.blitz.util.CurrentPlatform - recognized current operating system as being Linux
2019-02-28 16:27:56,743 174950     [2-thread-1] ERROR        ome.formats.importer.ImportLibrary - Error on import: could not insert: [ome.model.core.OriginalFile]; SQL [insert into originalfile (atime, ctime, creation_id, external_id, group_id, owner_id, permissions, update_id, hash, hasher, mimetype, mtime, name, path, repo, "size", version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; constraint [originalfile_repo_path_index]; nested exception is org.hibernate.exception.ConstraintViolationException: could not insert: [ome.model.core.OriginalFile]
omero.ValidationException: null
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_191]
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_191]
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_191]
   at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_191]
   at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_191]
   at IceInternal.BasicStream.createUserException(BasicStream.java:2779) ~[ice.jar:na]
   at IceInternal.BasicStream.access$300(BasicStream.java:14) ~[ice.jar:na]
   at IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3298) ~[ice.jar:na]
   at IceInternal.BasicStream.throwException(BasicStream.java:2291) ~[ice.jar:na]
   at IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399) ~[ice.jar:na]
   at omero.grid.ManagedRepositoryPrxHelper.end_importFileset(ManagedRepositoryPrxHelper.java:175) ~[blitz.jar:na]
   at omero.grid.ManagedRepositoryPrxHelper.importFileset(ManagedRepositoryPrxHelper.java:46) ~[blitz.jar:na]
   at omero.grid.ManagedRepositoryPrxHelper.importFileset(ManagedRepositoryPrxHelper.java:33) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary.createImport(ImportLibrary.java:420) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:563) ~[blitz.jar:na]
   at ome.formats.importer.ImportLibrary$1.call(ImportLibrary.java:306) [blitz.jar:na]
   at ome.formats.importer.ImportLibrary$1.call(ImportLibrary.java:286) [blitz.jar:na]
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_191]
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_191]
   at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_191]
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_191]
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_191]
   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]
2019-02-28 16:27:56,743 174950     [2-thread-1] INFO         ome.formats.importer.ImportLibrary - Exiting on error


Hope this helps!

Thanks,
Frank Gu
frankgu968
 
Posts: 16
Joined: Fri Nov 23, 2018 5:31 pm

Re: PlaneID support for Columbus imports

Postby dgault » Thu Feb 28, 2019 5:01 pm

Thanks Frank,

Would it also be possible to upload the original unmodified Columbus dataset to https://www.openmicroscopy.org/qa2/qa/upload/

We received a partial dataset last time but having the full set would allow us to hopefully put in place a long term fix.
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: PlaneID support for Columbus imports

Postby frankgu968 » Thu Feb 28, 2019 5:14 pm

Hi David,

The dataset is 115G. Can the upload portal support this?

Frank
frankgu968
 
Posts: 16
Joined: Fri Nov 23, 2018 5:31 pm

Re: PlaneID support for Columbus imports

Postby mlinkert » Thu Feb 28, 2019 7:04 pm

Hi Frank,

Datasets that large cannot be uploaded to the QA page, but we have an FTP server that can be used instead. I have sent instructions for uploading via FTP in a private message.

-Melissa
User avatar
mlinkert
Team Member
 
Posts: 353
Joined: Fri May 29, 2009 2:12 pm
Location: Southwest Wisconsin

Previous

Return to User Discussion [Legacy]

Who is online

Users browsing this forum: No registered users and 1 guest