We're Hiring!

VSI Pyramidal File format

Historical discussions about the Bio-Formats library. Please look for and ask new questions at https://forum.image.sc/tags/bio-formats
Please note:
Historical discussions about the Bio-Formats library. Please look for and ask new questions at https://forum.image.sc/tags/bio-formats

If you are having trouble with image files, there is information about reporting bugs in the Bio-Formats documentation. Please send us the data and let us know what version of Bio-Formats you are using. For issues with your code, please provide a link to a public repository, ideally GitHub.

Re: VSI Pyramidal File format

Postby Oli » Mon Jun 08, 2015 2:53 pm

Dear Sebastian and all,

thank you for this updates version of the CellSens Reader that has allowed us to make great strides in managing the files from our Olympus Slide scanner!

Thank you for all the work!

Kind regards

Oli
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

Re: VSI Pyramidal File format

Postby Oli » Wed Nov 11, 2015 10:32 am

Hi again everyone,

We have an odd behavior with BioFormats reading certain VSI files (from the same batch)

I have sent a file though the sample image upload interface.
https://www.openmicroscopy.org/qa2/qa/f ... bf6bf38fc3

Basically bioformats get stuck at "Analysing..." and does not move forward. Other images from the same acquisition run just fine and all open in the OlyVIA software.

Any thoughts?

Best

Oli
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

Re: VSI Pyramidal File format

Postby dgault » Thu Nov 12, 2015 2:22 pm

Hi Oli,

Thanks for uploading the files for testing. I have tried to reproduce the issue this morning but have been unable to see the same behaviour so far.

If it is Fiji which you are using for the import then I suspect with the large file size that you might be seeing an issue related to memory issues which we have seen recently. You can see the related thread below, the issue should since have been resolved from Image J daily build (1.50e18).

viewtopic.php?f=4&t=7926

If you start a new instance of Fiji and import the file first do you see the same issue?
If it loads correctly the first time then that would suggest its related to memory issues mentioned above. If however its still an issue then could you let me know which version of Fiji or bioformats you are using?

With Thanks,
David
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: VSI Pyramidal File format

Postby Oli » Mon Nov 16, 2015 1:44 pm

Hi David,

Thanks for trying it out, however installing the latest daily build of ImageJ (1.50e30) does not solve the issue.

When opening the file the ImageJ menu reads "Analyzing Image_02.vsi" and the CPU spikes to roughly 15%-25% being used by Fiji. Memory usage goes up about a minute afterwards by a small amount (500MB). The system has 24GB of ram of which 8GB are currently in use. Fiji has asked for 1.3GB of RAM and remains stable.

I have added here the contents of the Log window in the BioFormats Configuration window.

Code: Select all
Bio-Formats Plugins configuration - 2015-11-16 14:43:14

-- System properties --
ij.dir = C:\Fiji.app
java.runtime.name = Java(TM) SE Runtime Environment
fiji.dir = C:\Fiji.app
imagej.dir = C:\Fiji.app
sun.boot.library.path = C:\Fiji.app\java\win64\jdk1.6.0_24\jre\bin
java.vm.version = 19.1-b02
fiji.defaultLibPath = bin/server/jvm.dll
java.vm.vendor = Sun Microsystems Inc.
java.vendor.url = http://java.sun.com/
path.separator = ;
java.vm.name = Java HotSpot(TM) 64-Bit Server VM
file.encoding.pkg = sun.io
user.country = US
sun.os.patch.level =
java.vm.specification.name = Java Virtual Machine Specification
user.dir = C:\Fiji.app
java.runtime.version = 1.6.0_24-b07
java.awt.graphicsenv = sun.awt.Win32GraphicsEnvironment
java.endorsed.dirs = C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\endorsed
os.arch = amd64
java.io.tmpdir = C:\Users\oburri\AppData\Local\Temp\
line.separator =

java.vm.specification.vendor = Sun Microsystems Inc.
user.variant =
os.name = Windows NT (unknown)
sun.java2d.noddraw = true
sun.jnu.encoding = Cp1252
java.library.path = C:\Fiji.app/lib/win64;C:\Fiji.app/mm/win64
java.specification.name = Java Platform API Specification
java.class.version = 50.0
sun.management.compiler = HotSpot 64-Bit Server Compiler
os.version = 6.2
user.home = C:\Users\oburri
user.timezone = Europe/Berlin
java.awt.printerjob = sun.awt.windows.WPrinterJob
file.encoding = Cp1252
java.specification.version = 1.6
plugins.dir = C:\Fiji.app
java.class.path = C:\Fiji.app/jars/imagej-launcher-3.1.6.jar
user.name = oburri
java.vm.specification.version = 1.0
java.home = C:\Fiji.app\java\win64\jdk1.6.0_24\jre
sun.arch.data.model = 64
user.language = en
java.specification.vendor = Sun Microsystems Inc.
awt.toolkit = sun.awt.windows.WToolkit
java.vm.info = mixed mode
java.version = 1.6.0_24
java.ext.dirs = C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\ext;C:\Windows\Sun\Java\lib\ext
sun.boot.class.path = C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\resources.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\rt.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\sunrsasign.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\jsse.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\jce.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\charsets.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\modules\jdk.boot.jar;C:\Fiji.app\java\win64\jdk1.6.0_24\jre\classes
java.vendor = Sun Microsystems Inc.
ij.executable = C:\Fiji.app\ImageJ-win64.exe
python.cachedir.skip = true
file.separator = \
java.vendor.url.bug = http://java.sun.com/cgi-bin/bugreport.cgi
sun.io.unicode.encoding = UnicodeLittle
sun.cpu.endian = little
sun.desktop = windows
fiji.executable = C:\Fiji.app\ImageJ-win64.exe
sun.awt.exception.handler = org.scijava.log.DefaultUncaughtExceptionHandler
sun.cpu.isalist = amd64

-- Formats --
Successfully queried FilePattern reader.
Successfully queried Zip reader.
Successfully queried APNG reader.
Successfully queried JPEG reader.
Successfully queried PGM reader.
Successfully queried Fits reader.
Successfully queried PCX reader.
Successfully queried GIF reader.
Successfully queried BMP reader.
Successfully queried IPLab reader.
Successfully queried Ivision reader.
Successfully queried Deltavision reader.
Successfully queried MRC reader.
Successfully queried Gatan reader.
Successfully queried GatanDM2 reader.
Successfully queried Imaris reader.
Successfully queried OpenlabRaw reader.
Successfully queried OMEXML reader.
Successfully queried LIF reader.
Successfully queried AVI reader.
Successfully queried Pict reader.
Initialized extra widgets for Pict reader.
Successfully queried SDT reader.
Initialized extra widgets for SDT reader.
Successfully queried EPS reader.
Successfully queried SlideBook6 reader.
Successfully queried Slidebook reader.
Successfully queried Alicona reader.
Successfully queried MNG reader.
Successfully queried Khoros reader.
Successfully queried Visitech reader.
Successfully queried LIM reader.
Successfully queried PSD reader.
Successfully queried InCell reader.
Successfully queried L2D reader.
Successfully queried FEI reader.
Successfully queried NAF reader.
Successfully queried MINC reader.
Successfully queried QT reader.
Initialized extra widgets for QT reader.
Successfully queried MRW reader.
Successfully queried TillVision reader.
Successfully queried ARF reader.
Successfully queried Cellomics reader.
Successfully queried LiFlim reader.
Successfully queried Targa reader.
Successfully queried OxfordInstruments reader.
Successfully queried VGSAM reader.
Successfully queried HIS reader.
Successfully queried WATOP reader.
Successfully queried Seiko reader.
Successfully queried Topometrix reader.
Successfully queried UBM reader.
Successfully queried Quesant reader.
Successfully queried BioRadGel reader.
Successfully queried RHK reader.
Successfully queried MolecularImaging reader.
Successfully queried CellWorx reader.
Successfully queried Ecat7 reader.
Successfully queried VarianFDF reader.
Successfully queried AIM reader.
Successfully queried InCell3000 reader.
Successfully queried Spider reader.
Successfully queried Volocity reader.
Successfully queried Imagic reader.
Successfully queried HamamatsuVMS reader.
Successfully queried CellSens reader.
Successfully queried INR reader.
Successfully queried Kodak reader.
Successfully queried VolocityClipping reader.
Successfully queried ZeissCZI reader.
Successfully queried SIF reader.
Successfully queried NDPIS reader.
Successfully queried Povray reader.
Successfully queried IMOD reader.
Successfully queried Fake reader.
Successfully queried AFI reader.
Successfully queried Imspector reader.
Successfully queried BioRadSCN reader.
Successfully queried ZeissLMS reader.
Successfully queried PQBin reader.
Successfully queried FlowSight reader.
Successfully queried IM3 reader.
Successfully queried I2I reader.
Successfully queried JEOL reader.
Successfully queried Nifti reader.
Successfully queried Analyze reader.
Successfully queried APL reader.
Successfully queried NRRD reader.
Successfully queried ICS reader.
Successfully queried PerkinElmer reader.
Successfully queried Amira reader.
Successfully queried Scanr reader.
Successfully queried BD reader.
Successfully queried Unisoku reader.
Successfully queried PDS reader.
Successfully queried Fuji reader.
Successfully queried Operetta reader.
Successfully queried Inveon reader.
Successfully queried CellVoyager reader.
Successfully queried BioRad reader.
Successfully queried FV1000 reader.
Successfully queried ZeissZVI reader.
Successfully queried IPW reader.
Successfully queried JPEG2000 reader.
Successfully queried JPX reader.
Successfully queried ND2 reader.
Initialized extra widgets for ND2 reader.
Successfully queried PCI reader.
Successfully queried ImarisHDF reader.
Successfully queried CellH5 reader.
Successfully queried Wlz reader.
Successfully queried Veeco reader.
Successfully queried ZeissLSM reader.
Successfully queried SEQ reader.
Successfully queried Gel reader.
Successfully queried ImarisTiff reader.
Successfully queried Flex reader.
Error constructing widgets for Flex reader:
java.lang.RuntimeException: Unable to instantiate service for interface loci.formats.services.LuraWaveService
   at loci.plugins.config.FlexWidgets.<init>(FlexWidgets.java:66)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
   at java.lang.Class.newInstance0(Class.java:355)
   at java.lang.Class.newInstance(Class.java:308)
   at loci.plugins.config.FormatEntry.<init>(FormatEntry.java:75)
   at loci.plugins.config.ConfigWindow.run(ConfigWindow.java:311)
   at java.lang.Thread.run(Thread.java:662)
Caused by: Unable to instantiate service for interface loci.formats.services.LuraWaveService
   at loci.common.services.ServiceFactory.getInstance(ServiceFactory.java:165)
   at loci.plugins.config.FlexWidgets.<init>(FlexWidgets.java:63)
   ... 9 more
Caused by: java.lang.reflect.InvocationTargetException
   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
   at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
   at loci.common.services.ServiceFactory.getInstance(ServiceFactory.java:163)
   ... 10 more
Caused by: The LuraWave decoding library, lwf_jsdk2.6.jar, is required to decode this file.
Please make sure it is present in your classpath.
   at loci.formats.services.LuraWaveServiceImpl.<init>(LuraWaveServiceImpl.java:86)
   ... 15 more
Successfully queried SVS reader.
Successfully queried Imacon reader.
Successfully queried LEO reader.
Successfully queried JPK reader.
Successfully queried NDPI reader.
Successfully queried PCORAW reader.
Successfully queried OMETiff reader.
Successfully queried PyramidTiff reader.
Successfully queried MIAS reader.
Successfully queried TCS reader.
Successfully queried Leica reader.
Successfully queried Nikon reader.
Successfully queried Fluoview reader.
Successfully queried Prairie reader.
Successfully queried Metamorph reader.
Successfully queried Micromanager reader.
Successfully queried ImprovisionTiff reader.
Successfully queried MetamorphTiff reader.
Successfully queried NikonTiff reader.
Successfully queried PhotoshopTiff reader.
Successfully queried FEITiff reader.
Successfully queried SimplePCITiff reader.
Successfully queried NikonElementsTiff reader.
Successfully queried Trestle reader.
Successfully queried SIS reader.
Successfully queried DNG reader.
Successfully queried ZeissTIFF reader.
Successfully queried LeicaSCN reader.
Successfully queried SlidebookTiff reader.
Successfully queried TiffDelegate reader.
Initialized extra widgets for TiffDelegate reader.
Successfully queried Text reader.
Successfully queried Burleigh reader.
Successfully queried Openlab reader.
Successfully queried Dicom reader.
Successfully queried SMCamera reader.
Successfully queried SBIG reader.
Successfully queried HRDGDF reader.
Successfully queried Hitachi reader.
Successfully queried Bruker reader.
Successfully queried CanonRaw reader.
Successfully queried OBF reader.

-- Libraries --
Found library Java:
    Version = 1.6.0_24 (Sun Microsystems Inc.)
    Path = C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\rt.jar
Found library ImageJ:
    Path = C:\Fiji.app\jars\ij-1.50d.jar
Found library Java3D:
    Version = 1.5.2 ()
    Path = C:\Fiji.app\java\win64\jdk1.6.0_24\jre\lib\ext\vecmath.jar
Found library Jython:
    Version = 2.5.3 (SciJava)
    Path = C:\Fiji.app\jars\jython-shaded-2.5.3.jar
No library MATLAB.
No library QuickTime for Java.
No library JAI Image I/O Tools - native codecs.
No library Nikon ND2 plugin.
Found library Bio-Formats plugins:
    Version = 13 November 2015
    Path = C:\Fiji.app\plugins\bio-formats_plugins.jar
No library OME plugins.
Found library Image5D:
    Version = 1.2.6 (Fiji)
    Path = C:\Fiji.app\plugins\Image_5D-1.2.6.jar
Found library View5D:
    Version = 1.3.1 (ImageJ)
    Path = C:\Fiji.app\plugins\View5D_-1.3.1.jar
Found library Bio-Formats:
    Version = 13 November 2015
    Path = C:\Fiji.app\jars\bio-formats\formats-api.jar
No library OME I/O.
No library VisBio.
Found library Java Advanced Imaging:
    Version = 1.1.3 (Sun Microsystems, Inc.)
    Path = C:\Fiji.app\jars\jai-core-1.1.3.jar
No library JAI Image I/O Tools - Java wrapper.
Found library JAI Image I/O Tools - Java codecs:
    Path = C:\Fiji.app\jars\bio-formats\jai_imageio.jar
Found library MDB Tools (Java port):
    Path = C:\Fiji.app\jars\bio-formats\mdbtools-java.jar
Found library NetCDF Java:
    Version = 4.3.19 (UCAR/Unidata)
    Path = C:\Fiji.app\jars\netcdf-4.3.19.jar
Found library Apache Jakarta POI (OME version):
    Path = C:\Fiji.app\jars\bio-formats\ome-poi.jar
Found library Simple Logging Facade for Java:
    Version = 1.7.6
    Path = C:\Fiji.app\jars\slf4j-api-1.7.6.jar
Found library OME-XML Java library:
    Path = C:\Fiji.app\jars\bio-formats\ome-xml.jar
Found library OME-Java API:
    Path = C:\Fiji.app\plugins\OMERO.insight-ij-5.1.4-ice35-b55\libs\ome-java.jar
No library Apache Jakarta Commons HttpClient.
Found library Apache Jakarta Commons Logging:
    Version = 1.1.1 (Apache Software Foundation)
    Path = C:\Fiji.app\jars\commons-logging-1.1.1.jar
No library Apache XML-RPC.
Found library OMERO Common:
    Path = C:\Fiji.app\plugins\OMERO.insight-ij-5.1.4-ice35-b55\libs\common.jar
No library OMERO Client.
Found library OMERO Data Model:
    Path = C:\Fiji.app\jars\omero\omero-client-5.1.2-ice35.jar
No library OMERO Importer.
No library Spring.
No library JBoss Client.
Found library JGoodies Forms:
    Version = 1.7.2 2013-11-13 09:48:14 (JGoodies Software GmbH)
    Path = C:\Fiji.app\jars\jgoodies-forms-1.7.2.jar
No library OME Notes.


All the best

Oli
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

Re: VSI Pyramidal File format

Postby Oli » Tue Nov 17, 2015 9:19 am

Hi again,

Also to note that running the latest Java8 version does not solve the issue for me...

Best

Oli
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

Re: VSI Pyramidal File format

Postby dgault » Wed Nov 18, 2015 5:38 pm

Hi Oli,

Thanks for providing the further details. I just wanted to provide a quick update as to how things are progressing. I carried some more testing this afternoon with as close a setup as I could to yours and unfortunately I have so far not been able to reproduce the exact issue.

I plan on carrying out some more exhaustive testing tomorrow across a number of different platforms to see if I can produce any different results.

In the mean time if you have any additional files producing the same issue that you could upload it might be helpful to have.

With Thanks,
David
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: VSI Pyramidal File format

Postby Oli » Tue Nov 24, 2015 3:54 pm

Hi David,

Thanks for the update. I will prepare a series of files that seem to open slowly on our machines. We have seen this behaviour in Win7 Enterprise and Win8.1 Enterprise.

I will post again once I have uploaded them

Best regards
OIi
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

Re: VSI Pyramidal File format

Postby Oli » Tue Apr 19, 2016 12:56 pm

Hi,
I haven't managed to take time for this issue as of right now and I apologise for not getting back to you sooner.

There seems to be a new behaviour when openint VSI files using the Bioformats Macro Extensions.

Overall it takes a lot of more time to open an image, even very small sizes. One issue I have encountered is that using Ext.setId() is much slower than it used to be. (Bioformats version of 14th April 2016 and running on Java8)

As I am calling Ext.setId() over several functions, you can imagine that this makes for a rather slow opening of files.

Any idea what changed to make things slower?

EDIT: I tried opening a region 1000x1000 pixels inside a large series using Ext.openSubImage()
The selection has 4 channels.
Code: Select all
run("Bio-Formats Macro Extensions");
Ext.setId(id);
Ext.getImageCount(n);
for (i=0; i<n; i++) {
  Ext.openSubImage("Image"+i, i, 0,0, 1000,1000 );
}

This takes over 90 seconds on a very recent i7 for a 4 channel 1000x1000 16 bit image, the files are local and we never had any issues before.
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

Re: VSI Pyramidal File format

Postby sbesson » Tue Apr 19, 2016 1:52 pm

Hi Oli,

Thanks for the quick report. Can you confirm the performance regression happens with the version 5.1.9 of Bio-Formats? And do you remember which previous version of Bio-Formats used to work fine for you? Looking at the code history, no modification has been brought to the reader since May 2015 so we would need to investigate more at this point.

Also would it be possible for you to upload a small sample file at http://qa.openmicroscopy.org.uk/qa/upload/ so we can reproduce the performance degradation using the code snippet you pasted in your previous comment?

Many thanks,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm

Re: VSI Pyramidal File format

Postby Oli » Tue Apr 19, 2016 2:18 pm

Hi Sebastien,

So I've been playing around and no regression is making it any faster. So I deeply apologize for thinking the problem lies within BioFormats.

However it does happen that some datasets are arbitrarily slower than others...
I am uploading a zip file called VSI Compilation with 3 datasets.
The one called Fast gets parsed in 0.36 s and the thumbnail I need opens in 1.76 s (470x761 px)
The one called Slow gets parsed in 2.14 s and the thumbnail I need opens in 8.54 s (513x81 px)
The one called Very Slow gets parsed in 22.04 s and the thumbnail I need opens in 91.24 s (580x100 px)

The Sizes of the files are 289MB, 181MB and 736MB Respectively

So could it be that Olympus changed something in the way they store their files that makes the reading go through some other path that takes longer? Can you find a difference between the first and second files that would justify this 4x loss in speed?

The upload should finish by the time this is posted.

All the best
Oli
Oli
 
Posts: 71
Joined: Mon Nov 29, 2010 4:57 pm

PreviousNext

Return to User Discussion [Legacy]

Who is online

Users browsing this forum: Google [Bot] and 1 guest