Bio-Formats

Why is Bio-Formats being developed?

Bio-Formats's primary purpose is to convert proprietary microscopy data into an open standard called the OME data model, particularly into the OME-TIFF file format. See the Bio-Formats statement of purpose for a thorough explanation and rationale.

How do I get started using Bio-Formats?

To use Bio-Formats with ImageJ, we recommend using the Fiji distribution. Start Fiji, click the File menu, choose Import, then "Bio-Formats," and choose a data file to import.

To use it as a library, you can use loci_tools.jar (it is a bundle containing all the functionality of Bio-Formats plus optional libraries), or you can use bio-formats.jar separately and download only those optional libraries you need. We have created a guide to using Bio-Formats as a library that will hopefully be useful to you. For further details, see the Bio-Formats as a Java library page.

How do I use ImageJ with Bio-Formats?

To use Bio-Formats with ImageJ, we recommend using the Fiji distribution. Start Fiji, click the File menu, choose Import, then "Bio-Formats," and choose a data file to import.

For more information, see the Bio-Formats ImageJ page.

How do I use MATLAB with Bio-Formats?

See the Bio-Formats MATLAB page for information on using Bio-Formats with MATLAB.

Do you support XYZ format?

See the Bio-Formats supported formats page for a detailed list of the supported formats.

We have a lot of data from a home built system. Can you read it?

We can certainly add support for your data.  In order to do so, we ask that you send any documentation that you may have regarding the file format in which the data is stored, along with a representative collection of example datasets.  Please contact the Bio-Formats team for details on how to send data.

My dataset has multiple files. Which file should I open (or import)?

Please see this table: http://trac.openmicroscopy.org/ome/wiki/BioFormats-DatasetStructureTable

How do I save a dataset to multiple files?

Examples of how to save a dataset to multiple files are available on the Bio-Formats export wiki page.

I see an 'OutOfMemory' or 'NegativeArraySize' error message when attempting to open an SVS or JPEG-2000 file. What does this mean?

An 'OutOfMemory' error message indicates that the amount of pixel data in a single image plane exceeds the amount of memory allocated to the JVM.  A 'NegativeArraySize' error message indicates that the amount of pixel data in a single image plane exceeds 2 GB.

In the former case, increasing the amount of memory that is allocated to the JVM should solve the problem.  In the latter case, you will need to open the image in sections.  If you are using Bio-Formats as a library, this means using the 'openBytes(int, int, int, int, int)' method in loci.formats.IFormatReader; if you are using Bio-Formats within ImageJ, this means selecting the 'Crop on import' option.

Note that JPEG-2000 is a very efficient compression algorithm - thus the size of the file on disk will be substantially smaller than the amount of memory required to store the uncompressed pixel data.  It is not uncommon for a JPEG-2000 or SVS file to occupy less than 200 MB on disk, and yet have over 2 GB of uncompressed pixel data.

Why do my Slidebook files take a long time to open?

The Slidebook file format is quite complex, and unfortunately we do not have a lot of information about it.  As a result, Bio-Formats must scan through the entire file to find the pixel data, which can be quite slow depending upon the size of the file.

As recent versions of Slidebook support exporting to OME-TIFF, we recommend that you use the Slidebook software to export your .sld and .spl files to OME-TIFF.  You can then use Bio-Formats to open the OME-TIFFs directly.

Why is Bio-Formats written in Java?

From a practical perspective, we use Java because it is cross-platform and widely used, with a vast array of libraries for handling common programming tasks. C++ or Python would also have been reasonable choices, but we chose Java because it is easier than C++ and faster than Python, with a larger assortment of de facto standard libraries (e.g., Swing) than either.

Java is also one of the easiest languages from which to deploy cross-platform software. In contrast to C++, which has a large number of complex platform issues to consider, and Python, which leans heavily on C and C++ for many of its components (e.g., NumPy and SciPy), Java code is compiled one time into platform-independent byte code, which can be deployed as is to all supported platforms. And despite this enormous flexibility, Java manages to provide time performance nearly equal to C++, often better in the case of I/O operations.

Historically, LOCI's software projects grew around efforts to harness VisAD, a Java component library for interactive and collaborative visualization and analysis of numerical data, within our VisBio application for visualization of multidimensional microscopy data. We also added support for several microscopy formats to VisAD before splitting the code into a standalone library, Bio-Formats. The choice to use Java and VisAD (rather than, e.g., C++ and VTK) was partially motivated by the fact that LOCI's lead software architect Curtis Rueden was already an expert on Java and VisAD technologies. Choosing Java also enabled cross-platform integration with ImageJ, one of the most popular freely available image processing tools in the life sciences field, as well as with the OMERO system for visualization, management, and annotation of microscopy data.

Isn't Java too slow?

Java's time performance has been comparable to C++ for many years now, especially in the realm of file I/O where Bio-Formats is focused. According to one thorough study performed in 2005, "Java often outperforms C++ in operations such as memory allocation and file I/O while C++ often outperforms Java in arithmetic and trigonometric operations." This statement is corroborated in an earlier study from 2004 that also includes an I/O benchmark: "If we exclude the trigonometry component, Java performed virtually identically to Visual C++, the fastest of Microsoft's languages." Some of the theoretical basis for such results is discussed in another 2004 article, which also finds that "Java performance on numerical code is comparable to that of C++, with hints that Java's relative performance is continuing to improve."

We have seen two refutations ([1], [2]) of these figures, but neither includes an I/O benchmark, and according to their results Java's computational performance is within a factor of two of C++'s. Even from a pessimistic perspective, we believe the trade-off is acceptable when considering the other advantages of Java such as cross-platform deployment, widespread support and ease of development—as the second article above states: "It seems that it's much, much easier to create a well performing program in Java. So, please consider it for a moment before you start recoding your Java program in C++ just to make it faster."

For further reading, check out this June 2010 article on popular Java myths, and Wikipedia's comparison of Java performance to other languages.

Are you considering translating Bio-Formats to any other languages?

The community has expressed substantial interest in a fully native C/C++ solution for Bio-Formats. We are currently funded to develop complete and robust native Bio-Formats integration capabilities, and we aim to make Bio-Formats the single solution for scientific image access. We will do what is necessary to make that happen.

That said, there are many factors to consider when weighing potential solutions: performance, ease of use, integration, and many others. Our current assessment is that it would not be an efficient use of resources to create a full language translation of Bio-Formats to another language such as C or C++. It would be a project requiring thousands of person-hours that would be better spent improving the codebase we already have. Further, the advantages of Java are too numerous to justify switching completely to another language. While it would be possible (with prolonged effort) to provide the core Bio-Formats library in C or C++ with Java bindings on top, doing so would sacrifice Bio-Formats's platform independence and greatly complicate deployment, especially to Java-based systems such as ImageJ and OMERO. We also expect that a C or C++ version of Bio-Formats would offer no significant improvement in time performance.

Instead, there are several effective ways to harness Java code from other environments; see the Bio-Formats article on interfacing from non-Java code for details. We are committed to exploring and developing these methods further to make Bio-Formats work for as many applications as possible, and welcome community feedback and ideas on this issue.

How can I invoke Bio-Formats from my language of choice?

See the article on interfacing from non-Java code.

Can you send me the specifications for format XYZ?

The Bio-Formats website mentions that you have specification documents for format XYZ? Can you please send them to me?

In short, no.  Most of the specification documents that we have are covered by non-disclosure agreements; distributing the documents would be a violation of those agreements.  The list of specifications that we have for each format is provided merely as an indication of how well the format is supported and how supportive the format's owner has been of Bio-Formats.

If you really need the specification for a particular format, we advise that you contact the format owner directly.

How do I programatically retrieve metadata from a file?

The GetPhysicalMetadata, PrintTimestamps, and PrintLensNA examples illustrate how to retrieve some commonly used metadata values.  All three examples use the loci.formats.meta.MetadataRetrieve class to retrieve metadata, so it's a good idea to read the Javadocs for MetadataRetrieve in order to determine which methods will give you the metadata that you are looking for.

What happened to the 'openImage' method in IFormatReader?

The 'openImage' method is now in loci.formats.gui.BufferedImageReader.  If 'reader' is the IFormatReader object that you are currently using to open images, then all you need to do is add this line of code after declaring 'reader':

BufferedImageReader imageReader = new BufferedImageReader(reader);

You will then need to replace 'reader' with 'imageReader' in subsequent lines of code.

How do I add support for a file format?

Bio-Formats is an open project, and as such we strongly encourage other developers to contribute file format reader modules. Our time is limited, and we can use all the help we can get. We have created a guide to coding new file format readers that will hopefully be useful to you in your endeavor, and we would be happy to assist you with any questions or problems you encounter—just email us.

How can I cooperate with the Bio-Formats developers to improve support for a format?

If you are a programmer, you can help by writing and contributing a file format reader module for Bio-Formats—see How do I add support for a file format to Bio-Formats?.

If you are a user, but do not control the format, you can help by letting us know you are interested in the format, and sending us some sample data if possible. Several currently supported formats were added specifically because someone emailed us asking about them. We have an FTP server set up for users to upload large data to us—email us for details.

If you do control the format, you can start by releasing technical details to the community. Publishing a file format specification document, sample data files, and example source code to your web site is extremely useful for external developers to successfully deal with your format. Once you do, we invite you to let us know and we will be glad to work on adding support for the format as our time permits.

If you are a company unwilling to release details publicly, we urge you to reconsider—see Why should my organization open our file format to the community?. If you insist upon keeping your format secret, you could implement your own Bio-Formats file format reader module to enable support in third party applications such as ImageJ, OME and VisBio. Or you could send technical details to the Bio-Formats developers, who will attempt to add support for it, as time permits.

Why should my organization open our file format to the community?

You may feel that by concealing the details of your format, you can "lock in" your customers to a particular software package, or protect your organization in case of errors, but hiding your format has drawbacks:

1. You will generate animosity among your users when they cannot easily perform needed operations on their data. As a result, your users may switch to another vendor, or even build their own systems, the next time they upgrade their hardware (or perhaps even immediately).

2. Your format will almost certainly be reverse engineered anyway, unless you poison your format by hiding behind the DMCA or similar law, in which case your format will be unusable and customers will abandon it completely. Since a reverse-engineered implementation is less likely to completely and correctly reflect the quality and depth of your format, it would be more in your interest to assist the community in an open implementation, or at least provide them with a specification document so that they can create one themselves.

OME-TIFF is insufficient to my organization's needs, so we provide our own format instead. You'll support it via Bio-Formats anyway, right?

Such an attitude does not serve the best interests of the community, nor is it likely to be advantageous to you, ultimately. Supporting OME-TIFF directly will earn good karma points with your users/customers, while providing yet another impenetrable, incompatible format that makes your users feel trapped and powerless will certainly earn some bad karma.

As OME-TIFF becomes more prevalent, users will increasingly demand direct support for it at acquisition time. It makes more sense to provide this support yourself, rather than rely on a third party such as Bio-Formats to convert the data after the fact.

It may be that OME-XML does not encompass every piece of information you wish to record. However, you can work with the community to extend the specification to accommodate your needs. There are advantages to leveraging the community's efforts:

1. Your developers will spend their time more efficiently, and you will produce a more versatile final product. Though interfacing with the community will require some time investment, you will easily regain any time lost by harnessing community code, and your end result will be both more powerful and more maintainable than otherwise.
2. You will save on support, since many types of support requests will be directed at the community rather than at you specifically. Of course, you will still need to fix bugs in your software. But you will have help from external developers with tasks such as fixing bugs in Bio-Formats and extending the OME-XML specification, whereas if you develop your own proprietary solution, all of these support tasks become your organization's sole responsibility.
3. You will be recognized as a progressive, forward-thinking organization dedicated to improving available microscopy tools.