To start I'd just like to preface by saying that OMERO is, at this point, not a system designed specifically for digital pathology. To date, we have not taken many digital pathology use cases into account and this is especially true of whole-slide scanning. Metadata support in particular is quite poor at present.
We are trying our best to begin to understand these use cases as best we can, and implement some solutions, in the run up to Beta 4.2.0.
For some technical background on why you're getting these errors, it's probably best to start with the file and a simple explanation on what the native format of an Aperio SVS file is and how OMERO treats image data. Firstly, while the image is 17MB it's also lossily compressed using JPEG2000. It is not uncommon to have whole-slide scans that are 20,000 x 10,000 or larger. Uncompressed this is many 100's of MB of data.
OMERO is a planar system and it uses a 2D plane as its
unit of work due to the multi-dimensionality of digital microscopy images. These can often be 10's of GB in size. With many microscopists performing quantitative operations on images, lossy compression and invasive operations on the pixel data are avoided
at all costs. Preserving the integrity of the data is something the OME project values very highly.
When OMERO.importer uses Bio-Formats to decompress an Aperio SVS file it does so on the
entire image. Using 20,000 x 10,000 as an example, this requires approximately 600MB of memory. It also copies this entire 600MB of pixel data to the server as a single byte array. Furthermore, many operations within OMERO must operate on the entire image, this includes:
- Thumbnailing
- Visualization
- Projections
Removing this requirement is an ongoing project that we currently refer to internally as "big images" and it applies not only to digital pathology but also to large detector electron microscopy and stitched image high-content screening.
So, with all that said here's what you're going to need to do in order to support this in OMERO Beta 4.1.1:
- Run the OMERO.server on a 64-bit system with at least a Java virtual machine maximum heap size of 2GB
- Adjust the MessageSizeMax on the OMERO.server, OMERO.insight and OMERO.importer to at least 1GB
These adjustments may require re-compiling some of the OMERO support libraries and running debug builds of all client software. We are also actively trying to solicit help narrowing down use cases and file format support from the digital pathology community. You will see resources put up to this effect over the next few weeks.
Are you really sure you want to do this?
Is there a specific use case you're trying to solve?
