Bioformats OME.tiff saving - performance

[yuriy_alexandrov]

Hi All,

We are now close to using OME.tiff as acquisition format for OPT, but stumbled on some (quite mysterious) performance issue.

We use recent (5.0.5) "bioformats_package.jar" and BFmatlab to call OME.tiff saving services from Matlab; the procedure is basically a modified "bfsave.m". In its turn, it might be called from LabView acquisition, where it is supposed to be used.

The XY frames are 1191x2559 uint16 images, and we found that it takes 13 seconds to write a frame into OME.tiff on acquisition workstation. This is too slow.

Then we tried the same code on the same data on similar workstation and on a laptop and got 11s, but on several other laptops - 0.34s, 0.65s and 1s. All are Windows machines; we also found that this behaviour didn't depend neither on Matlab version (2013a or 2013b), nor Java (1.6 or 1.7).

To summarize these tests, here is speed per frame -

Win 7 pro, HP workstation = 13s
Win 7 pro, HP workstation = 11s (another one)
Win 8.1 Enterprise, HP laptop = 11s

Win 8.1 Enterprise, HP laptop = 0.34s
Win 8.1 Enterprise, HP laptop = 0.34s (yet another one)
Win 7 Enterprise, - HP laptop = 0.65s
Win 8.1 Enterprise, HP laptop = 1s

There is definitely some clustering.

The line of code that is critical is almost for sure this -

Code: Select all

writer.saveBytes(index-1, getBytes(I));

So the question is - is there something that might slow it down, and how to fix it?
For our OPT team, the 0.34s vs 13s difference is really a matter of practical/nonpractical.

Best regards,
Y.

[sbesson]

Hi Yuriy,

There are several reasons why writing files to disk could be slower on one system than on another. Some of them are independent of Bio-Formats. Could you test this speed difference is mostly related to Bio-Formats itself by using MATLAB built-in write functionalities, e.g. by running the following code sample :

Code: Select all

I=zeros(1191, 2559, 'uint16');
imwrite(I, filepath);


on both systems and let us know if both systems perform the operation with comparable times?

Sebastien

[yuriy_alexandrov]

Thanks Sebastien,

I tried the code You suggested, and it shows that Matlab "imwrite" speed isn't compromised:

saveBytes_speed =

11.6448

imwrite_speed =

0.0302

On "fast saveBytes" laptop, the "imwrite" speed was practically the same:

saveBytes_speed =

0.3836

imwrite_speed =

0.0261

So it looks like, there is something else here.

Best,
Y.

[sbesson]

Thanks Yuriy,

this seems to rule out performance issues in-between machines indeed. What is the JVM configuration in each MATLAB? I suppose both tests were saving files locally and not on a mounted storage system? Do you have the exact test code that was run on the machine?

Sebastien

[yuriy_alexandrov]

Hi Sebastien,

Yes everything is local, - it is simply a test folder containing all the code, BFmatlab, test data etc., it is copied somewhere on disk, then a tic-toc test is run and that's it.
Do You want me to pack and send it as Zip? - no problem at all.

We ran it on Ian's Mac today, it is also in "fast" category.

On configuring JVM - we tried changing JAVA_OPTS on problematic laptop, with no avail.
Are there some specific properties of interest?

Best,
Y.

[sbesson]

Hi Yuriy,

regarding JVM, I would definitely check Heap space is similar for all system. You can print it using

Code: Select all

java.lang.Runtime.getRuntime.maxMemory
java.lang.Runtime.getRuntime.totalMemory
java.lang.Runtime.getRuntime.freeMemory


Else do you know of any other programs that would be running concurrently on the slow machines that could affect writing?
Either a zip with the binaries or a pointer to the code + examples would certainly help us debugging your issue.

Sebastien

[yuriy_alexandrov]

Checked - response isn't very different

fast machine (my laptop)

>> java.lang.Runtime.getRuntime.maxMemory
java.lang.Runtime.getRuntime.totalMemory
java.lang.Runtime.getRuntime.freeMemory

ans =

2.9202e+09

ans =

451739648

ans =

175184840

slow machine (workstation)

>> java.lang.Runtime.getRuntime.maxMemory
java.lang.Runtime.getRuntime.totalMemory
java.lang.Runtime.getRuntime.freeMemory

ans =

3.9179e+09

ans =

400293888

ans =

276617112

I will pack the test folder and send via qa route.

Best,
Y.

[sbesson]

Thanks,

as you said, it looks like JVM memory settings does not seem not explain such a discrepancy. I will be waiting for your zip bundle to try to reproduce your issue locally or on a remote node.

Sebastien

[yuriy_alexandrov]

Hi Sebastien,

Please check Your gmail account, I've sent the test tool via IC file exchange, should be OK.

I also observed strange thing - once on the same computer, the saving suddenly switched to "slow" mode with 26s per frame vs 0.3s. Then it returned back after rewriting whole folder, the only difference was several lines of code, absolutely cosmetic, and location of test data in sub-tree..
Another thing that I found - if one starts the test with test folder not local but on the server - it is also "slow".

Also "imwrite" speed is noticeably lower for the first run if compared to a repeated run:

>> test_add_plane_to_OMEtiff

saveBytes_speed =

26.0609

imwrite_speed =

0.2000

>> test_add_plane_to_OMEtiff

saveBytes_speed =

25.6662

imwrite_speed =

0.0661

Cheers,
Y.

[sbesson]

Hi Yuriy,

received your code and I could not easily reproduce your slow behavior on a Linux node (using local and/or remote volume for writing).

The remote storage performance issue is not completely surprising. I would generally expect any local I/O test to perform better than a remote I/O test. I could definitely see a small performance degradation when moving the test data to a remote share but not within the order of magnitude you describe in this thread.

Looking at your example, I see the compression and the BigTiff parameters are set to LZW and true respectively. Have you tried modifying these values to see if they affect the writing performance on your system in any way? Also regardinf your performance test reported earlier: does 11.6448s correspond to the performance of the `bfsave` call (without any arguments) on a simple 1200x2500 image?

Sebastien