pixeldata threads/repetitions for big TIFF pyramid workload

[dtrudg]

Hi,

We are bringing an OMERO instance into production use soon. It's on decent (virtualized) hardware with 128GB RAM available, 32 cores and fast 20TB storage for the repository. I'm a bit confused r.e. the best configuration settings to use for good pixeldata pyramid performance. Most of our imports will be large 50MPixel+ TIFF files, so there is a lot of pyramid creation happening.

At present I set omero.pixeldata.threads = 8 then during pyramid creation and I'm confused that I'm seeing CPU usage via 'top' of up to 1200% (more than 8 threads/processes), but the Pixeldata-0 log file only ever prints messages that seem to come from up to 5 threads:

Code: Select all

2015-03-02 13:08:29,174 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:793 1/819 (0%).
2015-03-02 13:08:52,518 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:803 82/819 (9%).
2015-03-02 13:08:54,320 INFO  [                ome.io.nio.PixelsService] (2-thread-5) Pyramid creation for Pixels:815 82/819 (9%).
2015-03-02 13:08:55,224 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:827 82/819 (9%).
2015-03-02 13:08:55,978 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:793 82/819 (9%).
2015-03-02 13:09:15,767 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:803 163/819 (19%).
2015-03-02 13:09:17,576 INFO  [                ome.io.nio.PixelsService] (2-thread-5) Pyramid creation for Pixels:815 163/819 (19%).
2015-03-02 13:09:18,780 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:827 163/819 (19%).
2015-03-02 13:09:20,772 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:793 163/819 (19%).

We thought that given our expected workload (very large batches of TIFFs uploaded by multiple users concurrently) that omero.pixeldata.threads < cores and omero.pixeldata.repetitions > 1 would be useful, so it might split the processing between pending images from different users better - running multiple pixeldata pyramid batches when there's a backlog. I'm confused as to how repetitions truly works though. We don't seem to have any RAM worries here, given the 128GB available. The processes are always showing well under the heap size allocated.

I'd be grateful for any pointers on the best settings for this scenario.

Thanks,

Dave Trudgian

[jmoore]

Hi,

Hi Dave,

At present I set omero.pixeldata.threads = 8 then during pyramid creation and I'm confused that I'm seeing CPU usage via 'top' of up to 1200% (more than 8 threads/processes), but the Pixeldata-0 log file only ever prints messages that seem to come from up to 5 threads:

The 5 threads should be the five OMERO calling threads that are starting the pyramid creation. Interesting, there have been other recent questions about this (e.g. viewtopic.php?f=4&t=7750) and so we've been looking into just this issue. (Sorry for the slow response!)

We thought that given our expected workload (very large batches of TIFFs uploaded by multiple users concurrently) that omero.pixeldata.threads < cores and omero.pixeldata.repetitions > 1 would be useful, so it might split the processing between pending images from different users better running multiple pixeldata pyramid batches when there's a backlog. I'm confused as to how repetitions truly works though.

Certainly a largish number of threads but less than cores is what's intended. What server version are you using and what other configuration options? Recently, the need for repetitions was largely removed.

There are 2 tickets touching on the general control of low-level pyramid generation:

• http://trac.openmicroscopy.org/ome/ticket/6220
• http://trac.openmicroscopy.org/ome/ticket/6224

We'll keep you posted on anything we can find out.

All the best,
~Josh.

[dtrudg]

Thanks for the links to the issues and forum threads. I hadn't managed to find those already - should have searched a bit harder.

Certainly a largish number of threads but less than cores is what's intended. What server version are you using and what other configuration options? Recently, the need for repetitions was largely removed.

We're currently at OMERO.server-5.0.6-ice35-b53. Is the work to remove the need for repetitions in there, or would we need 5.0.8 or the upcoming 5.0.10?

R.E. configuration options, I've tried various things for repetitions / threads, e.g. 4/8 1/32 1/24 etc. I don't think they seem to make much difference to the speed with which our pyramids are being generated, though we do see a speedup over the default 1 rep 2 thread config. I currently have:

Code: Select all

omero.jvmcfg.percent.blitz=15
omero.jvmcfg.percent.indexer=5
omero.jvmcfg.percent.pixeldata=30
omero.pixeldata.repetitions=1
omero.pixeldata.threads=24

Given that this is a VM provisioned with 118GB the memory allocations are quite large. With some test imports of ~450 TIFFs, each 5368 x 9934 in size I have to leave it overnight for the pyramid creation to be complete.

Thanks again,

Dave Trudgian

[jmoore]

We're currently at OMERO.server-5.0.6-ice35-b53. Is the work to remove the need for repetitions in there, or would we need 5.0.8 or the upcoming 5.0.10?

The initial version of ELQ was added in 5.0.3 (https://github.com/openmicroscopy/openm ... /pull/2639)

With some test imports of ~450 TIFFs, each 5368 x 9934 in size I have to leave it overnight for the pyramid creation to be complete.

Would you be up for uploading one such TIFF for testing?

Thanks,
~Josh

[dtrudg]

Checking with the user - if okay I'll upload to qa. Thanks!

[dtrudg]

Have uploaded 5 of the large TIFFs I was talking about to qa.

Thanks again,

Dave Trudgian

[sbesson]

Hi Dave,

thanks for the upload. We'll try to reproduce the issue on our testing servers and get back to you as soon as possible.

Best,
Sebastien

[dtrudg]

Great, thanks.

Looking at the PixelData logs now (using 1 rep, 24 threads on this 32 thread VM) I'm seeing pyramid generation taking around 2-4 mins per TIFF, so when the user uploads a batch of 300 it's quite a wait.

e.g.:

Code: Select all

[omero@imagebank log]$ grep 1202 PixelData-0.log
2015-03-04 15:10:26,005 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 1/936 (0%).
2015-03-04 15:10:47,980 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 94/936 (9%).
2015-03-04 15:11:09,451 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 187/936 (19%).
2015-03-04 15:11:31,818 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 280/936 (29%).
2015-03-04 15:11:54,275 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 373/936 (39%).
2015-03-04 15:12:17,477 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 466/936 (49%).
2015-03-04 15:12:39,501 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 559/936 (59%).
2015-03-04 15:13:03,041 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 652/936 (69%).
2015-03-04 15:13:28,214 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 745/936 (79%).
2015-03-04 15:13:52,320 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 838/936 (89%).
2015-03-04 15:14:17,977 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 931/936 (99%).
2015-03-04 15:14:19,781 INFO  [                ome.io.nio.PixelsService] (2-thread-1) SUCCESS -- Pyramid created for pixels id:1202

Also see errors such as:

Code: Select all

2015-03-09 16:35:10,475 ERROR [        ome.services.util.ServiceHandler] (2-thread-5) Method interface ome.services.util.Executor$Work.doWork invocation took 243290

Presumably this is just really a warning that it's taking a lot longer than migh be expected?

Thanks,

DT

[sbesson]

Hi Dave,

Looking at the PixelData logs now (using 1 rep, 24 threads on this 32 thread VM) I'm seeing pyramid generation taking around 2-4 mins per TIFF, so when the user uploads a batch of 300 it's quite a wait.

e.g.:

Code: Select all

[omero@imagebank log]$ grep 1202 PixelData-0.log
2015-03-04 15:10:26,005 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 1/936 (0%).
2015-03-04 15:10:47,980 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 94/936 (9%).
2015-03-04 15:11:09,451 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 187/936 (19%).
2015-03-04 15:11:31,818 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 280/936 (29%).
2015-03-04 15:11:54,275 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 373/936 (39%).
2015-03-04 15:12:17,477 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 466/936 (49%).
2015-03-04 15:12:39,501 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 559/936 (59%).
2015-03-04 15:13:03,041 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 652/936 (69%).
2015-03-04 15:13:28,214 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 745/936 (79%).
2015-03-04 15:13:52,320 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 838/936 (89%).
2015-03-04 15:14:17,977 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:1202 931/936 (99%).
2015-03-04 15:14:19,781 INFO  [                ome.io.nio.PixelsService] (2-thread-1) SUCCESS -- Pyramid created for pixels id:1202

I can definitely reproduce similar metrics on a 5.0.8 server using the TIFF samples you uploaded.

Code: Select all

$ grep 1403 var/log/PixelData-0.log
2015-03-09 21:02:08,070 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 1/819 (0%).
2015-03-09 21:02:20,935 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 82/819 (9%).
2015-03-09 21:02:33,858 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 163/819 (19%).
2015-03-09 21:02:47,345 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 244/819 (29%).
2015-03-09 21:03:01,321 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 325/819 (39%).
2015-03-09 21:03:15,386 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 406/819 (49%).
2015-03-09 21:03:29,474 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 487/819 (59%).
2015-03-09 21:03:43,588 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 568/819 (69%).
2015-03-09 21:03:57,384 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 649/819 (79%).
2015-03-09 21:04:10,199 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 730/819 (89%).
2015-03-09 21:04:22,967 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:1403 811/819 (98%).
2015-03-09 21:04:24,387 INFO  [                ome.io.nio.PixelsService] (2-thread-2) SUCCESS -- Pyramid created for pixels id:1403
2015-03-09 21:04:25,586 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-2) HANDLED EventLog:76349(entityId=1403) [137565 ms.]


Additionally, as you mentioned, I can reproduce the fact that multi-threading does not happen efficiently.

Also see errors such as:

Code: Select all

2015-03-09 16:35:10,475 ERROR [        ome.services.util.ServiceHandler] (2-thread-5) Method interface ome.services.util.Executor$Work.doWork invocation took 243290

Presumably this is just really a warning that it's taking a lot longer than migh be expected?

Absolutely, this ERROR is mostly warning you of a long operation.

On the positive side, I tested your sample image on the development line for OMERO 5.1.x and it looks like there has been improvement. For instance, testing the pyramid generation on one of our Continuous Integration servers, I get:

Code: Select all

$ grep 1344 var/log/PixelData-0.log
2015-03-09 21:12:12,086 WARN  [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) Pixels:1344 -- Already locked! /repositories/OMERO-5.1-merge-integration/Pixels/Dir-001/.1344_pyramid.pyr_lock
2015-03-09 21:12:12,152 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 1/819 (0%).
2015-03-09 21:12:13,846 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 82/819 (9%).
2015-03-09 21:12:15,583 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 163/819 (19%).
2015-03-09 21:12:17,960 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 244/819 (29%).
2015-03-09 21:12:20,995 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 325/819 (39%).
2015-03-09 21:12:24,195 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 406/819 (49%).
2015-03-09 21:12:28,172 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 487/819 (59%).
2015-03-09 21:12:31,584 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 568/819 (69%).
2015-03-09 21:12:34,161 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 649/819 (79%).
2015-03-09 21:12:35,869 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 730/819 (89%).
2015-03-09 21:12:37,804 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:1344 811/819 (98%).
2015-03-09 21:12:38,018 INFO  [                ome.io.nio.PixelsService] (2-thread-3) SUCCESS -- Pyramid created for pixels id:1344
2015-03-09 21:12:40,035 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-3) HANDLED EventLog:46801(entityId=1344) [27982 ms.]


which suggests the performance of single pyramid generation has been improved for your data type. I will conduct some more testing on the multi-threaded aspect of pyramid generation.

Best,
Sebastien

[sbesson]

Hi Dave

restested the pyramid generation in multi-threaded mode with your test data (with `omero.pixeldata.thread` set to 5):

Code: Select all

$ grep 2090 var/log/PixelData-0.log
2015-03-10 11:27:41,035 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 1/819 (0%).
2015-03-10 11:27:46,139 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 1/819 (0%).
2015-03-10 11:27:46,172 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 1/819 (0%).
2015-03-10 11:27:46,212 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 1/819 (0%).
2015-03-10 11:27:58,710 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 82/819 (9%).
2015-03-10 11:28:12,291 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 82/819 (9%).
2015-03-10 11:28:18,124 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 82/819 (9%).
2015-03-10 11:28:18,343 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 82/819 (9%).
2015-03-10 11:28:19,090 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 163/819 (19%).
2015-03-10 11:28:19,953 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 163/819 (19%).
2015-03-10 11:28:22,868 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 163/819 (19%).
2015-03-10 11:28:22,922 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 163/819 (19%).
2015-03-10 11:28:24,317 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 244/819 (29%).
2015-03-10 11:28:25,122 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 244/819 (29%).
2015-03-10 11:28:25,396 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 244/819 (29%).
2015-03-10 11:28:25,424 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 244/819 (29%).
2015-03-10 11:28:27,330 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 325/819 (39%).
2015-03-10 11:28:28,111 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 325/819 (39%).
2015-03-10 11:28:28,351 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 325/819 (39%).
2015-03-10 11:28:28,419 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 325/819 (39%).
2015-03-10 11:28:30,467 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 406/819 (49%).
2015-03-10 11:28:33,126 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 406/819 (49%).
2015-03-10 11:28:33,438 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 406/819 (49%).
2015-03-10 11:28:33,550 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 406/819 (49%).
2015-03-10 11:28:35,628 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 487/819 (59%).
2015-03-10 11:28:36,012 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 487/819 (59%).
2015-03-10 11:28:36,511 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 487/819 (59%).
2015-03-10 11:28:36,535 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 487/819 (59%).
2015-03-10 11:28:38,781 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 568/819 (69%).
2015-03-10 11:28:38,979 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 568/819 (69%).
2015-03-10 11:28:39,436 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 568/819 (69%).
2015-03-10 11:28:39,436 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 568/819 (69%).
2015-03-10 11:28:41,431 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 649/819 (79%).
2015-03-10 11:28:41,483 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 649/819 (79%).
2015-03-10 11:28:41,891 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 649/819 (79%).
2015-03-10 11:28:42,144 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 649/819 (79%).
2015-03-10 11:28:43,400 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 730/819 (89%).
2015-03-10 11:28:44,138 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 730/819 (89%).
2015-03-10 11:28:44,324 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 730/819 (89%).
2015-03-10 11:28:44,739 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 730/819 (89%).
2015-03-10 11:28:46,097 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20904 811/819 (98%).
2015-03-10 11:28:46,163 INFO  [                ome.io.nio.PixelsService] (2-thread-3) Pyramid creation for Pixels:20903 811/819 (98%).
2015-03-10 11:28:46,266 INFO  [                ome.io.nio.PixelsService] (2-thread-1) Pyramid creation for Pixels:20901 811/819 (98%).
2015-03-10 11:28:46,363 INFO  [                ome.io.nio.PixelsService] (2-thread-2) SUCCESS -- Pyramid created for pixels id:20904
2015-03-10 11:28:46,506 INFO  [                ome.io.nio.PixelsService] (2-thread-3) SUCCESS -- Pyramid created for pixels id:20903
2015-03-10 11:28:46,612 INFO  [                ome.io.nio.PixelsService] (2-thread-1) SUCCESS -- Pyramid created for pixels id:20901
2015-03-10 11:28:47,593 INFO  [                ome.io.nio.PixelsService] (2-thread-4) Pyramid creation for Pixels:20902 811/819 (98%).
2015-03-10 11:28:47,837 INFO  [                ome.io.nio.PixelsService] (2-thread-4) SUCCESS -- Pyramid created for pixels id:20902
2015-03-10 11:28:53,453 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-2) HANDLED EventLog:3097608(entityId=20904) [67554 ms.]
2015-03-10 11:28:53,526 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-3) HANDLED EventLog:3097609(entityId=20903) [67673 ms.]
2015-03-10 11:28:53,543 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 1/819 (0%).
2015-03-10 11:28:53,632 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-4) HANDLED EventLog:3097610(entityId=20902) [67771 ms.]
2015-03-10 11:28:53,673 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-1) HANDLED EventLog:3097607(entityId=20901) [73619 ms.]


to be compared with the time required for generating pyramids with a single thread:

Code: Select all

2015-03-10 11:28:55,592 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 82/819 (9%).
2015-03-10 11:28:57,073 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 163/819 (19%).
2015-03-10 11:28:59,107 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 244/819 (29%).
2015-03-10 11:29:01,886 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 325/819 (39%).
2015-03-10 11:29:04,924 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 406/819 (49%).
2015-03-10 11:29:07,942 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 487/819 (59%).
2015-03-10 11:29:11,174 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 568/819 (69%).
2015-03-10 11:29:13,885 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 649/819 (79%).
2015-03-10 11:29:15,745 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 730/819 (89%).
2015-03-10 11:29:17,503 INFO  [                ome.io.nio.PixelsService] (2-thread-2) Pyramid creation for Pixels:20905 811/819 (98%).
2015-03-10 11:29:17,637 INFO  [                ome.io.nio.PixelsService] (2-thread-2) SUCCESS -- Pyramid created for pixels id:20905
2015-03-10 11:29:18,921 INFO  [ ome.services.pixeldata.PixelDataHandler] (2-thread-2) HANDLED EventLog:3097611(entityId=20905) [25445 ms.]


Short summary is that:

• there should be significant performance improvements for the pyramid generation of this type of data notably due to PRs like https://github.com/openmicroscopy/bioformats/pull/1643 on the 5.1.x line
• the multi-threaded PixelData could certainly benefit for further investigation/optimization but this work is unlikely to be done for 5.1.0 at this point.

Best,
Sebastien