MessageSize and server OOM (possibly) related problems

[hroque]

Hi there,

I've updated the clients and tried imported again from Insigth (don't think server has been updated to this version also).
Now there is only a error screen that pops up after quite a while trying to import saying that if failed attempting to reconnect and the program will now exit. After OK the program exits.

There is the last bits of the log (let me know if you need the whole thing):

Code: Select all

....SNIP....
Connection ErrorIce.ConnectionLostException
    error = 0
   at IceInternal.Outgoing.invoke(Outgoing.java:158)
   at omero.api._ServiceFactoryDelM.getAdminService(_ServiceFactoryDelM.java:641)
   at omero.api.ServiceFactoryPrxHelper.getAdminService(ServiceFactoryPrxHelper.java:2446)
   at omero.api.ServiceFactoryPrxHelper.getAdminService(ServiceFactoryPrxHelper.java:2418)
   at ome.formats.OMEROMetadataStoreClient.initializeServices(OMEROMetadataStoreClient.java:413)
   at ome.formats.OMEROMetadataStoreClient.createRoot(OMEROMetadataStoreClient.java:1049)
   at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:769)
   at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6369)
   at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importCandidates(OmeroImageServiceImpl.java:236)
   at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1506)
   at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:77)
   at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:53)
   at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:102)
   at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
   at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
   at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
   at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
   at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
   at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
   at java.lang.Thread.run(Thread.java:695)
Exception in thread "Thread-44"


I've discovered that Fiji opens these images, so I'm now opening them in Fiji and saving them as tifs and then uploading them.
But would still be must better if I could upload directly from the scope.

Cheers,
Helio

[jmoore]

Hi Helio,

Thanks for your continued efforts. Yes, the entire insight log would be very helpful.

Cheers,
~Josh.

[hroque]

Here you go,

https://www.dropbox.com/s/5lpe76zc11u7p ... nsight.log

Helio

[jmoore]

Thanks, Helio. This is a new error which may be a sign of progress:

Code: Select all

Caused by: omero.ResourceError
    serverStackTrace = "ome.conditions.ResourceError: Map failed
                           at ome.services.RawPixelsBean.save(RawPixelsBean.java:196)
                           at ome.services.RawPixelsBean.close(RawPixelsBean.java:210)
                           at sun.reflect.GeneratedMethodAccessor771.invoke(Unknown Source)


which occurs when checksumming the pixels: RawPixelsBean.java

If I could get the server log from you or Douglas, that would be great. I'm primarily interested in knowing if this is a 32-bit issue while parsing such large files.

Cheers,
~Josh.

[dpwrussell]

Done: https://www.openmicroscopy.org/qa2/qa/feedback/7697/

[jmoore]

Thanks, Douglas. What are the dimensions for this image?

Code: Select all

...
2013-11-10 18:23:45,221 INFO  [             ome.io.nio.RomioPixelBuffer] (erver-2160) Retrieving plane: 25x0x1757
2013-11-10 18:23:45,243 INFO  [             ome.io.nio.RomioPixelBuffer] (erver-2160) Retrieving plane: 26x0x1757
2013-11-10 18:23:45,253 INFO  [             ome.io.nio.RomioPixelBuffer] (erver-2160) Retrieving plane: 27x0x1757
2013-11-10 18:23:46,822 WARN  [              ome.services.RawPixelsBean] (erver-2160) calculateMessageDigest failed on /home/omero/OMERO.data/Pixels/Dir-042/42337
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
        at ome.io.nio.RomioPixelBuffer.getRegion(RomioPixelBuffer.java:343)
        at ome.io.nio.RomioPixelBuffer.getPlane(RomioPixelBuffer.java:489)
        at ome.io.nio.RomioPixelBuffer.calculateMessageDigest(RomioPixelBuffer.java:766)
        at ome.services.RawPixelsBean.save(RawPixelsBean.java:182)
        at ome.services.RawPixelsBean.close(RawPixelsBean.java:210)
...
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
        ... 58 more


Cheers,
~Josh

[hroque]

Hi Josh,

Not sure this is what you mean but here it is.
This has happen with several images of different sizes:

Here is 2 examples:

Pixels XY - 336x256
Pixel Type: Unsigned 16bit
Pixel size XY- 0.222x0.222
z-sections/timepoints - 44 x 1696
Channels - 0
Folder size: 13.14 Gb


Pixel XY - 704x512
Pixel Type Unsigned 16bit
Pixel Size XY - 0.111x0.111
Z sections/timepoints 37x1841
Folder Size: 49.36 Gb

Cheers,
Helio

[dpwrussell]

Server Java is:

java version "1.6.0_33"
Java(TM) SE Runtime Environment (build 1.6.0_33-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03, mixed mode)

[manics]

Thanks Douglas. This might be a Java bug, though it's impossible to tell without testing. I've added a trac ticket to investigate it http://trac.openmicroscopy.org.uk/ome/ticket/11675

[dpwrussell]

I note that your default Java is now openjdk rather than oracle java which is what was recommended at the time I installed this server. I can try switching to see if the problem goes away if you like?