Page 2 of 3

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Sat Nov 09, 2013 4:18 pm
by hroque
Hi there,

I've updated the clients and tried imported again from Insigth (don't think server has been updated to this version also).
Now there is only a error screen that pops up after quite a while trying to import saying that if failed attempting to reconnect and the program will now exit. After OK the program exits.

There is the last bits of the log (let me know if you need the whole thing):
Code: Select all
....SNIP....
Connection ErrorIce.ConnectionLostException
    error = 0
   at IceInternal.Outgoing.invoke(Outgoing.java:158)
   at omero.api._ServiceFactoryDelM.getAdminService(_ServiceFactoryDelM.java:641)
   at omero.api.ServiceFactoryPrxHelper.getAdminService(ServiceFactoryPrxHelper.java:2446)
   at omero.api.ServiceFactoryPrxHelper.getAdminService(ServiceFactoryPrxHelper.java:2418)
   at ome.formats.OMEROMetadataStoreClient.initializeServices(OMEROMetadataStoreClient.java:413)
   at ome.formats.OMEROMetadataStoreClient.createRoot(OMEROMetadataStoreClient.java:1049)
   at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:769)
   at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(OMEROGateway.java:6369)
   at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importCandidates(OmeroImageServiceImpl.java:236)
   at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.importFile(OmeroImageServiceImpl.java:1506)
   at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.importFile(ImagesImporter.java:77)
   at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter.access$000(ImagesImporter.java:53)
   at org.openmicroscopy.shoola.env.data.views.calls.ImagesImporter$1.doCall(ImagesImporter.java:102)
   at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(BatchCall.java:144)
   at org.openmicroscopy.shoola.util.concur.tasks.CompositeTask.doStep(CompositeTask.java:226)
   at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.doStep(CompositeBatchCall.java:126)
   at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.exec(ExecCommand.java:165)
   at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.run(ExecCommand.java:276)
   at org.openmicroscopy.shoola.util.concur.tasks.AsyncProcessor$Runner.run(AsyncProcessor.java:91)
   at java.lang.Thread.run(Thread.java:695)
Exception in thread "Thread-44"


I've discovered that Fiji opens these images, so I'm now opening them in Fiji and saving them as tifs and then uploading them.
But would still be must better if I could upload directly from the scope.

Cheers,
Helio

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Sun Nov 10, 2013 8:55 pm
by jmoore
Hi Helio,

Thanks for your continued efforts. Yes, the entire insight log would be very helpful.

Cheers,
~Josh.

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Sun Nov 10, 2013 9:16 pm
by hroque

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Mon Nov 11, 2013 10:58 am
by jmoore
Thanks, Helio. This is a new error which may be a sign of progress:

Code: Select all
Caused by: omero.ResourceError
    serverStackTrace = "ome.conditions.ResourceError: Map failed
                           at ome.services.RawPixelsBean.save(RawPixelsBean.java:196)
                           at ome.services.RawPixelsBean.close(RawPixelsBean.java:210)
                           at sun.reflect.GeneratedMethodAccessor771.invoke(Unknown Source)


which occurs when checksumming the pixels: RawPixelsBean.java

If I could get the server log from you or Douglas, that would be great. I'm primarily interested in knowing if this is a 32-bit issue while parsing such large files.

Cheers,
~Josh.

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Mon Nov 11, 2013 5:12 pm
by dpwrussell

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Tue Nov 12, 2013 8:00 am
by jmoore
Thanks, Douglas. What are the dimensions for this image?

Code: Select all
...
2013-11-10 18:23:45,221 INFO  [             ome.io.nio.RomioPixelBuffer] (erver-2160) Retrieving plane: 25x0x1757
2013-11-10 18:23:45,243 INFO  [             ome.io.nio.RomioPixelBuffer] (erver-2160) Retrieving plane: 26x0x1757
2013-11-10 18:23:45,253 INFO  [             ome.io.nio.RomioPixelBuffer] (erver-2160) Retrieving plane: 27x0x1757
2013-11-10 18:23:46,822 WARN  [              ome.services.RawPixelsBean] (erver-2160) calculateMessageDigest failed on /home/omero/OMERO.data/Pixels/Dir-042/42337
java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
        at ome.io.nio.RomioPixelBuffer.getRegion(RomioPixelBuffer.java:343)
        at ome.io.nio.RomioPixelBuffer.getPlane(RomioPixelBuffer.java:489)
        at ome.io.nio.RomioPixelBuffer.calculateMessageDigest(RomioPixelBuffer.java:766)
        at ome.services.RawPixelsBean.save(RawPixelsBean.java:182)
        at ome.services.RawPixelsBean.close(RawPixelsBean.java:210)
...
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
        ... 58 more


Cheers,
~Josh

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Tue Nov 12, 2013 11:36 am
by hroque
Hi Josh,

Not sure this is what you mean but here it is.
This has happen with several images of different sizes:

Here is 2 examples:

Pixels XY - 336x256
Pixel Type: Unsigned 16bit
Pixel size XY- 0.222x0.222
z-sections/timepoints - 44 x 1696
Channels - 0
Folder size: 13.14 Gb


Pixel XY - 704x512
Pixel Type Unsigned 16bit
Pixel Size XY - 0.111x0.111
Z sections/timepoints 37x1841
Folder Size: 49.36 Gb

Cheers,
Helio

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Wed Nov 13, 2013 10:14 am
by dpwrussell
Server Java is:

java version "1.6.0_33"
Java(TM) SE Runtime Environment (build 1.6.0_33-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.8-b03, mixed mode)

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Wed Nov 13, 2013 10:50 am
by manics
Thanks Douglas. This might be a Java bug, though it's impossible to tell without testing. I've added a trac ticket to investigate it http://trac.openmicroscopy.org.uk/ome/ticket/11675

Re: MessageSize and server OOM (possibly) related problems

PostPosted: Wed Nov 13, 2013 11:34 am
by dpwrussell
I note that your default Java is now openjdk rather than oracle java which is what was recommended at the time I installed this server. I can try switching to see if the problem goes away if you like?