We're Hiring!

out of memory error

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

out of memory error

Postby toh2 » Wed Apr 13, 2011 6:10 pm

Hi all,

I stuck with uploading .svs files by using omero.importer.
I got error message like as follows when I tried to upload .svs file.
Code: Select all
2011-04-13 11:37:39,158 456098     [2-thread-1] DEBUG      ome.formats.OMEROMetadataStoreClient  - Creating root!
2011-04-13 11:37:39,173 456113     [2-thread-1] ERROR    ome.formats.importer.gui.ImportHandler  - Generic error while importing image.
2011-04-13 11:37:39,173 456113     [2-thread-1] DEBUG      ome.formats.importer.gui.GuiImporter  - java.lang.OutOfMemoryError: Java heap space
at ome.formats.importer.ImportLibrary.writeDataPlanarBased(ImportLibrary.java:845)
at ome.formats.importer.ImportLibrary.importData(ImportLibrary.java:730)
at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:557)
at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:433)
at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:376)
at ome.formats.importer.gui.ImportHandler.importImages(ImportHandler.java:216)
at ome.formats.importer.gui.ImportHandler.access$100(ImportHandler.java:59)
at ome.formats.importer.gui.ImportHandler$2.run(ImportHandler.java:134)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

omero server is cent os, and I tested uploading on windows OS based pc.
Does Omero beta 4.2.2 support large scale of images to upload ?
also, Does Omero support .svs file format to upload ?
If Omero did, how I can extend memory heap size ?
It would be great if you guys give me any advices.

Thanks,
Steve Oh
toh2
 
Posts: 40
Joined: Tue Mar 01, 2011 6:59 pm

Re: out of memory error

Postby cxallan » Fri Apr 15, 2011 12:43 pm

We're solving many of these issues as part of Beta-4.3:

https://trac.openmicroscopy.org.uk/ome/ ... RO-Beta4.3

Yes we support SVS files but only if their total area is smaller than 4096x4096 with the default server and client settings. With heap size tweaks we can support up to approximately 8000x8000.

How large are the SVS files you're trying to work with?
cxallan
Site Admin
 
Posts: 509
Joined: Fri May 01, 2009 8:07 am

Re: out of memory error

Postby toh2 » Mon Apr 18, 2011 1:42 pm

It seems that largest size is 352MB when I checked image sizes.
Does Omero also support .ndpi format ?
ndpi image for our dataset seems pretty large. the largest one is 3.46GB
also, how I can fix out of memory error problem to upload svs ?
I couldn't find the fixing information for out of memory and enlarging heap size in the URL that you posted.
Thanks for your help in advance !!

Steve Oh
toh2
 
Posts: 40
Joined: Tue Mar 01, 2011 6:59 pm

Re: out of memory error

Postby toh2 » Mon Apr 18, 2011 2:05 pm

Also, you mean that only Beta-4.3 version support uploading svs format ?
Then, do we need to upgrade current version (4.2 version) into new version ?
or, is there anyway to fix uploading svs format without upgrading ?

Thanks for your comments in advance.

Steve Oh
toh2
 
Posts: 40
Joined: Tue Mar 01, 2011 6:59 pm

Re: out of memory error

Postby cxallan » Mon Apr 18, 2011 7:58 pm

OMERO Beta-4.2.x supports uploading SVS, the files just cannot be larger than 8000x8000 with the default heap sizes. If you want to increase the heap size you can find out how to do so on the server here:

http://www.openmicroscopy.org/site/supp ... leshooting

Support for Hamamatsu NanoZoomer data (NDPI) will be available in Beta-4.3. You can see the status of its testing and development here:

http://trac.openmicroscopy.org.uk/ome/ticket/4122

Adjusting the OMERO.importer heap size has to be done via the shell / batch script that starts the importer itself or via the plist file on Mac OS X.

OMERO Beta-4.3 will support SVS, NDPI and really any file format of unlimited XY size. You will need to upgrade to Beta-4.3 in order to have support for very large XY images when it is released.
cxallan
Site Admin
 
Posts: 509
Joined: Fri May 01, 2009 8:07 am

Re: out of memory error

Postby toh2 » Thu May 12, 2011 4:45 pm

Thanks for your comments.
most image datasets that we deal with are over '4096 X 4096' dimension.
it's like '38001 X 33348'. Although I tried to adjust heap size from 512MB to 1024MB, it seems not work.
In this case, we might need to wait until 4.3 Beta version is released. Could you let me know expecting release date of 4.3 Omero Beta version ? Or It would be great help to give me any advices or alternatives to upload svs file with higher dimension based on 4.2 version.

Thanks,
Steve Oh
toh2
 
Posts: 40
Joined: Tue Mar 01, 2011 6:59 pm

Re: out of memory error

Postby toh2 » Thu May 12, 2011 8:33 pm

Also, I have a question for the image viewer.
Emory University are tying to use the OMERO tool to do an image analysis for pathology images.
As you know, the pathology image itself has a high volume of image capacity (almost 3GB) and
high dimension like '100K X 100K' comparing to radiology images.
Even though OMERO beta 4.3 supports for uploading unlimited images sizes, dimensions and formats (svs, ndpi),
I am wondering whether the image visualization tool (such as omero.insight) can deal with such large size of pathology images efficiently. It's going to be helpful if OMERO can support a visualization tool that enables to deal with such large size of images efficiently and effectively.

Thanks,
Steve Oh
toh2
 
Posts: 40
Joined: Tue Mar 01, 2011 6:59 pm

Re: out of memory error

Postby wmoore » Fri May 20, 2011 9:56 am

Hi Steve,

We are currently developing "Big Image" viewers for both the OMERO.insight client and the OMERO.web web client. These should allow you to view images of 'unlimited' size, zoom, pan etc. They will be included in the OMERO 4.3 release.

Will.
User avatar
wmoore
Team Member
 
Posts: 674
Joined: Mon May 18, 2009 12:46 pm


Return to User Discussion

Who is online

Users browsing this forum: Google [Bot] and 0 guests