We're Hiring!

BIoFormatsCache size

Having a problem deploying OMERO? Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

The OMERO.server installation documentation begins here and you can find OMERO.web deployment documentation here.

BIoFormatsCache size

Postby ratamero » Mon Jul 09, 2018 9:34 am

Dear all,

at the moment we're running our OMERO server on a machine with very limited storage, which means we're constantly getting Nagios warnings about running out of space - the main offender is always BioFormatsCache, which means every warning involves me having to ssh into the server and delete it. So two questions:

1) Is it possible to set the maximum size of BioFormatsCache?
2) If not, any idea how much storage I should provision for that?

Thanks in advance!

Erick
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: BIoFormatsCache size

Postby sbesson » Tue Jul 10, 2018 12:53 pm

Hi Erick,

The BioFormatsCache directory contains the cached readers for all the filesets imported in the ManagedRepository. If these cache files do not exist or are invalid e.g. following a server upgrade, they will be recreated on-the-fly when the server tries to access the raw data.

In terms of the size, it is hard to give precise numbers. In general, the expected disk usage will depend on two elements:
- the number of filesets imported in your database
- the nature of the file formats as certain readers cache much more metadata than others

If you want to maintain your current solution of deleting the cache directory, an option might be to adjust omero.pixeldata.memoizer_wait to a very large value. This should effectively disable the creation of the Bio-Formats cache files.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm

Re: BIoFormatsCache size

Postby kennethgillen » Tue Jul 10, 2018 1:21 pm

Hi Erick,

I'm not the expert here, but removing the Bio-Formats caches will have a performance impact for your end-users - as Sebastien mentioned, the re-generation of the files on-the-fly, so it might be worth considering a server migration to a system with more local disk.

Best regards,

Kenny
kennethgillen
 
Posts: 254
Joined: Mon Nov 05, 2012 3:39 pm

Re: BIoFormatsCache size

Postby ratamero » Tue Jul 10, 2018 1:44 pm

For the moment I'll probably settle for a middle-of-the-road solution: setting memoizer_wait to something vaguely sensible so that large files get cached, but small ones don't (our lack of space issue right now comes from TBs of data being imported in the form of many many very small TIFFs). Hopefully it won't impact performance too much and it should save a lot of space. In the meantime, it's time to go back to IT Services to beg for more storage, I guess. Thanks!
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: BIoFormatsCache size

Postby ratamero » Tue Jul 10, 2018 4:50 pm

Update: we managed to point the BioFormatsCache folder to our external storage server, where all the data sits. As long as cache doesn't hit 200TB we should be fine. Thanks again!
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm


Return to Installation and Deployment

Who is online

Users browsing this forum: No registered users and 1 guest

cron