We're Hiring!

postgres volume recommended size

Having a problem deploying OMERO? Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

The OMERO.server installation documentation begins here and you can find OMERO.web deployment documentation here.

postgres volume recommended size

Postby moisbo » Thu Jan 04, 2018 3:35 am

Hi,

(First thanks for the responses!)

I would like to create a volume for postgres in my OMERO.db vm. I would like to know your thoughts on the recommended volume size for postgres database. I see there are some recommendations on the docs, but not specifically for the postgres data_directory size.

Cheers,
Moises
moisbo
 
Posts: 8
Joined: Thu Aug 03, 2017 7:18 am
Location: Australia

Re: postgres volume recommended size

Postby manics » Thu Jan 04, 2018 9:35 am

Hi Moises

Could you give us some details about your OMERO server, such as number of users, number of images/datasets, types of images, whether you plan to store additional metadata, etc?
User avatar
manics
Team Member
 
Posts: 261
Joined: Mon Oct 08, 2012 11:01 am
Location: Dundee

Re: postgres volume recommended size

Postby moisbo » Fri Mar 02, 2018 12:43 am

Hi Manics,

Appreciate your reply. We are gathering that information and will get back to you. For the moment we are implementing in a filesystem where we are able to grow the storage on the postgres database with an initial 10GB and it is being monitored.

Many Thanks,
Moises
moisbo
 
Posts: 8
Joined: Thu Aug 03, 2017 7:18 am
Location: Australia

Re: postgres volume recommended size

Postby moisbo » Mon Mar 12, 2018 1:15 am

Hi Manics,

Could you give us some details about your OMERO server, such as number of users

Around 150 Users
number of images/datasets

1.8 million images, around 10 TB per year
types of images

raw images from microscope
whether you plan to store additional metadata, etc?

YES

Many Thanks,
Moises
moisbo
 
Posts: 8
Joined: Thu Aug 03, 2017 7:18 am
Location: Australia

Re: postgres volume recommended size

Postby manics » Mon Mar 12, 2018 3:53 pm

Hi Moises

Here are the stats for our internal production OMERO:
- 30 TB data (375,000 images)
- 70 GB postgres
- 450 users
- 270,000 annotations

For comparison the IDR (https://idr.openmicroscopy.org/):
- 50TB data (3,000,000 images)
- 150 GB postgres
- 3,200,000 annotations

So I think around 40GB for postgres would be a good value for you to start with, especially if it's easy for you to expand it later. If you prefer to minimise storage changes then 100GB would be safer.
User avatar
manics
Team Member
 
Posts: 261
Joined: Mon Oct 08, 2012 11:01 am
Location: Dundee

Re: postgres volume recommended size

Postby moisbo » Thu Mar 15, 2018 2:56 am

Hi Manics,

Thanks a lot for your response. As a side question (I don't know if I need to do a new post)

We are currently hosting on a VM with 16GB RAM with . Would you care to share your configuration?
This is ours:
Code: Select all
omero.jvmcfg.heap_size=12g
omero.jvmcfg.percent.blitz=60
omero.jvmcfg.perm_gen=512m
omero.jvmcfg.strategy=percent


We would like to get more, however, we need to build a strong case for the IT department to provide for more.

Cheers,
Moises
moisbo
 
Posts: 8
Joined: Thu Aug 03, 2017 7:18 am
Location: Australia

Re: postgres volume recommended size

Postby manics » Thu Mar 15, 2018 10:15 am

Our internal OMERO is a VMWare VM with 32 GB RAM, and the following settings:
Code: Select all
omero.jvmcfg.percent.blitz=50
omero.jvmcfg.percent.indexer=20
omero.jvmcfg.percent.pixeldata=20
omero.jvmcfg.system_memory=17000


Most of the setup is done with Ansible, if you're interested have a look here: https://github.com/openmicroscopy/prod- ... o.yml#L270

Simon
User avatar
manics
Team Member
 
Posts: 261
Joined: Mon Oct 08, 2012 11:01 am
Location: Dundee

Re: postgres volume recommended size

Postby moisbo » Thu Mar 15, 2018 10:39 pm

Thanks for that. We've deployed with Ansible as well so that is helpful

Moises
moisbo
 
Posts: 8
Joined: Thu Aug 03, 2017 7:18 am
Location: Australia


Return to Installation and Deployment

Who is online

Users browsing this forum: No registered users and 1 guest