Page 1 of 1

Multiple Omero Data Directories

PostPosted: Thu Apr 26, 2018 10:22 am
by dwj
Hi OMERO developers.

In our OMERO environment we use a network share as ‘omero.data.dir’. That works very smoothly.
Our network administration offers network shares with a maximum size of 16TB. Since we need more than 16TB, we could get a second network share with the same size of 16TB.

Is it possible to configure OMERO to use two or more network shares as ‘omero.data.dir’ so that we can use the complete 32TB?

I also think about using a Software-RAID to abstract multiple network shares to one big share. Does anybody have experiences with a software RAID? We use CentOS 7 as operation system.

Thanks,
Daniel

Re: Multiple Omero Data Directories

PostPosted: Thu Apr 26, 2018 2:45 pm
by kennethgillen
dwj wrote:Hi OMERO developers.


Hi Daniel,

dwj wrote:Is it possible to configure OMERO to use two or more network shares as ‘omero.data.dir’ so that we can use the complete 32TB?


Yes, The OMERO managed repository, where newly imported data ends up, could be organised in such a way that some data is on one of your existing storage paths (in this case your network share) and newly imported data goes to another.

Our docs show the process under https://docs.openmicroscopy.org/latest/ ... repository - a good diagram showing the repository as new data is added is at http://downloads.openmicroscopy.org/pre ... paris/#/16

The binary repository layout itself (including managed repository) is discussed in our docs here: https://docs.openmicroscopy.org/latest/ ... itory.html

dwj wrote:I also think about using a Software-RAID to abstract multiple network shares to one big share. Does anybody have experiences with a software RAID? We use CentOS 7 as operation system.


Software RAID isn't designed to work with remote filesystems like NFS mounts. You would be better off with a different filesystem type which does support sizes >16TB. There are a few options out there, which can be discussed with the team who are providing the storage infrastructure you're currently using.

Hope that's useful.

Kenny

Re: Multiple Omero Data Directories

PostPosted: Mon May 07, 2018 2:27 pm
by dwj
Hi Kenny,

Thanks for your hints.
I changed the ‘omero.managed.dir’ settings and all new imaged files are stored in that new repository. So that works fine.

Is it possible that new images for one group (or user) are stored in one repository and the images of a second group (or user) are stored in a second repository?

Thanks,
Daniel

Re: Multiple Omero Data Directories

PostPosted: Mon May 07, 2018 3:38 pm
by jmoore
Hi Daniel,

dwj wrote:Is it possible that new images for one group (or user) are stored in one repository and the images of a second group (or user) are stored in a second repository?


There's only one repository, but you can change the template prefix to additionally take group information into account.

Note though that this only holds for new imports and if a user changes groups the data is not automatically migrated.

Cheers,
~Josh

Re: Multiple Omero Data Directories

PostPosted: Tue May 08, 2018 11:50 am
by dwj
OK,
thank you Josh!