We're Hiring!

Enforcing storage limit per user/group

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

Enforcing storage limit per user/group

Postby ratamero » Mon Jan 29, 2018 6:10 pm

Hi there!

we're in the process of setting up an OMERO install at Warwick Medical School - we expect this to use large amounts of storage in the next few years (petabyte-scale), so ideally we'd want to enforce maximum storage limits per group/PI. I did some searching on my own, but couldn't find any way of doing it. Is it a feature that exists? If not, is it a feature that is planned in the near future? :)

Thanks in advance!
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Enforcing storage limit per user/group

Postby jmoore » Mon Jan 29, 2018 7:23 pm

OMERO does not currently provide a quota feature. The closest that is provided by default is the ability to configure the layout of /OMERO/ManagedRepository so that you can both define quotas at the filesystem level as well as expand storage as your server grows.

See http://downloads.openmicroscopy.org/presentations/2014/fs-workshop-paris/#/14 for graphical examples of various configurations.

Regards,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Enforcing storage limit per user/group

Postby ratamero » Tue Jan 30, 2018 11:56 am

Cool, thanks! That will hopefully be enough to enforce quotas. I've edited omero.properties to use:

omero.fs.repo.path=%groupId%/%user%_%userId%//%year%-%month%/%day%/%time%

I've restarted the server, but it still seems that new data is being saved in the old repo path format - is tere anything I'm missing here?
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Enforcing storage limit per user/group

Postby mtbc » Tue Jan 30, 2018 12:28 pm

I can't see what you'd be missing. In a local test build I tried changing mine to:
Code: Select all
$ bin/omero config get omero.fs.repo.path
%groupId%/%user%_%userId%//%year%-%month%/%day%/%time%

Code: Select all
$ find ManagedRepository/ -name '*.dcm'
ManagedRepository/user-2_3/2018-01/30/12-19-06.099/ankle.dcm
ManagedRepository/3/user-2_3/2018-01/30/12-20-08.576/brain.dcm

I imported "ankle.dcm" before the change, then restarted the server with the new configuration and imported "brain.dcm".

My initial advice would be to,
  • use "config get" to double-check the configuration
  • restart the server
  • double-check that you're finding a genuinely new import in "ManagedRepository/".
If there's still a baffling lack of picking up the new configuration, please go ahead and send us the output of "bin/omero admin diagnostics" and paste a transcript of the checks you did via shell then we'll try to help you to get to the bottom of what's going on.

Cheers,
Mark
User avatar
mtbc
Team Member
 
Posts: 282
Joined: Tue Oct 23, 2012 10:59 am
Location: Dundee, Scotland

Re: Enforcing storage limit per user/group

Postby ratamero » Tue Jan 30, 2018 1:41 pm

For some reason omero.properties seem to be completely ignored - bin/omero config get omero.fs.repo.path returned empty. By doing bin/omero config set omero.fs.repo.path with the correct string, it seems to work now. Thanks again!
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Enforcing storage limit per user/group

Postby jmoore » Tue Jan 30, 2018 2:04 pm

ratamero wrote:For some reason omero.properties seem to be completely ignored


Ah, that would explain it. See the top of the file for an explanation.

Let us know if you have any other issues.
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Enforcing storage limit per user/group

Postby ratamero » Tue Jan 30, 2018 2:14 pm

Serves me right for not reading things :lol:
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm


Return to User Discussion

Who is online

Users browsing this forum: Google [Bot] and 1 guest

cron