We're Hiring!

Dropbox: clean up (remove) imported files

Having a problem deploying OMERO? Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

The OMERO.server installation documentation begins here and you can find OMERO.web deployment documentation here.

Dropbox: clean up (remove) imported files

Postby ehrenfeu » Wed Dec 10, 2014 1:16 pm

Dear all,

we're having the following scenario: OMERO 5.0.5 is running on a dedicated machine with the space for the binary repository being provided by a central storage machine via NFS. To use OMERO.dropbox, our server has an additional ~1TB of local storage attached, which is configured as the "watchDir" and being exposed to the users via Samba / CIFS.

From the thread auto-importer I'm reading that we could configure OMERO to remove files from the watchDir once the import has succeeded using
Code: Select all
--transfer=upload_rm


Is this correct? From how I understand, we can't use the "ln_rm" option as our watchDir and the ManagedRepository reside on independent file systems each, right?

Thanks
~Niko
User avatar
ehrenfeu
 
Posts: 90
Joined: Fri May 11, 2012 8:21 am
Location: Basel, Switzerland

Re: Dropbox: clean up (remove) imported files

Postby jmoore » Wed Dec 10, 2014 8:08 pm

Hi Niko,

ehrenfeu wrote:From the thread auto-importer I'm reading that we could configure OMERO to remove files from the watchDir once the import has succeeded using --transfer=upload_rm


Yup, that's just what it's intended for.

From how I understand, we can't use the "ln_rm" option as our watchDir and the ManagedRepository reside on independent file systems each, right?


Also correct: regular hard-linking boundaries apply. And a "ln_s_rm" wouldn't get you very much! :) (That being said, if you'd like to try out a "cp_rm" we could certainly give it a try.)

Cheers,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Dropbox: clean up (remove) imported files

Postby ehrenfeu » Fri Dec 12, 2014 3:54 pm

Hi Josh,

thanks for the update! Are we correct in assuming this option has to go here?

Code: Select all
<property name="omero.fs.importArgs"  value=""/>


Cheers
~Niko
User avatar
ehrenfeu
 
Posts: 90
Joined: Fri May 11, 2012 8:21 am
Location: Basel, Switzerland

Re: Dropbox: clean up (remove) imported files

Postby cblackburn » Mon Dec 15, 2014 8:22 am

Hi Niko,

Yes, that's correct. You can also set the configuration via the command line, as illustrated here,

https://www.openmicroscopy.org/site/sup ... ml#dropbox

Cheers,

Colin
cblackburn
 
Posts: 85
Joined: Mon May 25, 2009 9:03 pm

Re: Dropbox: clean up (remove) imported files

Postby ehrenfeu » Mon Dec 22, 2014 4:05 pm

Thanks for the info! We'll try and see how it goes...

Josh, could you tell us details what's different in "cp_rm"?

Thanks
~Niko
User avatar
ehrenfeu
 
Posts: 90
Joined: Fri May 11, 2012 8:21 am
Location: Basel, Switzerland

Re: Dropbox: clean up (remove) imported files

Postby jmoore » Tue Dec 23, 2014 9:32 am

Hi Niko,

`cp_rm` doesn't exist yet, but it's a pretty simple manner of modifying this line in HardlinkFileTransfer to use "cp" rather than "ln". The difference is that a full copy takes place but locally and likely much faster than uploading via Ice (though I haven't compared the speed yet).

Cheers,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Dropbox: clean up (remove) imported files

Postby jmoore » Mon Jan 26, 2015 10:37 am

Hi Niko,

just a follow-up that https://github.com/openmicroscopy/openm ... /pull/3379 has been merged so `cp_rm` should be in the nightly builds as well as 5.0.7 for your review.

All the best,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany


Return to Installation and Deployment

Who is online

Users browsing this forum: No registered users and 0 guests