We're Hiring!

Deleting images doesn't free space

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

Deleting images doesn't free space

Postby dgarneau » Fri May 06, 2016 2:40 pm

Hi,

I recently deleted many images to free some space on our server but found out, after the images were removed, that the space doesn't seems to be freed. I didn't delete the images directly but the project (or the dataset I don't remember) containing the images. Can it be the problem ?

Can the images still be there or is it a problem with storage calculation ? If the images are still there how can I delete them ?

As additional information, I looked at the space using the Admin tool in Omero web and I deleted the images with Insight. We are using OMERO 5.2.2. In insight if I go in the "Image" tab (in the left) it mention there is no images for that user.

Thank you
Daniel
dgarneau
 
Posts: 22
Joined: Fri Apr 01, 2016 8:16 pm

Re: Deleting images doesn't free space

Postby wmoore » Mon May 09, 2016 2:39 pm

Hi Daniel,

When you delete a Dataset, this should delete the images it contains *unless* the images are also in another Dataset.

For the images you've already deleted, you could try to search for them (in Insight or web client) to see if you can find them in another Dataset.

If you're trying this again with other images, before you delete the Dataset, you can tell if the images are in other Datasets by selecting an image and in the right panel look under "Located in" pane in Insight (bottom of General tab) or under the 'tree' icon at top toolbar in right panel of webclient.
Also, if you click the 'link' button (top of right panel in webclient) you can get a bookmark link to an image. Then you can check after deleting whether the image has gone by trying to visit that link in your browser.

You can directly delete the images (select the images and 'delete') which will delete them even if they are in multiple Datasets.

How did you import these images?
If the are an "in place" import via the command line, then their location on disk will not affect the disk usage in OMERO so you will not see a change when they are deleted.

If you see no images for that user in the Images tab in Insight, that would suggest that they have been deleted (if you saw them there before deleting?).

Hope some of this is useful. Let us know if this doesn't explain what you're seeing.

Cheers,

Will.
User avatar
wmoore
Team Member
 
Posts: 674
Joined: Mon May 18, 2009 12:46 pm

Re: Deleting images doesn't free space

Postby dgarneau » Mon May 09, 2016 4:12 pm

Hi,

I cannot find these images anywhere else. What's weird is, if I look under the group and user in the Admin tool in OMERO web, it tells me the user is using 168 GB of space but if I browse for images under this group/user, I cannot see any images. One other thing, if I go to History in OMERO web, I can still see some folder icons but when I click on them I got a message "No data".

the images were imported using Insight.
Is there a way I can clear that space ?

Thank you again for your help,
Daniel
dgarneau
 
Posts: 22
Joined: Fri Apr 01, 2016 8:16 pm

Re: Deleting images doesn't free space

Postby manics » Mon May 09, 2016 9:53 pm

Hi Daniel

If you've got access to the omero command line could you try the following, to list any additional files that might have been uploaded?

Code: Select all
$ omero -u <username> -g <groupname> --sudo root hql 'FROM OriginalFile'
Previous session expired for <username> on xxx:4064
Server: [eel:4064]
Password for root:
Created session XXXXX (<username>@xxx:4064). Idle timeout: 10 min. Current group: private-1
...


See https://www.openmicroscopy.org/site/sup ... sions.html if you want more explanation about the command line parameters. If you also run this with Image instead of OriginalFile you should see whether there are additional images that Insight isn't seeing for whatever reason.
User avatar
manics
Team Member
 
Posts: 261
Joined: Mon Oct 08, 2012 11:01 am
Location: Dundee

Re: Deleting images doesn't free space

Postby dgarneau » Wed May 18, 2016 3:42 pm

Hi,

We tried the commands suggested above. The first one gives us many results (we couldn't get through all of it). If we try Image instead of OriginalFile, we got some images listed but not the images concerned. If we browse through the folder, we can see the images. I suppose it's not safe to delete them directly ? Is there a way to erase these images ?

Thank you again,
Daniel
dgarneau
 
Posts: 22
Joined: Fri Apr 01, 2016 8:16 pm

Re: Deleting images doesn't free space

Postby cblackburn » Thu May 19, 2016 9:03 am

Hi Daniel,

It's possible that the delete failed to remove the underlying image files for some reason. You could try to remove any of these remaining files using the cleanse command. It can be used in a dry-run mode to identify any files that need removing. If you can try:

Code: Select all
bin/omero admin cleanse --dry-run data_dir


as an admin with the full path to the OMERO data directory. If this does identify the files you tried to delete than running it without the dry-run option will attempt to delete those files.

Could you let us know, first, whether the command with the dry-run option does identify the files you expected to have been deleted? If it does then could you let us know whether a run without the dry-run option does succeed?

Cheers,

Colin
cblackburn
 
Posts: 85
Joined: Mon May 25, 2009 9:03 pm

Re: Deleting images doesn't free space

Postby dgarneau » Fri May 20, 2016 3:54 pm

Hi,

We tried the cleanse command. The dry-run identified many files/folders but didn't identified the file we expected. We still tried the cleanse without the dry option but we still have these images. Now the cleanse --dry-run return nothing.

We really need to delete these files, is there any further thing we could try ?

Thanks,
Daniel
dgarneau
 
Posts: 22
Joined: Fri Apr 01, 2016 8:16 pm

Re: Deleting images doesn't free space

Postby jmoore » Mon May 23, 2016 9:35 am

Hi Daniel,

sorry for the troubles, but we'll need to figure out what's consuming the space to help you delete it. Could you show us the output of the CLI command `fs usage`:

Code: Select all
$ omero -q fs usage ExperimenterGroup:751 --report --human-readable
Total disk usage: 1.7 GB bytes in 4614 files
user | group | component    |     size | files
------+-------+--------------+----------+-------
3204 | 751   | Pixels       | 970.8 MB |   281
3204 | 751   | Annotation   | 294.7 KB |   284
3204 | 751   | Thumbnail    |   2.0 MB |   423
3204 | 751   | OriginalFile |   3.1 KB |     2
3251 | 751   | Thumbnail    |   1.7 MB |   906
3251 | 751   | Pixels       | 805.3 MB |  1812
3251 | 751   | Annotation   | 274.3 KB |   906
(7 rows)


Cheers,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Deleting images doesn't free space

Postby dgarneau » Wed May 25, 2016 1:07 pm

Hi,
here are the results

Code: Select all
omero@omero3:~$ omero -q fs usage ExperimenterGroup:3 --report --human-readable Total disk usage: 175.0 GB bytes in 21591 files
user | group | component    |     size | files
------+-------+--------------+----------+-------
2    | 3     | FilesetEntry |   7.1 GB |    26
2    | 3     | Job          | 711.7 KB |    23
2    | 3     | Thumbnail    |  42.4 KB |    20
2    | 3     | Annotation   |  41.6 MB |     2
2    | 3     | OriginalFile |  17.3 KB |    28
803  | 3     | Job          |   3.9 KB |     1
803  | 3     | FilesetEntry | 167.9 GB | 21493
(7 rows)


omero@omero3:~$ omero -q fs usage Experimenter:803 --report --human-readable Total disk usage: 167.9 GB bytes in 21494 files
user | group | component    |     size | files
------+-------+--------------+----------+-------
803  | 3     | FilesetEntry | 167.9 GB | 21493
803  | 3     | Job          |   3.9 KB |     1
(2 rows)


Thanks for the help,
Daniel
dgarneau
 
Posts: 22
Joined: Fri Apr 01, 2016 8:16 pm

Re: Deleting images doesn't free space

Postby jmoore » Wed May 25, 2016 1:22 pm

Hi Daniel,

do you know which of the two users you're looking to delete from? (bin/omero user list) Also, do you have a rough idea of when the import was performed?

Cheers,
~Josh.
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Next

Return to User Discussion

Who is online

Users browsing this forum: No registered users and 1 guest