We're Hiring!

Error on import: Directory exists but is not registered

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

Error on import: Directory exists but is not registered

Postby ratamero » Thu Jul 12, 2018 10:26 am

Dear all,

we're having a big issue here - after a few crashed imports, we're getting a "Error on import: Directory exists but is not registered: CheckedPath(petabyte)" error whenever trying to import anything. I've found a couple of previous cases where the same error was caused by an empty directory inside ManagedRepository (in our case, we would expect it to be the "petabyte" directory), and deleting the empty directory restored it.

Now here's the problem: We're using a custom omero.fs.repo.path that looks something like "petabyte/%groupId%/%user%_%userId%//%year%-%month%/%day%/%time%". So all our data is inside the "petabyte" folder (which is actually just a symlink to a mounted storage server share). It's not an empty directory - very much the opposite, in fact.

I've tried removing the symlink and importing data - works normally and creates a "petabyte" directory, as expected. However, "petabyte" has always been a symlink to the external storage server, and all our data is already sitting there. I'm trying to chase down the data from the crashed imports and thought maybe deleting it will sort it, but let me know if you have any other ideas. Worst case scenario is having to restore the server to its previous backup status.

best regards

Erick
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Error on import: Directory exists but is not registered

Postby mtbc » Thu Jul 12, 2018 10:52 am

Dear Erick,

What does "bin/omero fs repos" report? Specifically, I am wondering if there's some wrong duplication there.

In general before making any manual change like adjusting directories under the server I would recommend a full backup in case the resulting behavior is to effect some troublesome state changes from which it is difficult to recover. Also be aware that we offer OMERO Docker images in which you might experiment to test the effects of various server/FS changes or recovery procedures in a safe place.

I wonder what originally happened to cause this problem. Could there have been some glitch in your networked storage that perhaps OMERO.server should have handled better? Nothing else is known to have changed that might have caused the problem?

Cheers,
Mark
User avatar
mtbc
Team Member
 
Posts: 282
Joined: Tue Oct 23, 2012 10:59 am
Location: Dundee, Scotland

Re: Error on import: Directory exists but is not registered

Postby ratamero » Thu Jul 12, 2018 10:57 am

I think it might have run out of memory (I was running multiple imports in parallel when it happened). Another possibility is that it might have been a consequence of the server crashing when I tried to view a Slidebook file (Slidebook6Reader just loves to crash our server).

You're right, there's some strange duplication on the repos:

Code: Select all
# | Id     | UUID                                 | Type    | Path                                                 
---+--------+--------------------------------------+---------+------------------------------------------------------
0 | 57     | ee6b9e6a-a545-4269-916f-f65d89f01711 | Public  | /data/omerodata                                     
1 | 58     | ScriptRepo                           | Script  | /home/admin/OMERO.server-5.4.6-ice36-b87/lib/scripts
2 | 59     | e4910386-fcb6-4f3e-934b-d8e55c022f45 | Public  | /data/omerodata/ManagedRepository                   
3 | 2051   | 8f452952-9c48-49e1-981d-28d2a0659441 | Public  | /data/omerodata                                     
4 | 2052   | a552ca72-9238-4e8b-939e-ded1701e0ca9 | Public  | /data/omerodata/ManagedRepository                   
5 | 211101 | 958ce441-ce74-403e-a412-88e923764264 | Managed | /data/omerodata/ManagedRepository 



So what's the safe way to fix this?
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Error on import: Directory exists but is not registered

Postby mtbc » Thu Jul 12, 2018 12:19 pm

Dear Erick,

Out of curiosity, how is your external storage server mounted? Is it NFS v3 or somesuch? I wonder if at some point the storage disappeared under the server.

What are the contents of the binary repository's repo_uuid files? (There is probably one in .omero/repository/*/repo_uuid and in ManagedRepository/.omero/repository/*/repo_uuid.)

Which managed repository has the bulk of your data? Something like,
Code: Select all
psql -c "SELECT repo, COUNT(*) FROM originalfile WHERE repo IS NOT NULL GROUP BY repo ORDER BY COUNT(*) DESC" -U db_user omero_database

Also, I wonder about the petabyte directories now registered with OMERO,
Code: Select all
psql -c "SELECT id, path, repo FROM originalfile WHERE name = 'petabyte'" -U db_user omero_database

As background, you'll want the repo_uuid files to match what's in the repo column for the originalfile database table rows that list the image data that you actually want to keep.

Cheers,
Mark
User avatar
mtbc
Team Member
 
Posts: 282
Joined: Tue Oct 23, 2012 10:59 am
Location: Dundee, Scotland

Re: Error on import: Directory exists but is not registered

Postby ratamero » Thu Jul 12, 2018 1:17 pm

it's an SMB3 mount, IIRC. I'm pretty sure it hasn't disappeared at any given point.

I have /data/omerodata/.omero/repository/0f9bd6b7-c89d-4414-a13e-047ceaa85d4f/repo_uuid with the contents

Code: Select all
^@$8f452952-9c48-49e1-981d-28d2a0659441


and /data/omerodata/ManagedRepository/.omero/repository/0f9bd6b7-c89d-4414-a13e-047ceaa85d4f/repo_uuid with contents

Code: Select all
^@$958ce441-ce74-403e-a412-88e923764264


running the first postgres command have me

Code: Select all
                 repo                 | count 
--------------------------------------+--------
a552ca72-9238-4e8b-939e-ded1701e0ca9 | 193228
e4910386-fcb6-4f3e-934b-d8e55c022f45 |    234
ScriptRepo                           |    150
958ce441-ce74-403e-a412-88e923764264 |      6


and the second one gave me

Code: Select all
   id   | path |                 repo                 
--------+------+--------------------------------------
   2071 | /    | a552ca72-9238-4e8b-939e-ded1701e0ca9
211423 | /    | 958ce441-ce74-403e-a412-88e923764264


Thanks in advance!
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Error on import: Directory exists but is not registered

Postby ratamero » Thu Jul 12, 2018 1:47 pm

Quick update: I think it's sorted! I edited the repo_uuid files to correspond to the repos I wanted and it seems to be back to normal after a restart. I'll report back if it breaks again :)
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm

Re: Error on import: Directory exists but is not registered

Postby mtbc » Thu Jul 12, 2018 1:54 pm

Great, thank you very much for letting us know, Erick. At some later point we may come back to this thread with further questions: we'd definitely like to help prevent users from getting into these states. If you notice any further problems or strangeness then indeed please don't hesitate to let us know the new state of things and we will try to help you sort them out.

Cheers,
Mark
User avatar
mtbc
Team Member
 
Posts: 282
Joined: Tue Oct 23, 2012 10:59 am
Location: Dundee, Scotland

Re: Error on import: Directory exists but is not registered

Postby ratamero » Thu Jul 12, 2018 2:19 pm

No worries. Thanks again, Mark!
ratamero
 
Posts: 46
Joined: Mon Jan 29, 2018 5:55 pm


Return to User Discussion

Who is online

Users browsing this forum: Google [Bot] and 0 guests

cron