Referencing an instrument

[maxgitt]

Hi Josh,

Your guidance has really pushed me in the right direction. I am finally piecing the puzzle together to get my own server up and running. With the location of the files found, I would like to do two things:

1) backup the files from within /opt/omero/web
2) edit the files from my Host computer

I feel this can be accomplished simultaneously via Docker volumes. I ran the following to get the OMERO.web container started.

Code: Select all

docker run -d --name omero-web -e OMEROHOST=10.108.21.205 -v ~/workspace/omero/web:/opt/omero/web -p 4080:4080 maxgitt/omero-web-standalone:5.4

Notice the docker image is from a forked repository of mine. I needed to make a change to the Ansible playbook from `localhost` to my IPV4 address. Now the issue:

* By including the -v arg, my docker container immediately exits.

* Also, if I take a look at my Host directory, most of the correct files are missing. I believe this is a permissions issue. See below attachment.

missing-omero-volumes.png (23.23 KiB) Viewed 3285 times

Here's what I've found,

Code: Select all

Host $UID = 1000
Docker $UID = 998


Is this mismatch creating issues? I tried entering the Docker container and changing the ownership of all files but even then I don't know the omero-web password.

chown-wrongpassword.png (64.46 KiB) Viewed 3285 times

I found the Dockerfile where the `USER omero-web` is being set but can't find where I am to set the password.

Recap:
I would like to use Docker volumes to edit the Django apps in an IDE (which simultaneously gives me the ability to back up the files easily), but Host can't access the files found within the omero-web container.

Thank you ahead of time,
Max

[jmoore]

Hi Max,

here's a short recipe that worked for me:

Code: Select all

docker run -d --name tmp-web openmicroscopy/omero-web-standalone:5.4
docker cp tmp-web:/opt/omero/web web
docker rm -f tmp-web
docker run -v $PWD/web:/opt/omero/web -p 4080:4080 openmicroscopy/omero-web-standalone:5.4


Then:

• load localhost:4080 in the browser
• edit web/OMERO.py-5.4.0-ice36-b74/lib/python/omeroweb/webclient/templates/webclient/login.html
• reload localhost:4080 and see the change

This isn't an optimal development workflow, so if you have suggestions, don't hesitate to let us know.

~Josh

[maxgitt]

I tried your commands verbatim:

Code: Select all

docker run -d --name tmp-web openmicroscopy/omero-web-standalone:5.4
docker cp tmp-web:/opt/omero/web web
docker rm -f tmp-web
docker run -v $PWD/web:/opt/omero/web -p 4080:4080 openmicroscopy/omero-web-standalone:5.4
Running /startup/50-config.py
WARNING:omero.util.TempFileManager:Invalid tmp dir: /opt/omero/web/omero/tmp
Traceback (most recent call last):
  File "/opt/omero/web/OMERO.web/lib/python/omero/util/temp_files.py", line 172, in tmpdir
    self.create(target)
  File "/opt/omero/web/OMERO.web/lib/python/omero/util/temp_files.py", line 240, in create
    dir.makedirs(0700)
  File "/opt/omero/web/OMERO.web/lib/python/path.py", line 1244, in makedirs
    os.makedirs(self, mode)
  File "/opt/omero/web/venv/lib64/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/opt/omero/web/omero/tmp'
WARNING:omero.util.TempFileManager:Invalid tmp dir: /opt/omero/web/omero/tmp
Traceback (most recent call last):
  File "/opt/omero/web/OMERO.web/lib/python/omero/util/temp_files.py", line 172, in tmpdir
    self.create(target)
  File "/opt/omero/web/OMERO.web/lib/python/omero/util/temp_files.py", line 240, in create
    dir.makedirs(0700)
  File "/opt/omero/web/OMERO.web/lib/python/path.py", line 1244, in makedirs
    os.makedirs(self, mode)
  File "/opt/omero/web/venv/lib64/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/opt/omero/web/omero/tmp'
Cannot modify /opt/omero/web/OMERO.web/etc/grid/config.xml
Traceback (most recent call last):
  File "/startup/50-config.py", line 17, in <module>
    assert rc == 0
AssertionError
maxgitt@mgpc:~/workspace/omero$


[jmoore]

Hi Max,

interesting. I tested this on https://docs.docker.com/docker-for-mac/ (17.09.0-ce, build afdb6d4). Can you let me know what environment you're working with?

Cheers,
~Josh

[maxgitt]

Ubuntu, docker 17.09.0-ce

If I throw in TTY as a Docker arg, I see that the commandline is throwing a missing directory:

web-standalone-missingdir.png (19.55 KiB) Viewed 3263 times

Does this give us some insight as to what may be going on?

[maxgitt]

What I find interesting is that when I run a sample docker container and set the UID and GID like so:

Code: Select all

docker run -it -d -v $PWD:/home --user 1000:1000 ubuntu

I have no problem accessing/creating files from within the container or host. But with omero-web practically none of the files within /opt/omero/web/ show up. The only exception is an empty ~/workspace/omero/web/OMERO.web/var/ directory.

What is to keep me from changing the Dockerfile's user from omero-web to maxgitt. And isn't the volume command supposed to produce a directory on the host machine? What is the purpose of "VOLUME ["/opt/omero/web/OMERO.web/var"]"?

[jmoore]

Hi Max,

it looks like:

Code: Select all

  -a, --archive       Archive mode (copy all uid/gid information)


should do what you want. I've tried it on centos 7 (17.06.0-ce, build 02c1d87) however, and it's not working:

Code: Select all

docker run -d --name tmp-web openmicroscopy/omero-web-standalone:5.4
docker cp -a tmp-web:/opt/omero/web web # -a added here
docker rm -f tmp-web
docker run -v $PWD/web:/opt/omero/web -p 4080:4080 openmicroscopy/omero-web-standalone:5.4


The ownership of the files switches from "omero-web" to "root". You might try on your system to see if it happens to work with the newer version. If not, we'll need to look into an alternative.

Cheers,
~Josh

[maxgitt]

No luck:

denied.png (94.13 KiB) Viewed 3262 times

My question is why we need to create this temporary docker container that is copied. Why are we having permission issues in the first place? Is there really no direct approach?

Edit:
Actually extremely close!

Although the above error happened, most of the directories were copied over. The only important one "OMERO.web" is still inaccessible via the commandline because it is simply a symbolic link to the corresponding file in the container.

mostly-configured.png (39.81 KiB) Viewed 3262 times

[jmoore]

What is to keep me from changing the Dockerfile's user from omero-web to maxgitt.

If you are building it yourself, nothing.

And isn't the volume command supposed to produce a directory on the host machine?

More or less.

What is the purpose of "VOLUME ["/opt/omero/web/OMERO.web/var"]"?

This tells docker that you want the 'var' directory to be accesible, mountable, etc. If you are building yourself, you might try doing the same with all of '/opt/omero/web'. Then with 'docker volume ls' followed by 'docker volume inspect $VID' you should be able to see the volume created. It will likely live under /var/lib/docker/volumes.

My question is why we need to create this temporary docker container that is copied. Why are we having permission issues in the first place? Is there really no direct approach?

If you are trying to edit files which are natively in the docker, then this is the best approach I know of. (Other than rebuilding after every edit.) If you were adding files to the docker, then you could mount those from your host into the container at the appropriate location.

Edit:
Actually extremely close!

Although the above error happened, most of the directories were copied over. The only important one "OMERO.web" is still inaccessible via the commandline because it is simply a symbolic link to the corresponding file in the container.

Makes sense that links would only work in the container. But are you able to restart the server with the mounted version?

At the moment, setting the UID in omero-web-docker is not simple. (See issue 6) I've worked around by using my own ID in a dockerfile:

Code: Select all

FROM openmicroscopy/omero-web-standalone:5.4

USER root
RUN usermod -u 11521 omero-web
RUN groupmod -g 8000 omero-web
RUN find /opt/omero -user 998 -exec chown -h omero-web {} \;
RUN find /opt/omero -group 997 -exec chgrp -h omero-web {} \;
USER omero-web


and then running this newly built container:

Code: Select all

IMAGE=omero-web-11521
docker run -d --name tmp-web $IMAGE
docker cp -a tmp-web:/opt/omero/web web
docker rm -f tmp-web
mkdir -p var
docker run -ti -v $PWD/var:/opt/omero/web/OMERO.web/var -v $PWD/web:/opt/omero/web -p 4080:4080 $IMAGE


~Josh

[evenhuis]

Hi Will,

We are also interested in a use case like the one describe above. It would great to hear how other OMERO users have tackled this.

For some instruments there lot of metadata that isn't captured by the manufacturer's software or is specific to the facility. For example, some of our instruments don't communicate to the attached lightsource so this won't be in the image file metadata.

Based on your posts above, I was thinking was solution would be to have images tagged to identify the specific instrument in the facility (something like "microscope:NikonTi-inverted"). Hopefully this can be done in the acquisition software someplace.

Then having a python script in OMERO that adds the more metadata by looking up records about the facility.

Cheers,

Chris