We're Hiring!

Omero don't start after upgrade

Having a problem deploying OMERO? Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

The OMERO.server installation documentation begins here and you can find OMERO.web deployment documentation here.

Omero don't start after upgrade

Postby phm » Wed Dec 06, 2017 2:42 pm

Hi,
I can't start omero after upgrade from 5.3.3 to 5.4.1.
Code: Select all
omero@omero:~$>omero admin start
Creating /home/omero/OMERO.server/var/master
Initializing /home/omero/OMERO.server/var/log
Creating /home/omero/OMERO.server/var/registry
No descriptor given. Using etc/grid/default.xml
Waiting on startup. Use CTRL-C to exit
................
.............
Failed to startup some components after 300 seconds
Calling "stop" on remaining components
Waiting on shutdown. Use CTRL-C to exit

omero@omero:~$>more OMERO.server/var/log/master.err
-! 12/06/17 15:26:57.450 OMERO.Glacier2: warning: unable to contact permissions verifier `BlitzVerifier@BlitzAdapters'
Reference.cpp:1637: Ice::NoEndpointException:
no suitable endpoint available for proxy `BlitzVerifier -t -e 1.1 @ BlitzAdapters'
-! 12/06/17 15:26:57.451 OMERO.Glacier2: warning: unable to contact session manager `BlitzManager@BlitzAdapters'
Reference.cpp:1637: Ice::NoEndpointException:
no suitable endpoint available for proxy `BlitzManager -t -e 1.1 @ BlitzAdapters'
!! 12/06/17 15:27:12.506 error: communicator not destroyed during global destruction.

omero@omero:~$>omero admin diagnostics

================================================================================
OMERO Diagnostics (admin) 5.4.1-ice36-b75
================================================================================

Commands: java -version 1.8.0 (/usr/bin/java)
Commands: python -V 2.7.5 (/usr/bin/python)
Commands: icegridnode --version 3.6.4 (/usr/bin/icegridnode)
Commands: icegridadmin --version 3.6.4 (/usr/bin/icegridadmin)
Commands: psql --version 9.6.5 (/usr/bin/psql)

Server: icegridnode not started

Log dir: /home/omero/OMERO.server/var/log exists
Log files: Blitz-0.log 20.0 KB errors=2 warnings=0
Log files: DropBox.log 1.0 KB errors=2 warnings=1
Log files: FileServer.log 0.0 KB
Log files: Indexer-0.log 15.0 KB errors=1 warnings=0
Log files: MonitorServer.log 0.0 KB
Log files: PixelData-0.log 15.0 KB errors=1 warnings=0
Log files: Processor-0.log 14.0 KB errors=4 warnings=8
Log files: Tables-0.log 16.0 KB errors=4 warnings=8
Log files: TestDropBox.log n/a
Log files: master.err 0.0 KB errors=1 warnings=2
Log files: master.out 0.0 KB
Log files: Total size 0.09 MB


Environment:OMERO_HOME=(unset)
Environment:OMERO_NODE=(unset)
Environment:OMERO_MASTER=(unset)
Environment:OMERO_USERDIR=(unset)
Environment:OMERO_TMPDIR=/tmp
Environment:PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/home/omero/OMERO.server/bin/
Environment:PYTHONPATH=(unset)
Environment:ICE_HOME=(unset)
Environment:LD_LIBRARY_PATH=(unset)
Environment:DYLD_LIBRARY_PATH=(unset)

Cannot list deployed applications.
OMERO data dir:'/OMERO' Exists? True Is writable? True
OMERO temp dir:'/tmp' Exists? True Is writable? True (Size: 32819)

JVM settings: Blitz-${index} -Xmx5027m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index} -Xmx3351m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index} -Xmx5027m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index} -Xmx3351m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions


I follow the procedures to do the upgrade as usual.

Philippe
phm
 
Posts: 185
Joined: Tue Mar 19, 2013 3:39 pm

Re: Omero don't start after upgrade

Postby jmoore » Wed Dec 06, 2017 2:58 pm

Hi Philippe,

can you share your Blitz-0.log with us?

~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Omero don't start after upgrade

Postby phm » Wed Dec 06, 2017 6:58 pm

jmoore wrote:Hi Philippe,

can you share your Blitz-0.log with us?

~Josh

Hi Josh,
I think that the problem come from my database. I did database upgrade before and may be it's not recommended. My database is 5.3.3 .
Philippe
phm
 
Posts: 185
Joined: Tue Mar 19, 2013 3:39 pm

Re: Omero don't start after upgrade

Postby jmoore » Wed Dec 06, 2017 10:19 pm

You'll definitely need to have your database on the correct version. Can you show us the content of the
Code: Select all
dbpatch
table?

You can start with the precheck from https://docs.openmicroscopy.org/omero/5.4.1/sysadmins/server-upgrade.html#upgrade-steps since it starts by determining the current version of the DB.

Cheers,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: Omero don't start after upgrade

Postby phm » Thu Dec 07, 2017 9:05 am

jmoore wrote:You'll definitely need to have your database on the correct version. Can you show us the content of the
Code: Select all
dbpatch
table?

You can start with the precheck from https://docs.openmicroscopy.org/omero/5.4.1/sysadmins/server-upgrade.html#upgrade-steps since it starts by determining the current version of the DB.

Cheers,
~Josh

Hi Josh,

The dbpatch command don't exist for me ??
omero@omero:~/OMERO.server$>dbpatch
-bash: dbpatch : commande introuvable


omero@omero:~/OMERO.server$>psql -h localhost -U db_omero omero_database < sql/psql/OMERO5.4__0/OMERO5.3__1-precheck.sql
Mot de passe pour l'utilisateur db_omero :
BEGIN
CREATE FUNCTION
ERREUR: wrong OMERO database version for this upgrade script
CONTEXTE : fonction PL/pgsql omero_assert_db_version(character varying,integer), ligne 12 à RAISE
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR: la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ROLLBACK


Philippe
phm
 
Posts: 185
Joined: Tue Mar 19, 2013 3:39 pm

Re: Omero don't start after upgrade

Postby mtbc » Thu Dec 07, 2017 10:00 am

Dear Philippe,

The contents of the dbpatch table can be captured in a file:
Code: Select all
psql -h localhost -U db_omero -c 'SELECT * FROM dbpatch' -o /tmp/dbpatch omero_database

If you could zip up your /tmp/dbpatch file and your Blitz-0.log together and upload them to http://qa.openmicroscopy.org.uk/qa/upload/ then maybe we will be able to spot your problem.

Did anything odd happen when you were following the upgrade procedure steps, like unusual or concerning output from any of the commands that we had you type?

Cheers,
Mark
User avatar
mtbc
Team Member
 
Posts: 282
Joined: Tue Oct 23, 2012 10:59 am
Location: Dundee, Scotland

Re: Omero don't start after upgrade

Postby phm » Thu Dec 07, 2017 12:31 pm

mtbc wrote:Dear Philippe,

The contents of the dbpatch table can be captured in a file:
Code: Select all
psql -h localhost -U db_omero -c 'SELECT * FROM dbpatch' -o /tmp/dbpatch omero_database

If you could zip up your /tmp/dbpatch file and your Blitz-0.log together and upload them to http://qa.openmicroscopy.org.uk/qa/upload/ then maybe we will be able to spot your problem.

Did anything odd happen when you were following the upgrade procedure steps, like unusual or concerning output from any of the commands that we had you type?

Cheers,
Mark


Hi Marks,
Here the results of dbpatch on the database version before udpate (I restored the dump before upgrade):
omero@omero:~/OMERO.server$>more /tmp/dbpatch
id | currentpatch | currentversion | permissions | finished | message
| previouspatch | previousversion | external_id
----+--------------+----------------+-------------+----------------------------+----------------------------------------------------
------------------------------------------+---------------+-----------------+-------------
1 | 0 | OMERO5.0 | -52 | 2014-08-26 14:46:26.490616 | Database ready.
| 0 | OMERO5.0 |
2 | 1 | OMERO5.1 | -52 | 2015-04-13 13:49:10.384326 | Database updated.
| 0 | OMERO5.0 |
3 | 1 | OMERO5.1 | -52 | 2015-04-13 13:49:34.089304 | DB check DBEnumCheck: done for Bio-Formats revision
df26d52d34fe79589a79ff013ef0dc8f7a1bc0f7 | 1 | OMERO5.1 |
4 | 1 | OMERO5.1 | -52 | 2015-05-29 15:04:57.394871 | DB check DBEnumCheck: done for Bio-Formats revision
923bf020f921d8bfcef203f407fc94855e5b03f0 | 1 | OMERO5.1 |
5 | 1 | OMERO5.1 | -52 | 2015-05-29 15:04:57.413108 | DB check DBUnicodeUnitsCheck: done
| 1 | OMERO5.1 |
6 | 0 | OMERO5.2 | -52 | 2015-11-04 11:05:08.7575 | Database updated.
| 1 | OMERO5.1 |
7 | 0 | OMERO5.2 | -52 | 2015-11-04 11:05:55.999559 | DB check DBEnumCheck: done for Bio-Formats revision
6d102caae2bf4bd9284e81768b9d6990aefd6eb2 | 0 | OMERO5.2 |
8 | 0 | OMERO5.3 | -52 | 2017-08-17 18:00:24.724944 | Database updated.
| 0 | OMERO5.2 |
9 | 0 | OMERO5.3 | -52 | 2017-08-17 18:01:13.095333 | DB check DBEnumCheck: done for Bio-Formats revision
4c2f2b00a12aa8a3894c54459a4cf02d5b02e161 | 0 | OMERO5.3 |
10 | 0 | OMERO5.3 | -52 | 2017-08-17 18:01:13.758311 | DB check DBMappedEnumCheck: done for OMERO version
OMERO5.3__0 | 0 | OMERO5.3 |
(10 lignes)


And find the Blizt.log zipped in attachment.
Thanks
Philippe
Attachments
Blitz-0.log.gz
(2.36 KiB) Downloaded 164 times
phm
 
Posts: 185
Joined: Tue Mar 19, 2013 3:39 pm

Re: Omero don't start after upgrade

Postby phm » Thu Dec 07, 2017 12:48 pm

I restarted the "old" version of omero (5.3.3) omero start correctly however omeroweb send some errors :
omero@omero:~$>omero web start
Traceback (most recent call last):
File "manage.py", line 56, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/python2.7/site-packages/django/core/management/__init__.py", line 76, in load_command_class
return module.Command()
File "/usr/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 59, in __init__
self.storage.path('')
File "/usr/lib/python2.7/site-packages/django/utils/functional.py", line 213, in inner
self._setup()
File "/usr/lib/python2.7/site-packages/django/contrib/staticfiles/storage.py", line 311, in _setup
self._wrapped = get_storage_class(settings.STATICFILES_STORAGE)()
File "/usr/lib/python2.7/site-packages/django/core/files/storage.py", line 281, in get_storage_class
return import_by_path(import_path or settings.DEFAULT_FILE_STORAGE)
File "/usr/lib/python2.7/site-packages/django/utils/module_loading.py", line 26, in import_by_path
sys.exc_info()[2])
File "/usr/lib/python2.7/site-packages/django/utils/module_loading.py", line 21, in import_by_path
module = import_module(module_path)
File "/usr/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
__import__(name)
django.core.exceptions.ImproperlyConfigured: Error importing module pipeline.storage: "No module named pipeline.storage"
omero@omero:~$>omero admin diagnostics

================================================================================
OMERO Diagnostics 5.3.3-ice36-b63
================================================================================

WARNING:omero.util.UpgradeCheck:UPGRADE AVAILABLE:Please upgrade to 5.4.0. See http://downloads.openmicroscopy.org/latest/omero for the latest version.

Commands: java -version 1.8.0 (/usr/bin/java)
Commands: python -V 2.7.5 (/usr/bin/python)
Commands: icegridnode --version 3.6.4 (/usr/bin/icegridnode)
Commands: icegridadmin --version 3.6.4 (/usr/bin/icegridadmin)
Commands: psql --version 9.6.5 (/usr/bin/psql)

Server: icegridnode running
Server: Blitz-0 active (pid = 19577, enabled)
Server: DropBox active (pid = 19601, enabled)
Server: FileServer active (pid = 19609, enabled)
Server: Indexer-0 active (pid = 19630, enabled)
Server: MonitorServer active (pid = 19610, enabled)
Server: OMERO.Glacier2 active (pid = 19634, enabled)
Server: OMERO.IceStorm active (pid = 19612, enabled)
Server: PixelData-0 active (pid = 19611, enabled)
Server: Processor-0 active (pid = 19614, enabled)
Server: Tables-0 active (pid = 19625, enabled)
Server: TestDropBox inactive (enabled)

Log dir: /home/omero/OMERO.server/var/log exists
Log files: Blitz-0.log 193.0 MB errors=33 warnings=333
Log files: Blitz-0.log.1 524.0 MB errors=104 warnings=1907
Log files: DropBox.log 30.0 KB errors=20 warnings=15
Log files: FileServer.log 5.0 KB
Log files: Indexer-0.log 7.0 MB errors=832 warnings=286
Log files: MonitorServer.log 11.0 KB
Log files: OMEROweb.lock 0.0 KB
Log files: OMEROweb.log 0.0 KB
Log files: PixelData-0.log 3.0 MB errors=457 warnings=171
Log files: Processor-0.log 2.0 MB errors=0 warnings=1
Log files: Processor-0.log.1 4.0 MB errors=38 warnings=85
Log files: Tables-0.log 128.0 KB errors=29 warnings=64
Log files: TestDropBox.log n/a
Log files: master.err 5.0 KB errors=6 warnings=5
Log files: master.out 0.0 KB
Log files: Total size 736.15 MB


Environment:OMERO_HOME=(unset)
Environment:OMERO_NODE=(unset)
Environment:OMERO_MASTER=(unset)
Environment:OMERO_USERDIR=(unset)
Environment:OMERO_TMPDIR=/tmp
Environment:PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/home/omero/OMERO.server/bin/
Environment:PYTHONPATH=(unset)
Environment:ICE_HOME=(unset)
Environment:LD_LIBRARY_PATH=(unset)
Environment:DYLD_LIBRARY_PATH=(unset)

OMERO SSL port:4064
OMERO TCP port:4063
OMERO data dir:'/OMERO' Exists? True Is writable? True
OMERO temp dir:'/tmp' Exists? True Is writable? True (Size: 134059)

JVM settings: Blitz-${index} -Xmx5027m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Indexer-${index} -Xmx3351m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: PixelData-${index} -Xmx5027m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions
JVM settings: Repository-${index} -Xmx3351m -XX:MaxPermSize=1g -XX:+IgnoreUnrecognizedVMOptions

OMERO.web status... [NOT STARTED]
Django version: 1.6.11.6


And what's about Omero.py when we do the omero upgrade ???? Should we do Omero.py upgrade to ???
omero@omero:~$>ls -l
total 152952
drwx------. 4 omero omero 4096 23 août 17:54 omero
drwxrwxr-x 9 omero omero 4096 21 sept. 17:36 Omero_Backup
-rwxrw-r-- 1 omero omero 745 6 oct. 09:16 omero_clean_and_backup.sh
lrwxrwxrwx 1 omero omero 24 18 août 14:21 OMERO.py -> OMERO.py-5.3.3-ice36-b63
drwxr-xr-x 8 omero omero 4096 18 août 14:29 OMERO.py-5.3.3-ice36-b63
lrwxrwxrwx. 1 omero omero 29 17 août 15:56 OMERO.server -> OMERO.server-5.3.3-ice36-b63/
drwxr-xr-x. 9 omero omero 4096 17 août 15:58 OMERO.server-5.3.3-ice36-b63
drwxr-xr-x 9 omero omero 4096 22 nov. 20:41 OMERO.server-5.4.1-ice36-b75
-rw-rw-r-- 1 omero omero 156590179 22 nov. 21:42 OMERO.server-5.4.1-ice36-b75.zip
lrwxrwxrwx 1 omero omero 28 6 déc. 15:16 OMERO.server-new -> OMERO.server-5.4.1-ice36-b75
drwxr-xr-x 5 root root 4096 18 août 12:56 omerowebvenv


Chers,
Philippe
phm
 
Posts: 185
Joined: Tue Mar 19, 2013 3:39 pm

Re: Omero don't start after upgrade

Postby kennethgillen » Thu Dec 07, 2017 4:40 pm

I see omerowebvenv in the directory listing, but when you're trying to start OMERO.web, it doesn't look like you've got the virtualenv activated.

If the OMERO.web you're using is in OMERO.py, then yes, it'll need to be upgraded alongside OMERO.server.

Best,

Kenny
kennethgillen
 
Posts: 254
Joined: Mon Nov 05, 2012 3:39 pm

Re: Omero don't start after upgrade

Postby phm » Fri Dec 08, 2017 1:39 pm

Hi,
I'm go back to the version 5.3.3, the omero server and omero web work correctly, that mean that the database is ok for this version. Now if I want to upgrade to version 5.4.1 (omero server and omero.py) I have to check the database as described in the doc " Check ahead for upgrade issues". However I think there is a misunderstanding in the precheck command. Should we use sql script in the current version or the new version. For example if I run the preckeck from the current version there is not script for version 5.4.1 :
Code: Select all
omero@omero:~/OMERO.server$>psql -h localhost -U db_user omero_database < sql/psql/OMERO5.4__0/OMERO5.3__0-precheck.sql
-bash: sql/psql/OMERO5.4__0/OMERO5.3__0-precheck.sql: Aucun fichier ou dossier de ce type
omero@omero:~/OMERO.server$>ls sql/psql/
OMERO3__4    OMERO3A__11  OMERO3A__7   OMERO4.3__0  OMERO5.3DEV__1   OMERO5.3DEV__2  OMERO5.3DEV__8
OMERO3__5    OMERO3A__2   OMERO3A__8   OMERO4.4__0  OMERO5.3DEV__10  OMERO5.3DEV__3  OMERO5.3DEV__9
OMERO3__6    OMERO3A__3   OMERO3A__9   OMERO5.0__0  OMERO5.3DEV__11  OMERO5.3DEV__4
OMERO3__7    OMERO3A__4   OMERO4__0    OMERO5.1__1  OMERO5.3DEV__12  OMERO5.3DEV__5
OMERO3A__1   OMERO3A__5   OMERO4.1__0  OMERO5.2__0  OMERO5.3DEV__13  OMERO5.3DEV__6
OMERO3A__10  OMERO3A__6   OMERO4.2__0  OMERO5.3__0  OMERO5.3DEV__14  OMERO5.3DEV__7



And if I run the script from the folder of version 5.4.1 :

omero@omero:~/OMERO.server-5.4.1-ice36-b75$>psql -h localhost -U db_omero omero_database < sql/psql/OMERO5.4__0/OMERO5.3__0-precheck.sql
Mot de passe pour l'utilisateur db_omero :
BEGIN
CREATE FUNCTION
omero_assert_db_version
-------------------------

(1 ligne)

DROP FUNCTION
CREATE FUNCTION
CREATE FUNCTION
assert_db_server_prerequisites
--------------------------------

(1 ligne)

DROP FUNCTION
DROP FUNCTION
SELECT 484276
CREATE INDEX
CREATE FUNCTION
assert_no_filepath_array_duplicates
-------------------------------------

(1 ligne)

DROP FUNCTION
DROP MATERIALIZED VIEW
status
-----------------------------------------------------------
+
+
+
YOUR DATABASE IS READY FOR UPGRADE TO VERSION OMERO5.4__0+
+
+

(1 ligne)

ROLLBACK


It sound good..
Which is also not the case with :
Code: Select all
omero@omero:~/OMERO.server-5.4.1-ice36-b75$>psql -h localhost -U db_omero omero_database < sql/psql/OMERO5.4__0/OMERO5.3__1-precheck.sql
Mot de passe pour l'utilisateur db_omero :
BEGIN
CREATE FUNCTION
ERREUR:  wrong OMERO database version for this upgrade script
CONTEXTE : fonction PL/pgsql omero_assert_db_version(character varying,integer), ligne 12 à RAISE
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ERREUR:  la transaction est annulée, les commandes sont ignorées jusqu'à la fin du bloc
de la transaction
ROLLBACK


The database is well in UTF8 :
Code: Select all
postgres@omero:~$>psql -l
                                    Liste des bases de données
      Nom       | Propriétaire | Encodage | Collationnement | Type caract. |    Droits d'accès     
----------------+--------------+----------+-----------------+--------------+-----------------------
omero_database | db_omero     | UTF8     | fr_FR.UTF-8     | fr_FR.UTF-8  |
postgres       | postgres     | UTF8     | fr_FR.UTF-8     | fr_FR.UTF-8  |
template0      | postgres     | UTF8     | fr_FR.UTF-8     | fr_FR.UTF-8  | =c/postgres          +
                |              |          |                 |              | postgres=CTc/postgres
template1      | postgres     | UTF8     | fr_FR.UTF-8     | fr_FR.UTF-8  | =c/postgres          +
                |              |          |                 |              | postgres=CTc/postgres
(4 lignes)


Now should I have to upgrade the database ?? Because in the documentation the sentence is not clear :
Warning This section only concerns users upgrading from a 5.3 or earlier server. If upgrading from a server, you do not need to upgrade the database.


Philippe
phm
 
Posts: 185
Joined: Tue Mar 19, 2013 3:39 pm

Next

Return to Installation and Deployment

Who is online

Users browsing this forum: No registered users and 1 guest

cron