I have now gained access to the apache log files, and increased the max user process count.
Most of the apache errors appear to be releated to when I start/stop omero and there is an attempt to access a page.
The following error is frequent, but only partially aligned with when OMERO.web viewer stops working.
- Code: Select all
[Fri Nov 23 07:15:03 2012] [error] [client 10.7.50.30] FastCGI: comm with server "/var/omero/dist/var/omero.fcgi" aborted: idle timeout (30 sec)
[Fri Nov 23 07:15:03 2012] [error] [client 10.7.50.30] FastCGI: incomplete headers (0 bytes) received from server "/var/omero/dist/var/omero.fcgi"
A pattern is starting to emerge that the errors starts after 30-32 hours. This is regardless of the number of pae hits, I used ab to do 10000 requests which works fine.
An obvious short term fix is to have a cron job restarting OMERO every night. I will monitor if the max user process improves things first.
The following is the content from the error page, not sure if it helps as my code is off the main branch, but anyway:
- Code: Select all
Traceback (most recent call last):
File "/var/omero/dist/lib/python/django/core/handlers/base.py", line
111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/var/omero/dist/lib/python/omeroweb/webemdb/views.py", line 1148,
in sliceviewer
conn = getConnection(request)
File "/var/omero/dist/lib/python/omeroweb/webemdb/views.py", line 1129,
in getConnection
logger.debug('emdb connection: %s server %s' % (conn._sessionUuid,
blitz.host))
AttributeError: 'NoneType' object has no attribute '_sessionUuid'
<WSGIRequest
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{'__utma': '222765775.502751138.1347566739.1352071664.1353232896.5',
'__utmz':
'222765775.1352071664.4.3.utmcsr=wwwdev.ebi.ac.uk|utmccn=(referral)|utmcmd=referral|utmcct=/emdb-srv/test/atlas/2055_mapoverview.html'},
META:{'DOCUMENT_ROOT': '/var/www/html',
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTP_ACCEPT': 'text/html, application/xhtml+xml, */*',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-gb',
'HTTP_CONNECTION': 'Keep-Alive',
'HTTP_COOKIE':
'__utma=222765775.502751138.1347566739.1352071664.1353232896.5;
__utmz=222765775.1352071664.4.3.utmcsr=wwwdev.ebi.ac.uk|utmccn=(referral)|utmcmd=referral|utmcct=/emdb-srv/test/atlas/2055_mapoverview.html',
'HTTP_HOST': 'www.ebi.ac.uk',
'HTTP_REFERER': 'http://www.ebi.ac.uk/emdb-srv/atlas/1906_sliceviewer.html',
'HTTP_USER_AGENT': 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0;
Trident/5.0)',
'HTTP_X_CLUSTER_CLIENT_IP': '86.152.93.194',
'PATH': '/sbin:/usr/sbin:/bin:/usr/bin',
'PATH_INFO': u'/webemdb/1906_sliceviewer/',
'PATH_TRANSLATED': '/var/www/html/webemdb/1906_sliceviewer/',
'QUERY_STRING': '',
'REMOTE_ADDR': '86.152.93.194',
'REMOTE_PORT': '58307',
'REQUEST_METHOD': 'GET',
'REQUEST_URI': '/emdb/omero/webemdb/1906_sliceviewer/',
'SCRIPT_FILENAME': '/var/omero/dist/var/omero.fcgi',
'SCRIPT_NAME': u'/emdb/omero',
'SERVER_ADDR': '10.3.2.129',
'SERVER_ADMIN': 'root@localhost',
'SERVER_NAME': 'www.ebi.ac.uk',
'SERVER_PORT': '80',
'SERVER_PROTOCOL': 'HTTP/1.1',
'SERVER_SIGNATURE': '<address>Apache/2.2.15 (Red Hat) Server at
www.ebi.ac.uk Port 80</address>\n',
'SERVER_SOFTWARE': 'Apache/2.2.15 (Red Hat)',
'ZEUS_LOAD_BALANCER_ADDR': '193.62.197.26',
'wsgi.errors': <flup.server.fcgi_base.TeeOutputStream object at 0x52124d0>,
'wsgi.input': <flup.server.fcgi_base.InputStream object at 0x5212390>,
'wsgi.multiprocess': True,
'wsgi.multithread': False,
'wsgi.run_once': False,
'wsgi.url_scheme': 'http',
'wsgi.version': (1, 0)}>
Cheers,
Ingvar