Failed upload of Attachments to a Screen ID

[janosch]

Hi,

We have problems with uploading file attachments to a Screen ID.

The python code generates quantitative csv files with the results from different models. Normal behaviour attaches those to the list of attachments per screen ID. The log snippet below shows the error for myfiletag and myfailedfilename for the remaining files of a batch upload.

I believe those are the error codes: erver-1191, erver-1188, erver-1187?

Any suggestions what is wrong and how we could restore normal functionality?

Best
Jan

Omero Error Message generating the well level files:
2018-11-02 18:35:16,325 INFO [ ome.security.basic.EventHandler] (erver-1191) Auth: user=152,group=3,event=null(User),sess=194e32e9-e0e7-4ef4-be2f-2e2d37b90cf9
2018-11-02 18:35:16,328 INFO [ org.perf4j.TimingLogger] (erver-1191) start[1541198116324] time[3] tag[omero.call.success.ome.logic.QueryImpl.findByQuery]
2018-11-02 18:35:16,328 INFO [ ome.services.util.ServiceHandler] (erver-1191) Rslt: ome.model.annotations.FileAnnotation:Id_287781
2018-11-02 18:35:16,337 INFO [ ome.services.util.ServiceHandler] (erver-1188) Meth: interface ome.api.IQuery.findByQuery
2018-11-02 18:35:16,337 INFO [ ome.services.util.ServiceHandler] (erver-1188) Args: [select obj from FileAnnotation obj join fetch obj.details.owner as owner join fetch obj.details.crea, PARAMS:ns=myfiletag name=myfailedfilename ]
2018-11-02 18:35:16,338 INFO [ ome.security.basic.EventHandler] (erver-1188) Auth: user=152,group=3,event=null(User),sess=194e32e9-e0e7-4ef4-be2f-2e2d37b90cf9
2018-11-02 18:35:16,341 INFO [ org.perf4j.TimingLogger] (erver-1188) start[1541198116337] time[3] tag[omero.call.success.ome.logic.QueryImpl.findByQuery]
2018-11-02 18:35:16,341 INFO [ ome.services.util.ServiceHandler] (erver-1188) Rslt: ome.model.annotations.FileAnnotation:Id_273482
2018-11-02 18:35:16,344 INFO [ omero.cmd.SessionI] (erver-1187) Added servant to adapter: 194e32e9-e0e7-4ef4-be2f-2e2d37b90cf9/IHandle94a8269d-336f-4b35-858f-04861a63f453(omero.cmd._HandleTie@2ed61d6e)
2018-11-02 18:35:16,346 INFO [ ome.services.util.ServiceHandler] (2-thread-4) Executor.doWork -- omero.cmd.HandleI.run (rw)[194e32e9-e0e7-4ef4-be2f-2e2d37b90cf9/IHandle94a8269d-336f-4b35-858f-04861a63f453, omero.cmd.graphs.Delete2I@39068a51]
2018-11-02 18:35:16,346 INFO [ ome.services.util.ServiceHandler] (2-thread-4) Args: [null, InternalSF@580515543]
2018-11-02 18:35:16,346 INFO [ ome.security.basic.BasicEventContext] (2-thread-4) cctx: group=-1
2018-11-02 18:35:16,346 INFO [ omero.cmd.graphs.Delete2I.@39068a51] (erver-1191) Add callback: NbDk\\U3TjX11d3Jo#hdk/fc272885-a687-4f13-b1a6-6e850ecf9e6f
2018-11-02 18:35:16,347 INFO [ omero.cmd.graphs.Delete2I.@39068a51] (erver-1188) getResponse: null
2018-11-02 18:35:16,356 INFO [ ome.security.basic.EventHandler] (2-thread-4) Auth: user=152,group=-1,event=12436847(User),sess=194e32e9-e0e7-4ef4-be2f-2e2d37b90cf9
2018-11-02 18:35:16,411 INFO [ org.perf4j.TimingLogger] (2-thread-4) start[1541198116356] time[55] tag[omero.request.step.0]
2018-11-02 18:35:16,411 INFO [ omero.cmd.graphs.Delete2I.@39068a51] (2-thread-4) notify step 1 of 6

[mtbc]

Dear Jan,

I'm not sure that log snippet is related to your issue: nothing looks awry in it from what I can see. I think those error codes might actually be a suffix of "Server" with some kind of thread or process ID or somesuch. Could you upload a more complete log? Just zip them up and submit to http://qa.openmicroscopy.org.uk/qa/upload/ if you like, especially if you can tell us when one of the failed uploads happened. It might also help if you can share with us enough of the Python script or some minimal example with the same issue for us to see exactly how the upload is effected client-side, maybe even so we can try to reproduce the fault at our end.

Cheers,
Mark

[jawi]

Hi Mark,

Thank you for your prompt response and apologies for my slow reply.

We have changed our backend substantially and I will get back to you if this issue still occurs.

Do you have cloud related optimisation suggestions and optimisation settings for the Omero backend (number of CPU's, Memory, splitting server/database using a dedicated DB server (e.g AWS DB instance types)? I can only find six forum entries searching for 'Cloud'. Those are all related to configuration settings.


Best
Jan

[manics]

Hi Jan

We don't really have any cloud specific recommendations, in general the CPU/memory recommendations are the same regardless of the underlying infrastructure, and depend on your expected OMERO usage (e.g. types and sizes of images, number of concurrent users, usage patterns, etc). Take a look at https://docs.openmicroscopy.org/omero/5 ... ments.html

An obvious benefit of the cloud is it's much easier to partition services into separate servers, and this is something we take advantage of for the IDR which runs on a private customised OpenStack cloud at the EBI. This uses:

• database: 8 vCPU 32GB RAM
• OMERO read-write: 16 vCPU 64GB RAM
• OMERO read-only: 8 vCPU 32GB RAM
• Nginx caching-proxy: 8 vCPU 32GB RAM

Unless you're running a very large server you can get away with a lot less, and you may decide that the complexity of managing multiple servers doesn't offer any worthwhile performance benefits.

We don't have any experience with AWS flavours (though others in the community have). I think more information on optimising for AWS is something we'd all find useful, especially with regards to which flavours are best for running PostgreSQL vs OMERO.server, so if you find a good configuration we'd love to hear from you.