2018-02-06

Dundee: Will, Kenny, Petr, Dominik, June, Balaji, Jean-Marie, Helen, Simon, Jason,

Remote: Mark, Sebastien, Wilma, Chris, David, Emil, Josh, Melissa, Roger, Liza

Start: 2:00 pm

1. Accepting minutes from last meeting

1. Project Status

(2-3 minutes each)

• West (Workflows)

  • Iviewer: moving towards 1.0

    • ROI saving speed last major issue

  • Pyramid issue. 5.4.4 in coming weeks

    • Since OMERO 5.3, generated pyramids corrupted.

  • Django version

    • 1.08 not supported in April

    • 1.11 will need new web apps

  • Helen & Liza working on OMERO plugins section for website

    • Figure and iviewer initially

  • Stand-up prep: list robot CI job

    • In ci-master now

• East (infra/IDR 0.5.0)

  • Read-only

    • Mark: Made progress with startup beans, now working on

      providers, taking Glencoe’s in-memory implementations as a basis. However, mostly on leave over this fortnight.

    • Branch on Mark’s fork

  • SubIFDs

    • See

      https://github.com/openmicroscopy/design/pull/83

    • Discuss at formats meeting tomorrow - please read before

      meeting

    • Align proposals using same style (PEP, JCP, etc)

    • Meetings running into Glencoe meeting

  • Formats builds

    • Shift times. LSC runs backups 12-2.00.

  • MS

    • Doodle poll for presentation

  • Ansible upgrade

• Glencoe

  • Pathviewer uses different approach to saving ROIs etc which may

    help with iviewer performance.

  • Thumbnail & rendering - discussion with JM

  • Start updating some customers to 5.4

  • Exploring sync functionality of Ice for microservices

• Community

  • IDR 0.4.5

    • Close to going live

    • Notebooks cleanup to fix crashes

    • Load balancing etc pushed to 0.4.6

    • Testing spreadsheet, for Thursday

  • Users meeting

    • 30th May - 1st June

    • Proposal increase in registration fee?

    • Chris: artificial separation. Unique price? June: also

      simplifies the registration process

    • Suggestions for alternative to wine tasting?

    • Any changes suggested for changes to program from last year

    • Helen: Call for Help - re: NEUBIAS

  • Jason: feedback from USA travels. - see standup

1. AOB

(5 min. max; tech. Discussion should be highlighted to relevant people and rescheduled)

1. Kubernetes presentation by Simon

(20-25 minutes plus 15 minutes questions max)

• Container orchestration

  • What, why, any bad?

• Kubernetes - open souce container orchestration etc.

• Containers - lightweight form of virtualisation

  • Methodology for packaging and shipping

  • Docker: platform and tools for building and running apps

• OMERO in Docker - e.g 4 containers

  • Postres, Server, Web, Nginx

• Manually: $ docker run (x 4)

• Easier: docker-compose

• OMERO at scale: Redis, Multiple OMERO.web, Web-apps, OMERO.scripts,

  JupyterHub etc.

• Solutions:

  • Run containers on different hosts

  • Automatically schedule containers across different hosts.

  • Tools: Kubernetes, Docker Swarm, Apache Mesos,

• Kubernetes is great because:

  • Schedule applications across multiple servers based on

    constraints

  • Infrastructure as code - version controlled

  • Automatically restart failed/slow applications - e.g. Health

    check ping of OMERO.web

  • Automatically move applications off a failed server - e.g.

    power-cut

  • Provide a service directory. (instead of manual docs for jobs

    etc)

• Microservices - 10s - 100s of copies

• Kubernetes model:

  • Pods - containerised app, e.g OMERO.web

  • Deployments - manages one or more Pods

  • Services - the interface to a deployment. E.g. provides

    load-balancing

  • Service Discovery - DNS provides named-based lookup of services

• Debugging multiple containers:

  • use centralised logging

  • Use Traceable identifiers

• Need shared data store to handle failure of services

• Kubernetes vv Docker swarm. - almost same features, Kubernetes been

  around longer

• Simon - devspace mixes containers - no clean separation of data vv

  stateless containers

• Josh - devspace is too much for some usages - e.g Just server & data

• Chris - running Kubernetes is too much investment for most of our

  users

• Jason - flagship projects will need this

• Simon/Balaji - important how we develop the app (regardless of the

  platform)

• Chris - and the problems remain (logging, shared storage, …) even if

  using unix services

• JRS - asking the community if anyone would be interested (Brisbane,

  Allen, …)

• Josh - something more formalised than docker-compose would be nice

  for our users

  • Internally - team needs to understand usage
• Kenny: Helm (kubernetes package manager) - Helm charts are like

  package file

  • E.g. running a Helm chart on amazon is very easy.

  • Currently setting up Kubernetes cluster “lochy”

  • First app is Redmine

  • JM: Training server

• Petr - time to swap DB (the data) has been the problem (spinning up

  the devspace is easy)