2019-05-07

Zoom: Seb, Josh, J-m, Frances, Petr, Dom, Mark, Will, Wilma, Riad, Simon, David, Melissa,

Andreas, Emil, Chris, Liza

Start: 2:00 pm

1. Accepting minutes from last meeting

2. Project Status

(2-3 minutes each)

• Data-In

  • Bio-Formats 6.1 update:

    • Testing stage. All major PRs opened. Ongoing integration.

      Release candidate this week

    • Incl. OMERO integration for testing (see below)

    • J-M: testing pyramid bug fix (blocker for OMERO 5.5)

• Data-Out

  • Insight: all artifacts now available. Dmg built by travis and

    uploaded when tag set, available on github

    • Manual step to build on windows.
  • Docs: ongoing work. Plan for Javadocs for OMERO 5.5. Riad

    looking into slice2html.

  • Josh: merge-ci status (Updated OpsX card)

    • Only 1 location for testing your stuff

      • https://merge-ci.openmicroscopy.org/jenkins/

      • https://merge-ci.openmicroscopy.org/web/

    • 1 Jenkins for testing B-F and OMERO. East and West going

      away. Duplicate jobs on ci-master will be shelved.

    • At the moment, fixing all remaining JAR weirdnesses to get

      things green

    • Only test merge BF + merge OMERO

    • Valuable DB on ci-merge server?

    • Will: various links to eel in PRs and cards.

    • To discuss: what DB to use.

• Glencoe (Chris)

  • First OMERO 5.4.10 with BF 6 starting to ship to customers.

  • BF 6 pyramids being created from stitching tools. Files being

    used by customers

  • Webinar next Tuesday (won’t be in this meeting)

  • Seb: heads up - BF 6.1 might be more tricky wrt dependencies.

  • Chris: BF 6 need was urgent. No need beyond 6.0.1 yet.

• Community

  • Petr and Dominic to Oxford for training tomorrow

1. AOB

(5 min. max; tech. Discussion should be highlighted to relevant people and rescheduled)

1. Main Topic: Pre-summer priorities

(20-25 minutes plus 15 minutes questions max)

• Josh

  • Update on status of IDR

  • Refocusing: IDR 0.6.8 rather than IDR 0.7.0, end of June

    • Gallery ✔ (new user experience)

      • IDR Cells + IDR Tissues

    • Search ! (no major deficiencies)

      • Need to work. Fixes something that appears as broken

      • Follow-up of read-only split on IDR. Might affect any

        big deployment similar to IDR

    • Ingest ! (speed of)

      • Aka import + curation.

      • Adding value. Simple fix: more data into the system. One

        limiting factor is people?

      • Import too slow (can’t be blocking)

        • Petr: reference point?

        • Seb: dealing with very large submissions (e.g.

          100-1K plates) can take up to several weeks

        • Chris: user-based expectation: import no longer than

          data transfer (close to the wall on 1536-well plates. ~20h)

        • Liza: does someone see that estimate? Setting

          expectations. E.g. Email on completion. (Emil: depends on deployment since some don’t even know there is an import. I.e. it should just be there.)

        • Riad: stats on bottlenecks? Josh: largely Postgres

          but needs more work

      • Petr: Do we want more devs working on import, OR more

        curators doing the imports?

      • Simon - can we script more of the IDR import steps?

      • Josh: limit to how many users can import at once without

        killing server. Could make use of scripts.

      • Seb: Frances spends more time on curation than running

        import commands. Fixing readers, formats and guidance for submitters etc.

      • Chris: faster import benefits all of us as well as wider

        community.

    • OMERO 5.5 & Bio-Formats 6.1 ? (*single codebase, new

      submissions)*

  • Other national/institute strategies

• Questions

  • Any thoughts on how to focus development priorities on those

    lines?

  • Are there any blockers in your way?

• Gallery report (Will)

  • Petr: searching for INCENP, are you still in cells or tissues?

    (Missed where that happened). Will: show a reduced number of hits (e.g. under tissues). But the auto-complete list of genes is not filtered. (Only bug)

  • Design to be decided. Liza: just worry about making it work

    well.

  • Configuration to be decided. Some IDR specific like whether

    to split by name etc. for a given key. Simon: structuring of metadata could be improved so it’s less hard-coded. Seb: we can’t be sure how someone else has setup their metadata (like ourselves) so good to have it configurable. Chris: burden on a potentially complex configuration system (becomes code). Simon: becoming the Lucene parser… Will: queries are written like Lucene (in the configuration) but doing the filtering in the page with Javascript. (Works at 60 studies). Simon: remember that mapr configuration is very complex. No one using it.

  • Simon: Tag the few PRs then can deploy on idr-testing (and

    fix the CORS)

  • Chris: The more we can utilize Lucene, the better. (Fix

    things that are broken, e.g. to prevent loading everything into RAM)