We're Hiring!

incorrect importing of Zeiss SmartSEM created TIFF

Historical discussions about the Bio-Formats library. Please look for and ask new questions at https://forum.image.sc/tags/bio-formats
Please note:
Historical discussions about the Bio-Formats library. Please look for and ask new questions at https://forum.image.sc/tags/bio-formats

If you are having trouble with image files, there is information about reporting bugs in the Bio-Formats documentation. Please send us the data and let us know what version of Bio-Formats you are using. For issues with your code, please provide a link to a public repository, ideally GitHub.

incorrect importing of Zeiss SmartSEM created TIFF

Postby dmankus » Thu Jun 28, 2018 12:57 pm

Hello,
I've found a bug in the bio-formats plugins for ImageJ. When importing a Zeiss TIFF acquired with SmartSEM software, it uses the wrong metadata to set pixel size. Zeiss saves two metadata fields, one is "Pixel Size" and the other is "Image Pixel Size." The two will be the same if the image acquired is 1024 pixels in width. When other acquisition settings are chosen, "Pixel Size" remains the same, but "Image Pixel Size" updates to the true value.

I've uploaded 3 images to demonstrate the effect here: https://keybase.pub/dmankus/

Has this previously been reported? Is there a fix or a workaround?
I'm using up-to-date as of writing FIJI, running Bio-Formats Plugins for ImageJ Release 5.90-SNAPSHOT Build Date: 27 June 2018
dmankus
 
Posts: 6
Joined: Mon Jun 25, 2018 2:56 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby sbesson » Fri Jun 29, 2018 12:22 pm

Hi,

many thanks for reporting this issue and providing some sample data allowing to reproduce it. I can confirm that the physical size populated by Bio-Formats 5.8.2 is the same across the three different images although it should be different.

I have captured this issue as well as some initial technical details from my investigation in https://trello.com/c/V4SzWlb7/254-leore ... ze-parsing. Briefly, the LEOReader which is used for reading these types of formats makes minimal and partly incorrect use of the TIFF tag containing all the metadata.

Without any change, the only workaround would be to write some code client-side to re-parse the tag and extract the correct Image Pixel Size value. The next step would be to update the reader parsing logic to read and populate the correct metadata. In terms of commitment, our current priority is to finalize our next minor release of Bio-Formats 5.9.0 and we will have no immediate capacity. If anyone has the time to help with this fix, they should definitely free to take ownership and we will be happy to give guidance and feedback.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby dmankus » Fri Jun 29, 2018 1:39 pm

Sebastien,
Thanks for looking into and confirming the issue for me, I'd be happy to contribute in any way I can to fixing the issue, though I'm wholly unfamiliar with the Bio-Formats codebase. Some advice on getting started would be welcome.
Looking at the LEOReader code, it appears that it's attempting to populate the EHT, Filament, Working Distance, Physical Pixel Size, and Acquisition Date fields. In theory this seems like a reasonable subset of tags to begin with, at the least.
In practice, the imported image only has Working Distance and Physical Pixel Size populated. And Working distance is listed with units of µm, but the value is in meters.
As far as I understand what LEOReader is doing, I'm not sure why it's completely missing the other tags, as it looks like it ought to read those values fine.
-David
dmankus
 
Posts: 6
Joined: Mon Jun 25, 2018 2:56 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby dmankus » Fri Jun 29, 2018 3:53 pm

Not really sure how to compile and test it locally, and this is a pretty hacked together attempt, but if we replace LEOReader Lines 105-109 with the following, the previously mentioned values should populate. I'm not sure if setting all physical sizes to micrometers is correct, but it's straightforward to change this to other units, if appropriate.
Code: Select all
    for (int row=36; row<lines.length; row++) {
     
      if (lines[row].equals("AP_IMAGE_PIXEL_SIZE")) {
// pixel size is stored in nm, converted now to micrometers
       double xSize = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" nm","e+03"));
      }
      else if (lines[line].equals("AP_WD")) {
//working distance stored in mm, converting now to micrometers
       double workingDistance = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" mm","e-03"));
      }
      else if (lines[line].equals("AP_ACTUALCURRENT")) {
        double filament = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" A",""));
      }
      else if (lines[line].equals("AP_ACTUALKV")) {
        double eht = Double.parseDouble(lines[row+ 1].split("\\s+=\\s+")[1].replace(" kV","e+03").replace(" V",""));
      }
    }



note: 94-101 should probably be removed, or someone familiar with the expected time/date format can parse the metadata lines:
Code: Select all
AP_TIME
Time :15:47:48
AP_DATE
Date :26 Jun 2018
dmankus
 
Posts: 6
Joined: Mon Jun 25, 2018 2:56 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby dmankus » Fri Jun 29, 2018 4:29 pm

The date issue wasn't as confusing as I thought, since DateTools is later called to fix formatting.

I have forked it on GitHub with the suggested edits in place: https://github.com/openmicroscopy/biofo ... 1?expand=1

I didn't test it, though, so I'm refraining from submitting a pull request. Perhaps someone else can check my work for me. Otherwise I'll have to go through installing all the tools when I have a chance in a few days.
dmankus
 
Posts: 6
Joined: Mon Jun 25, 2018 2:56 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby sbesson » Mon Jul 02, 2018 12:39 pm

Hi David,

thanks for driving this. Having looked briefly at the changes, your patch feels like a good start.

You can find more instructions about building Bio-Formats at https://docs.openmicroscopy.org/bio-for ... index.html if you want to test your changes before submitted.

To get some feedback from the OME team, opening a Pull Request against the Bio-Formats repository is certainly your best option. Initially, we can certainly provide a code review before including your contribution in the daily builds testing Bio-Formats changes against our curated datasets.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby dmankus » Mon Jul 02, 2018 4:58 pm

Did what I could, and while my testing shows the code seems to parse the text, I'm missing something and the result isn't understood down the line. I put in a pull request in the hopes someone can point out my errors.
dmankus
 
Posts: 6
Joined: Mon Jun 25, 2018 2:56 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby dgault » Wed Jul 04, 2018 10:58 am

Thank you for taking the time to open the PR, adding a link for anyone following this thread: https://github.com/openmicroscopy/bioformats/pull/3178
User avatar
dgault
Team Member
 
Posts: 208
Joined: Fri Aug 14, 2015 2:56 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby sbesson » Mon Jul 09, 2018 10:57 am

Hi David,

thank you very much for the contribution to the Bio-Formats code base. We are currently reviewing the changes and looking into integrating them in an upcoming Bio-Formats release.

The sample files you uploaded to us will also be added to our curated QA repository for the testing of future Bio-Formats releases. To facilitate community contributions and the maintenance of this library, would it be possible to share those files publicly under a CC-BY license and make them available at https://downloads.openmicroscopy.org/images/? If so, we can provide a first location where you can review the content and licensing. If this is not possible, we will add these sample files to our curated repository but keep them private.

Best,
Sebastien
User avatar
sbesson
Team Member
 
Posts: 421
Joined: Tue Feb 28, 2012 7:20 pm

Re: incorrect importing of Zeiss SmartSEM created TIFF

Postby dmankus » Mon Jul 09, 2018 6:43 pm

Absolutely, I acquired the images specifically to troubleshoot this issue, so I'd be happy to have them used as part of the repository. Let me know what I'll need to do to release them officially, and if you need them re-uploaded in some capacity.
dmankus
 
Posts: 6
Joined: Mon Jun 25, 2018 2:56 pm


Return to User Discussion [Legacy]

Who is online

Users browsing this forum: No registered users and 0 guests