We're Hiring!

importing time

General user discussion about using the OMERO platform to its fullest. Please ask new questions at https://forum.image.sc/tags/omero
Please note:
Historical discussions about OMERO. Please look for and ask new questions at https://forum.image.sc/tags/omero

There are workflow guides for various OMERO functions on our help site - http://help.openmicroscopy.org

You should find answers to any basic questions about using the clients there.

importing time

Postby bhcho » Mon Apr 04, 2011 3:54 pm

Hi all,

I'm trying to measure the time for importing.
when I use the OMERO.importer, the status says "prepping", "analyzing", "updating DB", transfer(1/3, 2/3, 3/3), "thumbnailing", and "done".
However, it takes too much time to go to the next image.
That is, it only took about 5 seconds (from "prepping" to "done") for importing an image, but it took another about 10 seconds before it started "prepping" for the next image.
I'm currently using 4.2.1.

Could you tell me about what OMERO is doing at the cooling time?
is this because I'm using 4.2.1? (may be solved from 4.2.2?)

Best,
BK
bhcho
 
Posts: 236
Joined: Mon Apr 05, 2010 2:15 pm

Re: importing time

Postby jmoore » Mon Apr 04, 2011 6:22 pm

Hi BK,

could you attach the importer.log file after one of these incidents? Does it always occur? Only for particular file types? How many files are in your queue? Is it only the transaction from the first file to the second file, or also the transition from the second to the third file?

Cheers,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: importing time

Postby bhcho » Mon Apr 04, 2011 6:42 pm

Hi Josh,

attached is the importer.log file.
Does it always occur?

Yes, it does.

Only for particular file types?

when I tried for ome-tif, tif, and bmp, it always happened.

How many files are in your queue?

from 25 images to 75 images. But it seems there is always the cooling time, even though the cooling time span varies a little bit.

Is it only the transaction from the first file to the second file, or also the transition from the second to the third file?

As I said, it's consistent along the entire queue.

BK
Attachments
importer.tgz
(113.48 KiB) Downloaded 200 times
bhcho
 
Posts: 236
Joined: Mon Apr 05, 2010 2:15 pm

Re: importing time

Postby jmoore » Mon Apr 04, 2011 9:15 pm

Hi BK,

could you possibly attach your var/log/Tables-0.log file as well?

Thanks,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: importing time

Postby bhcho » Mon Apr 04, 2011 9:54 pm

Attached.

BK
Attachments
Tables-0.tgz
(172.01 KiB) Downloaded 194 times
bhcho
 
Posts: 236
Joined: Mon Apr 05, 2010 2:15 pm

Re: importing time

Postby jmoore » Tue Apr 05, 2011 7:18 am

Thanks, BK.
It looks like the "Import History" functionality is causing the delays:
Code: Select all
2011-04-04 16:19:18,867 INFO  [                            omero.remote] (Dummy-11  )  Meth: TableI.update
2011-04-04 16:19:26,182 INFO  [                     omero.tables.TableI] (Dummy-11  ) Updated 5861 row(s) of data to Table-...

That's a 7 second window in which the import information is saved. To test this, could you run the following and then re-try import:
Code: Select all
bin/omero admin ice server disable Tables-0
bin/omero admin ice server stop Tables-0

If that's the case, we'll need to look into getting the fixes described under http://trac.openmicroscopy.org.uk/ome/ticket/4872.

Cheers,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany

Re: importing time

Postby bhcho » Tue Apr 05, 2011 1:40 pm

Thanks Josh,

After I stopped Tables-0, there was no idle time before it moves to the next image.
what does this Tables-0 do? updating import history?
Do I need to restart this service to complete the importing? and how?

Best,
BK
bhcho
 
Posts: 236
Joined: Mon Apr 05, 2010 2:15 pm

Re: importing time

Postby jmoore » Tue Apr 05, 2011 2:19 pm

Hi BK,

Tables-0 is the service which provides the OMERO.tables API, so leaving it turned off is not really an option in your case. The importer is also using the service to store your import history, but unfortunately there's no runtime way to turn this off (other than disabling the service itself).

If you're willing to compile your own client, you can comment out the line:
http://git.openmicroscopy.org/?p=ome.git;a=blob;f=components/tools/OmeroImporter/src/ome/formats/importer/gui/HistoryTableStore.java;h=6e30f16f5af888f3d41214e948b867abe5faa0d9;hb=4_2#l127

and the importer will not use the tables service. Alternatively, you can use the command-line importer for large imports, until we've fixed #4872.

Cheers,
~Josh
User avatar
jmoore
Site Admin
 
Posts: 1591
Joined: Fri May 22, 2009 1:29 pm
Location: Germany


Return to User Discussion

Who is online

Users browsing this forum: No registered users and 1 guest