Hardware structure that match OMERO

[janpeter]

All,

We are about to invest in data-storage infrastructure for our lab and have also recently become interested in the OMERO software and plan to compare it with other solutions. We have seen the system requirement (server) on your homepage and read some of the documentation, but we would also appreciate recommended sections to read and answer to question below might very well be found at your homepage.

We consider a cost-effective and scalable hardware structure and here we need your input. One idea is to combine a traditional Linux server (with a small hard disc) and an external scalable NAS (with large storage capacity) with high communication capacity (about 400 Mbyte/s) through iSCSI. Thus we like to have the OMERO-server on the server-computer (A) and then the much larger repository of images on the NAS (B) and perhaps even the SQL-database of (image metadata and experimental facts).

In the introductory description “Introduction OME / System overview” is said that the Image server and the Data server software very well can be on different machines. However we consider to make the cut slightly different.

1. We consider to run the Server software on A (mentioned above) but have the image repository in B
(high communication capacity using iSCSI). Is this a good idea?
2. With some effort we may be able to install Server software in the NAS B. It runs Linux_ 2.6 Debian.
The NAS B will however have less CPU capacity than A. Is this preferable to the idea above?
3. Does the “Data server software” communicate directly with the files in the “image server software”
or through “the image server software”?

Another important question for us is that we plan to use OMERO in combination with other software. Right now we think of mainly using OMERO image handling and browsing of images. Also the combination of image browsing and SQL-database of metadata is attractive. For image analysis we will also use our own software.

4. This other software produce a lot of data-files that we would like store in folders that OMERO keep
track of (i.e. the Data server software). Is this within the scope of OMERO?

Efficient backup is of course central in our evaluation. In your documentation we understand that there is freedom to choose among many backup software (and you mentioned freeware rdiff).

5. Can we use the freeware rsync for backup (preferred by our NAS vendor)

Regards

Jan Peter

[cxallan]

1. We consider to run the Server software on A (mentioned above) but have the image repository in B
(high communication capacity using iSCSI). Is this a good idea?

You'd run all the services on A with an iSCSI volume with the actual data mounted on A.

2. With some effort we may be able to install Server software in the NAS B. It runs Linux_ 2.6 Debian.
The NAS B will however have less CPU capacity than A. Is this preferable to the idea above?

I definitely wouldn't. I'd use your NAS as it was designed and have that storage available to A. There's no reason to run any services on NAS B.

3. Does the “Data server software” communicate directly with the files in the “image server software”
or through “the image server software”?

They're all part of the same common framework with the "data server software" data store being PostgreSQL and the "image server software" data store being the filesystem.

Another important question for us is that we plan to use OMERO in combination with other software. Right now we think of mainly using OMERO image handling and browsing of images. Also the combination of image browsing and SQL-database of metadata is attractive. For image analysis we will also use our own software.

4. This other software produce a lot of data-files that we would like store in folders that OMERO keep
track of (i.e. the Data server software). Is this within the scope of OMERO?

Not at present without "importing" those data files into OMERO. Our work on OMERO.fs as this year progresses and the new year rolls round will provide some of this functionality but it's not present in OMERO Beta 4.2.1.

Efficient backup is of course central in our evaluation. In your documentation we understand that there is freedom to choose among many backup software (and you mentioned freeware rdiff).

5. Can we use the freeware rsync for backup (preferred by our NAS vendor)

I don't see why not.

[janpeter]

Thank you for your answer!

We are testing a NAS now and have the possibility to use either iSCSI connection to the NAS which also implies that the host machine of OMERO take over the file system, or use standard protocols like CIFS or NFS and the NAS own file system (Ext4 under Linux ver 2.6 Debian).

Can someone outline pros and cons with the different possibilities sketched above? We are concerned with both performance as well as having a simple backup solution.

How to make efficient backup of OMERO-server data is not clear to us and we would appreciate some input. One idea is to use the backup solution provided with the NAS and when we use the NAS own file system we can use Rsync, but for iSCSI we do not know yet. This solution would imply separate backups of the image repository and the meta-data database which may create new problems.