OME::Analysis::Engine - OME analysis subsystem
$self->checkInputs($chain,$dataset,$user_inputs);
Verifies that every formal input has zero or one links feeding it. It a formal input has zero links feeding it, verifies that the user input provided for the input (defaulting to null) is valid, according to the input's optional and list specification. Also verifies that each link is well-formed; i.e., that the ``from output'' belongs to the ``from node'', the ``to input'' belongs to the ``to node'', and that the types of the ``from output'' and ``to input'' match.
$any_dataset_dependences = $self->calculateDependences($chain, $user_inputs);
Determines whether each node in the active chain is a global, per-dataset or per-image module. If a module outputs any global attributes (which is only allowed if all of its inputs are global attributes), then the module is global. If a module takes in any dataset inputs, or outputs any dataset outputs, or if any of its immediate predecessors nodes are per-dataset, then it as per-dataset. Otherwise, it is per-image. This notion of dataset-dependency comes in to play later when determine whether or not a module's results can be reused.
my $mex = $self->getPredecessorMEX($chex,$node,$formal_input,$target);
Returns the MEX that should be used to satisfy the given formal input of the given node for the given target. This target should match the dependence of the node. The method first looks for a match in user inputs. Then it checks to see if there is a universal execution for that works; if it doesn't, it then looks for a node execution in the current chain execution that works. If the link is between an image-dependent MEX and a dataset-dependent MEX, then there will be more than one MEX which satisfies (one per image in the dataset). In this case, the method will return an array of MEX's. Otherwise, it will return a single MEX object.
my $nex = $self->getUniversalExecution($node,$target);
Tries to find a universal execution for the specified node and target. The target should match the dependence of the node.
$self->recordUserInputs($chex,$user_inputs);
Writes the user inputs to data-base tables
$self->newJob($chex,$node,$target);
This method is called when a node is to be executed against one of its targets. The method's main function is add a job description describing the exeuciton to the AE's job queue.
Executes a node against one of its targets. The target should match the dependence of the node. If this dependence is global or dataset, this method will be called once per module execution, with either the target being undefined or the dataset, respectively. If the dependence is image, this method will be called per image in the dataset.
This method performs attribute reuse checks, assuming that the ReuseResults flag was set in the call to executeChain().
$nex = $self->getJob($worker_id);
Get's the worker (specified by worker_id) a job.
$self->finishedJob($chex,$nex,$node,$target);
A job processing $nex just finished. This method checks if the nex finished successfully and if so, what are the successor nexs that can be added to the AE's job queue. =cut
| sub finishedJob { | |
| my $self = shift; |
# N.B. usually you can get the node from the nex using
# $nex->analysis_chain_node();
# and the chex from the nex using
# $nex->analysis_chain_execution();
# but if it's a universal execution that's not possible
my $chex = shift;
my $nex = shift;
my $node = shift;
my $target = shift;
my $session = OME::Session->instance();
my $factory = $session->Factory();
my $target_name = "-";
$target_name = $target->name() if (defined $target);
logdbg ("debug", "finishedJob(".$node->module->name().", ". $target_name.")");
my $mex = $nex->module_execution;
$mex->refresh(); # update to referesh the module execution from DB to see any changes
if ($mex->status() eq "FINISHED") {
# if yes --> find succesor jobs and thus figure out
# if the chain execution is complete
my @next_nodes = @{OME::Tasks::ChainManager->getNodeSuccessors($node)};
if (scalar (@next_nodes)) {
foreach my $next_node (@next_nodes) {
if ($next_node->dependence() eq 'G') {
$self->newJob ($chex, $next_node, undef);
} elsif ($next_node->dependence() eq 'D') {
$self->newJob ($chex, $next_node, $chex->dataset());
} elsif ($next_node->dependence() eq 'I' and
$node->dependence() eq 'I') {
$self->newJob ($chex, $next_node, $target);
} elsif ($next_node->dependence() eq 'I' and
$node->dependence() ne 'I') {
# additional_jobs signals that there are more jobs in memory that
# need to be written into the DB
my @imgs = $chex->dataset()->images();
$chex->additional_jobs(1);
for (my $i=1; $i<scalar(@imgs); $i++) {
$self->newJob ($chex, $next_node, $imgs[$i]);
}
$chex->additional_jobs(0);
$self->newJob ($chex, $next_node, $imgs[0]);
} else {
croak ("current node dependence is ".$node->dependence()." and ".
"next node dependence is ".$next_node->dependence());
}
}
} else {
logdbg ("debug", $node->module->name()." is a leaf node ");
# no successor nodes implies we're a leaf node
# are there any remaining CHEX jobs ?
if ( not $chex->additional_jobs() ) {
my @CHEX_jobs = $factory->findObjects ('OME::Analysis::Engine::Job',
{
'NEX.analysis_chain_execution' => $chex,
});
if (scalar @CHEX_jobs == 0 ) {
logdbg ("debug", "Chain (CHEX=".$chex->id().") has finished executing");
# timing info
$chex->refresh(); # makes sure that $chex->timestamp() is valid
my $s = str2time($chex->timestamp());
my $usec = 0;
$chex->total_time(sprintf("%0.1f",tv_interval([$s,$usec])));
# chex status
if ($chex->count_node_executions('module_execution.status' => 'ERROR')) {
$chex->status('ERROR');
} else {
$chex->status('FINISHED');
}
# update the task
$chex->task->refresh();
$chex->task->step();
$chex->task->finish();
$chex->task->message("");
$chex->storeObject();
}
}
}
} else {
logdbg ("debug", "MEX status was not FINISHED");
}
$session->commitTransaction();
return;
}
my $chain_execution = OME::Analysis::Engine->executeChain($chain,$dataset,$user_inputs, $task
[ReuseResults => 1]);
The $user_inputs parameter must be a hash, with formal input ID's for keys and MEX objects for values. Values may also be arrays of MEX objects. The format of $user_inputs is known to be problematic (See Bug 391: http://bugs.openmicroscopy.org.uk/show_bug.cgi?id=391 ) and will be modified in the future, from: { $formal_inputA_id => [ $mex_objectA, $mex_objectB, $mex_objectC ], $formal_inputB_id => $mex_objectD, ... } perhaps to: { $nodeA_id => [ { $formal_inputA_id => [ $mex_objectA, $mex_objectB, $mex_objectC ] }, { $formal_inputB_id => $mex_objectD }, ], $nodeB_id => { $formal_inputA_id => [ $mex_objectE, $mex_objectF, $mex_objectG ], }, ... } Notice the arrays are optional when they would contain only one element.
Tom Macura <tmacura@nih.gov> Ilya Goldberg <igg@nih.gov> Douglas Creager <dcreager@alum.mit.edu>,
Open Microscopy Environment, MIT, NIH