Ontology Alignment Evaluation Initiative - OAEI-2014 Campaign

Large BioMed Track

Results OAEI 2014::Large BioMed Track


If you have any question/suggestion related to the results of this track or if you notice any kind of error (wrong numbers, incorrect information on a matching system, etc.), feel free to write an email to ernesto [at] cs [.] ox [.] ac [.] uk or ernesto [.] jimenez [.] ruiz [at] gmail [.] com

Evaluation setting

We have run the evaluation in a Ubuntu Laptop with an Intel Core i7-4600U CPU @ 2.10GHz x 4 and allocating 15Gb of RAM.

Precision, Recall and F-measure have been computed with respect to a UMLS-based reference alignment. Systems have been ordered in terms of F-measure.

Participation and success

In the OAEI 2014 largebio track 11 out of 14 participating OAEI 2014 systems have been able to cope with at least one of the tasks of the largebio track.

RiMOM-IM, InsMT and InsMTL are systems focusing in the instance matching track and they did not produce any alignment for the largebio track.

Use of background knowledge

LogMap-Bio uses BioPortal as mediating ontology provider, that is, it retrieves from BoipPortal the most suitable top-5 ontologies for the matching task.

Alignment coherence

Together with Precision, Recall, F-measure and Runtimes we have also evaluated the coherence of alignments. We have reported (1) number of unsatisfiabilities when reasoning with the input ontologies together with the computed mappings, and (2) the ratio/degree of unsatisfiable classes with respect to the size of the union of the input ontologies.

We have used the OWL 2 reasoner HermiT to compute the number of unsatisfiable classes. For the cases in which HermiT could not cope with the input ontologies and the mappings (in less than 2 hours) we have provided a lower bound on the number of unsatisfiable classes (indicated by ≥) using the OWL 2 EL reasoner ELK.

In this OAEI edition, only two systems have shown mapping repair facilities, namely: AML and LogMap (including LogMap-Bio and LogMap-C variants). The results show that even the most precise alignment sets may lead to a huge amount of unsatisfiable classes. This proves the importance of using techniques to assess the coherence of the generated alignments.

Results 2014

1. System runtimes and task completion

2. Results for the FMA-NCI matching problem

3. Results for the FMA-SNOMED matching problem

4. Results for the SNOMED-NCI matching problem

5. Summary results for the top systems

6. Harmonization of the mapping outputs

7. Mapping repair evaluation