Ontology Alignment Evaluation Initiative - OAEI-2015 Campaign

Results OAEI 2015::Large BioMed Track

Results of the SNOMED-NCI matching problem

The following tables summarize the results for the tasks in the SNOMED-NCI matching problem.

AML provided the best results in terms of both Recall and F-measure in Task 5 and 6, while RSDLWB and XMAP provided the best results in terms of precision in Task 5 and 6, respectively.

Unlike in the FMA-NCI and FMA-SNOMED mathcing problems, the use of the UML-Metathesaurus did not have an impact in the performance of XMAP-BK, which obtained almost identical results as XMAP.

As in the previous matching problems, efficiency decreases as the ontology size increases. Furthermore, LiLy, DKP-AOM-Lite, DKP-AOM, ServOMBI and CroMatcher could not complete neither Task 5 nor Task 6 in less than 12 hours.

* Uses background knowledge based on the UMLS-Metathesaurus as the LargeBio reference alignments.


Task 5: SNOMED-NCI small fragments

System Time (s) # Mappings Scores Incoherence Analysis
Precision  Recall  F-measure Unsat. Degree
AML 470 14,141 0.917 0.724 0.809 ≥0 ≥0.000%
LogMapBio 3,298 12,855 0.940 0.674 0.785 ≥0 ≥0.000%
LogMap 410 12,384 0.958 0.663 0.783 ≥0 ≥0.000%
XMAP-BK * 396 11,674 0.928 0.606 0.733 ≥1 ≥0.001%
XMAP 394 11,674 0.928 0.606 0.733 ≥1 ≥0.001%
LogMapLite 212 10,942 0.949 0.567 0.710 ≥60,450 ≥80.4%
Average 1,055 11,092 0.938 0.577 0.703 12,262 16.3%
LogMapC 3,039 9,975 0.914 0.510 0.655 ≥0 ≥0.000%
RSDLWB 221 5,096 0.967 0.267 0.418 ≥37,647 ≥50.0%
Table 1: Results for the largebio task 5.


Task 6: NCI whole ontology with SNOMED large fragment

System Time (s) # Mappings Scores Incoherence Analysis
Precision  Recall  F-measure Unsat. Degree
AML 584 12,821 0.904 0.650 0.756 ≥2 ≥0.001%
LogMapBio 3,327 12,745 0.853 0.609 0.711 ≥4 ≥0.002%
LogMap 1,062 12,222 0.870 0.596 0.708 ≥4 ≥0.002%
XMAP-BK * 925 10,454 0.913 0.536 0.675 ≥0 ≥0.000%
XMAP 905 10,454 0.913 0.535 0.675 ≥0 ≥0.000%
LogMapLite 427 12,894 0.797 0.567 0.663 ≥150,656 ≥79.5%
Average 1,402 10,764 0.878 0.526 0.649 29,971 15.8%
LogMapC 3,553 9,100 0.882 0.450 0.596 ≥2 ≥0.001%
RSDLWB 436 5,427 0.894 0.265 0.408 ≥89,106 ≥47.0%
Table 2: Results for the largebio task 6.