OAEI 2014::Large BioMed Track

Results OAEI 2014::Large BioMed Track

Results of the SNOMED-NCI matching problem

The following tables summarize the results for the tasks in the SNOMED-NCI matching problem.

AML provided the best results in terms of both Recall and F-measure in Task 5, while OMReasoner provided the best results in terms of precision. Task 6 was completely dominated by AML.

As in the previous matching problems, efficiency decreases as the ontology size increases. Furthermore, AOT, AOTL, RSDLWB could no complete neither Task 5 nor Task 6 in less than 10 hours. MaasMatch rose a "stack overflow" exception in Task 5 and an "out of memory" exception in Task 6, while OMReasoner could not complete Task 6 within the permitted time.

Task 5: SNOMED-NCI small fragments

**Table 1:** Results for the largebio task 5.
System	Time (s)	# Mappings	Scores			Incoherence Analysis
System	Time (s)	# Mappings	Precision	Recall	F-measure	Unsat.	Degree
AML	831	14,131	0.917	0.724	0.809	≥0	≥0.0%
LogMap-Bio	1,379	14,360	0.880	0.709	0.786	≥23	≥0.031%
LogMap	263	14,011	0.889	0.699	0.783	≥23	≥0.031%
XMap	182	14,223	0.849	0.665	0.746	≥65,512	≥87.1%
Average	1,522	12,177	0.911	0.611	0.722	23,078	30.7%
LogMapLite	76	10,962	0.948	0.567	0.710	≥60,426	≥80.3%
LogMap-C	2,723	10,432	0.909	0.531	0.670	≥0	≥0.0%
OMReasoner	5,206	7,120	0.983	0.383	0.551	≥35,568	≥47.3%

Task 6: NCI whole ontology with SNOMED large fragment

**Table 2:** Results for the largebio task 6.
System	Time (s)	# Mappings	Scores			Incoherence Analysis
System	Time (s)	# Mappings	Precision	Recall	F-measure	Unsat.	Degree
AML	497	12,626	0.912	0.645	0.756	≥0	≥0.0%
LogMap-Bio	2,545	12,507	0.852	0.599	0.703	≥37	≥0.020%
LogMap	917	12,167	0.863	0.590	0.701	≥36	≥0.019%
XMap	490	12,525	0.843	0.584	0.690	≥134,622	≥71.1%
Average	1,181	12,024	0.858	0.575	0.687	47,578	25.1%
LogMapLite	89	12,907	0.798	0.567	0.663	≥150,776	≥79.6%
LogMap-C	2,548	9,414	0.880	0.464	0.608	≥1	≥0.001%