This track consists of finding alignments between the Foundational Model of Anatomy (FMA), SNOMED CT, and the National Cancer Institute Thesaurus (NCI). These ontologies are semantically rich and contain tens of thousands of classes.
UMLS Metathesaurus has been selected as the basis for the track reference alignments. UMLS is currently the most comprehensive effort for integrating independently-developed medical thesauri and ontologies, including FMA, SNOMED CT, and NCI.
The generated system alignment should be an optimal solution to the matching problem with respect to both recall and precision with respect to the 2017.5 reference alignment.
The complete datasets for the OAEI 2017.5 campaign can be downloaded as a zip file (LargeBioMed_dataset_oaei2017.zip [17Mb]) or accessed via the HOBBIT platform.
Since 2014, instead of repairing the original UMLS reference alignments by removing mappings leading to unsatisfiable classes, we flagged the incoherence-causing mappings in the alignment by setting the relation to "?" (unknown). These "?" mappings will neither be considered as positive nor as negative when evaluating the participating ontology matching systems, but will simply be ignored. This way, systems that do not perform mapping repair are not penalized for finding mappings that (despite causing incoherences) may or may not be correct, and systems that do perform mapping repair are not penalized for removing such mappings either.
Nevertheless, we will still give a special attention to the number of unsatisfiabilities caused by the mappings computed by a participating system. Thus, we encourage system developers to implement mapping repair techniques or reuse state-of-the art techniques.
Regarding the use of background knowledge, the OAEI rules state that a resource (i.e. a third biomedical ontology) especially designed for the test is not allowed. Particularly, matching systems using the UMLS Metathesaurus as background knowledge will have a notable advantage since the reference alignment is also based on the UMLS Metathesaurus.
The evaluation of this track will be run with support of the HOBBIT platform. This requires that you wrap your matching system in a way that allows us to execute it using the HOBBIT intrastructure (see OAEI 2017.5 evaluation details).
A description of how the HOBBIT largebio benchmark was prepared is available here.
A system participating in the largebio benchmark should explicitly mention that implements its API: bench:LargebioAPI. In practical terms the system adapter should interpret the information that is sent from the benchmark and communicate back the results (a file containing the alignment in RDF Alignment format). See the implementation of the LogMap system and its metadata description (system.ttl).