next up previous
Next: Classification of OWL EL Up: Classification Previous: Classification Beginning: ORE 2012 Evaluation

Classification of OWL DL ontologies

Table 1 [*] depicts the qualitative results for classification of OWL DL ontologies, hence it only includes the outcomes for FaCT++ and HermiT. The NO-REF evaluation outcomes are due to the lack of reference classifications for these cases. FaCT++ demonstrated several instance parsing and datatype related errors. HermiT errors can be explained taking into account memory limit and parsing settings of OWL API 3.2. For example, classification of fma-lite ontology took more than 1 hour, which is the maximum time allocated for a single task. Furthermore, gazetteer ontology, which contains 150,979 classes, causes a out-of-memory exception with both reasoners and 1Gb of allocated memory.




Table 1: Qualitative results for the classification of OWL DL ontologies
  FaCT++ HermiT
CORRECT 44 48
INCORRECT 1 1
NO-REF 49 50
EXCEPTION 6 2
TIMEOUT 2 1

We detected different types of ontologies within the 102 test cases. For example, 76 ontologies were classified in less than 10,000 ms for both reasoners, while 87 ontologies were classified in less than 40,000 ms. Table 2 [*] shows the average loading and reasoning times for the second set of ontologies. However, we also detected a group of ontologies that were relatively hard for one or both reasoners (see Table 3 [*]).




Table 2: Loading and classification times of OWL DL ontologies
  FaCT++ HermiT
# tests 87 87
ALT, ms 368 363
ART, ms 3,369 1,837




Table 3: Reasoning times for hard cases
Ontologies FaCT++ (ms) HermiT (ms)
not-galen.owl 62,590 3,751
molecule-role 370,370 5,942
emap 120,187 7,243
teleost-taxonomy 1,392,537 14,259
obi 2,198 339,526
DLPOnts-Plans 658 621,033


next up previous
Next: Classification of OWL EL Up: Classification Previous: Classification