Classification of OWL DL ontologies

Table 1 depicts the qualitative results for classification of OWL DL ontologies, hence it only includes the outcomes for FaCT++ and HermiT. The NO-REF evaluation outcomes are due to the lack of reference classifications for these cases. FaCT++ demonstrated several instance parsing and datatype related errors. HermiT errors can be explained taking into account memory limit and parsing settings of OWL API 3.2. For example, classification of fma-lite ontology took more than 1 hour, which is the maximum time allocated for a single task. Furthermore, gazetteer ontology, which contains 150,979 classes, causes a out-of-memory exception with both reasoners and 1Gb of allocated memory.

Table 1: Qualitative results for the classification of OWL DL ontologies

	FaCT++	HermiT
CORRECT	44	48
INCORRECT	1	1
NO-REF	49	50
EXCEPTION	6	2
TIMEOUT	2	1

We detected different types of ontologies within the 102 test cases. For example, 76 ontologies were classified in less than 10,000 ms for both reasoners, while 87 ontologies were classified in less than 40,000 ms. Table 2 shows the average loading and reasoning times for the second set of ontologies. However, we also detected a group of ontologies that were relatively hard for one or both reasoners (see Table 3 ).

Table 2: Loading and classification times of OWL DL ontologies

	FaCT++	HermiT
# tests	87	87
ALT, ms	368	363
ART, ms	3,369	1,837

Table 3: Reasoning times for hard cases

Ontologies	FaCT++ (ms)	HermiT (ms)
not-galen.owl	62,590	3,751
molecule-role	370,370	5,942
emap	120,187	7,243
teleost-taxonomy	1,392,537	14,259
obi	2,198	339,526
DLPOnts-Plans	658	621,033