The RDF(S) Export Benchmark Suite is composed of the different benchmarks that have to be performed in each ontology development tool to evaluate their RDF(S) exporters.
This benchmark suite has been built regarding the common components of the knowledge model of ontology development tools (classes, datatype properties, object properties, and instances) and the combinations that can be obtained with these components.
In order to diminish the effort of executing the benchmark suites, a template for the RDF(S) Export Benchmark Suite results is provided.
Each benchmark exports one specific combination of elements of the knowledge model of the tool to RDF(S) and checks if the elements were correctly exported.
The description of a benchmark includes the benchmark identifier, a description of the benchmark in natural language and a graphical representation of the ontology. Then, for each ontology development tool, the benchmark must be instantiated for its own knowledge model.
This benchmark suite was updated according to the comments and suggestions identified by the benchmarking participants while performing the experimentation and during the Interoperability Working Days. The old benchmark suite can be found here.
The results of the Evaluation stage 1 of the benchmarking are available, which include the results of the execution of the RDF(S) Export Benchmark Suite in several tools.
The benchmarks are grouped according to the main component that they manage, and these are: classes, properties or instances. It also shows the instantiation of the benchmark for WebODE.
When the combination of components of the benchmark cannot be represented in the knowledge model of the tool, it is represented with "-".
| Legend:
|
| Class benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E01 | Export just one class |
|
Export just one concept |
| E02 | Export several classes |
|
Export several concepts |
| Metaclass benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E03 | Export one class that is instance of another class that is instance of a third one |
|
- |
| E04 | Export one class that is instance of several classes |
|
- |
| E05 | Export several classes that are instance of the same class |
|
- |
| E06 | Export one class that is instance of another class and viceversa |
|
- |
| E07 | Export just one class that is instance of himself |
|
- |
| Subclass benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E08 | Export one class that is subclass of another class that is subclass of a third one |
|
Export one concept that is subclass of another concept that is subclass of a third one |
| E09 | Export one class that is subclass of several classes |
|
Export one concept that is subclass of several concepts |
| E10 | Several classes that are subclass of the same class |
|
Export several concepts that are subclass of the same concept |
| E11 | Export one class that is subclass of another class and viceversa, forming a cycle |
|
- |
| E12 | Export just one class that is subclass of himself, forming a cycle |
|
- |
| Class and object property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E13 | Export one class that has an object property with another class. The property is supposed to be defined with a domain and a range of some metaclass of the classes |
|
- |
| E14 | Export one class that has the same object property with several classes. The property is supposed to be defined with a domain and a range of some metaclass of the classes |
|
- |
| E15 | Export several classes that have the same object property with the same class. The property is supposed to be defined with a domain and a range of some metaclass of the classes |
|
- |
| E16 | Export just one class that has an object property with itself. The property is supposed to be defined with a domain and a range of some metaclass of the class |
|
- |
| Class and datatype property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E17 | Export just one class that has a datatype property with a literal. The property is supposed to be defined with a domain and a range of some metaclass of the class |
|
- |
| E18 | Export just one class that has the same datatype property with several literals. The property is supposed to be defined with a domain and a range of some metaclass of the class |
|
- |
| Datatype property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E19 | Export just one datatype property |
|
- |
| E20 | Export several datatype properties |
|
- |
| E21 | Export just one datatype property that has as domain a resource and as range "String", without the resource definition |
|
- |
| E22 | Export just one datatype property that has as domain a class, with the class defined in the ontology |
|
- |
| E23 | Export just one datatype property that has as domain several classes, with the classes defined in the ontology |
|
- |
| E24 | Export several datatype properties that have as domain the same class, with the class defined in the ontology |
|
- |
| E25 | Export just one datatype property that has as range "String" |
|
- |
| E26 | Export several datatype properties that have as range "String" |
|
- |
| E27 | Export one datatype property that has as domain a class and as range "String", with the class defined in the ontology |
|
Export one concept that has an instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" |
| E28 | Export one datatype property that has as domain several classes and as range "String", with the classes defined in the ontology |
|
- |
| E29 | Export one datatype property that has as domain a class and as range the XML Schema datatype "string", with the class defined in the ontology |
|
Export one concept that has an instance attribute of type "xsd:string" with a minimum cardinality of "1" and a maximum cardinality of "N" |
| E30 | Export one datatype property that has as domain several classes and as range the XML Schema datatype "integer", with the classes defined in the ontology |
|
- |
| Object property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E31 | Export just one object property |
|
- |
| E32 | Export several object properties |
|
- |
| E33 | Export just one object property that has as domain a resource and as range another resource, without the resource definitions |
|
- |
| E34 | Export just one object property that has as domain a class, with the class defined in the ontology |
|
- |
| E35 | Export just one object property that has as domain several classes, with the classes defined in the ontology |
|
- |
| E36 | Export several object properties that have as domain the same class, with the class defined in the ontology |
|
- |
| E37 | Export just one object property that has as range a class, with the class defined in the ontology |
|
- |
| E38 | Export just one object property that has as range several classes, with the classes defined in the ontology |
|
- |
| E39 | Export several object properties that have as range the same class, with the class defined in the ontology |
|
- |
| E40 | Export just one object property that has as domain a class and as range another class, with the classes defined in the ontology |
|
Export one concept that has a relation with another concept with a cardinality of "N" |
| E41 | Export just one object property that has as domain a class and as range several classes, with the classes defined in the ontology |
|
- |
| E42 | Export just one object property that has as domain several classes and as range a class, with the classes defined in the ontology |
|
- |
| E43 | Export just one object property that has as domain several classes and as range several classes, with the classes defined in the ontology |
|
- |
| E44 | Export just one object property that has as domain and range the same class, with the class defined in the ontology |
|
Export one concept that has a relation with itself with a cardinality of "N" |
| Instance benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E45 | Export just one instance of a resource, without the resource definition |
|
- |
| E46 | Export one class and one instance of the class |
|
Export one concept and one instance of the concept |
| E47 | Export several classes and one instance of all of them |
|
- |
| E48 | Export one class and several instances of the class |
|
Export one concept and several instances of the concept |
| Instance and object property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E49 | Export one class and one instance of the class that has an object property with another instance of the same class, without the property definition |
|
- |
| E50 | Export two classes and one instance of one class that has an object property with an instance of the other class, without the property definition |
|
- |
| E51 | Export one class, one object property with domain and range the class, and one instance of the class that has the property with another instance of the same class |
|
Export one concept that has a relation with itself with a cardinality of "N" and one instance of the concept that has a relation instance with another instance of the same concept |
| E52 | Export one class, one object property with domain and range the class, and one instance of the class that has the property with several instances of the class |
|
Export one concept that has a relation with itself with a cardinality of "N" and one instance of the concept that has a relation instance with several instances of the same concept |
| E53 | Export one class, one object property with domain and range the class, and several instances of the class that have the property with the same instance of the class |
|
Export one concept that has a relation with itself with a cardinality of "N" and several instances of the concept that have a relation instance with another instance of the same concept |
| E54 | Export one class, one object property with domain and range the class, and one instance of the class that has the property with himself |
|
Export one concept that has a relation with itself with a cardinality of "N" and one instance of the concept that has a relation instance with itself |
| E55 | Export two classes, one object property with domain one class and range the other class, and one instance of one class that has the property with an instance of the other class |
|
Export one concept that has a relation with another concept with a cardinality of "N" and one instance of the concept that has a relation instance with an instance of the other concept |
| E56 | Export two classes, one object property with domain one class and range the other class, and one instance of one class that has the property with several instances of the other class |
|
Export one concept that has a relation with another concept with a cardinality of "N" and one instance of the concept that has a relation instance with several instances of the other concept |
| E57 | Export two classes, one object property with domain one class and range the other class, and several instances of one class that have the property with the same instance of the other class |
|
Export one concept that has a relation with another concept with a cardinality of "N" and several instances of the concept that have a relation instance with an instance of the other concept |
| Instance and datatype property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E58 | Export one class and one instance of the class that has a datatype property with a literal, without the property definition |
|
- |
| E59 | Export one class, one datatype property with domain the class and range "String", and one instance of the class that has the property with a literal |
|
Export one concept that has an instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with a string value in the instance attribute |
| E60 | Export one class, one datatype property with domain the class and range "String", and one instance of the class that has the property with several literals |
|
Export one concept that has an instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with several string values in the instance attribute |
| E61 | Export one class, one datatype property with domain the class and range the XML Schema datatype "string", and one instance of the class that has the property with a value |
|
Export one concept that has an instance attribute of type "xsd:string" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with a string value in the instance attribute |
| E62 | Export one class, one datatype property with domain the class and range the XML Schema datatype "integer", and one instance of the class that has the property with several integer values |
|
Export one concept that has an instance attributes of type "xsd:integer" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with several integer values in the instance attribute |
| Concepts and properties whose names start with a character that is not a letter or _ | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E63 | Export an ontology containing two classes named "1class" and "2class", each with one datatype property of type String named "-datatypeProperty1" and "-datatypeProperty2" respectively, and an object property between the classes named ".objectProperty" | Export an ontology containing two concepts named "1concept" and "2concept", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "-datatypeProperty1" and "-datatypeProperty2" respectively, and a relation between the concepts named ".objectProperty" with a cardinality of "N" | |
| Concepts and properties with spaces in their names | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E64 | Export an ontology containing two classes named "class 1" and "class 2", each with one datatype property of type String named "datatype property 1" and "datatype property 2" respectively, and an object property between the classes named "object property" | Export an ontology containing two concepts named "concept 1" and "concept 2", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "datatype property 1" and "datatype property 2" respectively, and a relation between the concepts named "object property" with a cardinality of "N" | |
| Concepts and properties with URI reserved characters in their names (;, /, ?, :, @, &, =, +, $, ,) | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E65 | Export an ontology containing two classes named "concept/1" and "concept:1", each with one datatype property of type String named "datatype/property/1" and "datatype=property=2" respectively, and an object property between the classes named "object$property" | Export an ontology containing two concepts named "concept/1" and "concept:1", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "datatype/property/1" and "datatype=property=2" respectively, and a relation between the concepts named "object$property" with a cardinality of "N" | |
| Concepts and properties with XML delimiter characters in their names ('<', '>', '#', '%', '"') | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| E66 | Export an ontology containing two classes named "class<1" and "class>1", each with one datatype property of type String named "datatype#property#1" and "datatype%property%2" respectively, and an object property between the classes named "object"property" | Export an ontology containing two concepts named "concept<1" and "concept>1", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "datatype#property#1" and "datatype%property%2" respectively, and a relation a relation between the concepts named "object"property" with a cardinality of "N" | |
