The RDF(S) Interoperability Benchmark Suite is composed of the different benchmarks that have to be performed in each ontology development tool to evaluate their interoperability by testing the interchange of ontologies, from one origin tool to a destination one, and vice versa.
This benchmark suite has been built regarding the common components of the knowledge model of ontology development tools (classes, datatype properties, object properties, and instances) and the combinations that can be obtained with these components. As these combinations are the same as those from the RDF(S) Export Benchmark Suite, the definitions of the interoperability benchmarks will be the same than the definitions of the export benchmarks.
The interoperability will be checked between each tool and all the participant tools, including the tool itself. As the RDF(S) exported files of all the tools are available from the export experiments of the Evaluation stage 1, participants do not have to export these ontologies again, they just have to import the exported files into their tools.
As there are benchmarks that define ontologies with combinations of components that cannot be modelled in some tools, these tool are not able of exporting these ontologies and, therefore, these benchmarks will not be executed.
In order to diminish the effort of executing the benchmark suites, templates for the RDF(S) Interoperability Benchmark Suite results are provided. The benchmarks colored in grey are those that must not be executed, as the origin tool cannot model the ontology (as mentioned above).
All the neccesary files for executing the RDF(S) Interoperability Benchmark Suite (Evaluation Stage 1 results, exported RDF(S) files, and result templates) can be downloaded in a single file.
Each benchmark interchanges one specific combination of elements of the knowledge model of the tool to RDF(S) and checks if the elements were correctly interchanged.
The description of a benchmark includes the benchmark identifier, a description of the benchmark in natural language and a graphical representation of the ontology. Then, for each ontology development tool, the benchmark must be instantiated for its own knowledge model.
The benchmarks are grouped according to the main component that they manage, and these are: classes, properties or instances. It also shows the instantiation of the benchmark for WebODE.
When the combination of components of the benchmark cannot be represented in the knowledge model of the tool, it is represented with "-".
| Legend:
|
| Class benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In01 | Interchange just one class |
|
Interchange just one concept |
| In02 | Interchange several classes |
|
Interchange several concepts |
| Metaclass benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In03 | Interchange one class that is instance of another class that is instance of a third one |
|
- |
| In04 | Interchange one class that is instance of several classes |
|
- |
| In05 | Interchange several classes that are instance of the same class |
|
- |
| In06 | Interchange one class that is instance of another class and viceversa |
|
- |
| In07 | Interchange just one class that is instance of himself |
|
- |
| Subclass benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In08 | Interchange one class that is subclass of another class that is subclass of a third one |
|
Interchange one concept that is subclass of another concept that is subclass of a third one |
| In09 | Interchange one class that is subclass of several classes |
|
Interchange one concept that is subclass of several concepts |
| In10 | Several classes that are subclass of the same class |
|
Interchange several concepts that are subclass of the same concept |
| In11 | Interchange one class that is subclass of another class and viceversa, forming a cycle |
|
- |
| In12 | Interchange just one class that is subclass of himself, forming a cycle |
|
- |
| Class and object property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In13 | Interchange one class that has an object property with another class. The property is supposed to be defined with a domain and a range of some metaclass of the classes |
|
- |
| In14 | Interchange one class that has the same object property with several classes. The property is supposed to be defined with a domain and a range of some metaclass of the classes |
|
- |
| In15 | Interchange several classes that have the same object property with the same class. The property is supposed to be defined with a domain and a range of some metaclass of the classes |
|
- |
| In16 | Interchange just one class that has an object property with itself. The property is supposed to be defined with a domain and a range of some metaclass of the class |
|
- |
| Class and datatype property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In17 | Interchange just one class that has a datatype property with a literal. The property is supposed to be defined with a domain and a range of some metaclass of the class |
|
- |
| In18 | Interchange just one class that has the same datatype property with several literals. The property is supposed to be defined with a domain and a range of some metaclass of the class |
|
- |
| Datatype property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In19 | Interchange just one datatype property |
|
- |
| In20 | Interchange several datatype properties |
|
- |
| In21 | Interchange just one datatype property that has as domain a resource and as range "String", without the resource definition |
|
- |
| In22 | Interchange just one datatype property that has as domain a class, with the class defined in the ontology |
|
- |
| In23 | Interchange just one datatype property that has as domain several classes, with the classes defined in the ontology |
|
- |
| In24 | Interchange several datatype properties that have as domain the same class, with the class defined in the ontology |
|
- |
| In25 | Interchange just one datatype property that has as range "String" |
|
- |
| In26 | Interchange several datatype properties that have as range "String" |
|
- |
| In27 | Interchange one datatype property that has as domain a class and as range "String", with the class defined in the ontology |
|
Interchange one concept that has an instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" |
| In28 | Interchange one datatype property that has as domain several classes and as range "String", with the classes defined in the ontology |
|
- |
| In29 | Interchange one datatype property that has as domain a class and as range the XML Schema datatype "string", with the class defined in the ontology |
|
Interchange one concept that has an instance attribute of type "xsd:string" with a minimum cardinality of "1" and a maximum cardinality of "N" |
| In30 | Interchange one datatype property that has as domain several classes and as range the XML Schema datatype "integer", with the classes defined in the ontology |
|
- |
| Object property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In31 | Interchange just one object property |
|
- |
| In32 | Interchange several object properties |
|
- |
| In33 | Interchange just one object property that has as domain a resource and as range another resource, without the resource definitions |
|
- |
| In34 | Interchange just one object property that has as domain a class, with the class defined in the ontology |
|
- |
| In35 | Interchange just one object property that has as domain several classes, with the classes defined in the ontology |
|
- |
| In36 | Interchange several object properties that have as domain the same class, with the class defined in the ontology |
|
- |
| In37 | Interchange just one object property that has as range a class, with the class defined in the ontology |
|
- |
| In38 | Interchange just one object property that has as range several classes, with the classes defined in the ontology |
|
- |
| In39 | Interchange several object properties that have as range the same class, with the class defined in the ontology |
|
- |
| In40 | Interchange just one object property that has as domain a class and as range another class, with the classes defined in the ontology |
|
Interchange one concept that has a relation with another concept with a cardinality of "N" |
| In41 | Interchange just one object property that has as domain a class and as range several classes, with the classes defined in the ontology |
|
- |
| In42 | Interchange just one object property that has as domain several classes and as range a class, with the classes defined in the ontology |
|
- |
| In43 | Interchange just one object property that has as domain several classes and as range several classes, with the classes defined in the ontology |
|
- |
| In44 | Interchange just one object property that has as domain and range the same class, with the class defined in the ontology |
|
Interchange one concept that has a relation with itself with a cardinality of "N" |
| Instance benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In45 | Interchange just one instance of a resource, without the resource definition |
|
- |
| In46 | Interchange one class and one instance of the class |
|
Interchange one concept and one instance of the concept |
| In47 | Interchange several classes and one instance of all of them |
|
- |
| In48 | Interchange one class and several instances of the class |
|
Interchange one concept and several instances of the concept |
| Instance and object property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In49 | Interchange one class and one instance of the class that has an object property with another instance of the same class, without the property definition |
|
- |
| In50 | Interchange two classes and one instance of one class that has an object property with an instance of the other class, without the property definition |
|
- |
| In51 | Interchange one class, one object property with domain and range the class, and one instance of the class that has the property with another instance of the same class |
|
Interchange one concept that has a relation with itself with a cardinality of "N" and one instance of the concept that has a relation instance with another instance of the same concept |
| In52 | Interchange one class, one object property with domain and range the class, and one instance of the class that has the property with several instances of the class |
|
Interchange one concept that has a relation with itself with a cardinality of "N" and one instance of the concept that has a relation instance with several instances of the same concept |
| In53 | Interchange one class, one object property with domain and range the class, and several instances of the class that have the property with the same instance of the class |
|
Interchange one concept that has a relation with itself with a cardinality of "N" and several instances of the concept that have a relation instance with another instance of the same concept |
| In54 | Interchange one class, one object property with domain and range the class, and one instance of the class that has the property with himself |
|
Interchange one concept that has a relation with itself with a cardinality of "N" and one instance of the concept that has a relation instance with itself |
| In55 | Interchange two classes, one object property with domain one class and range the other class, and one instance of one class that has the property with an instance of the other class |
|
Interchange one concept that has a relation with another concept with a cardinality of "N" and one instance of the concept that has a relation instance with an instance of the other concept |
| In56 | Interchange two classes, one object property with domain one class and range the other class, and one instance of one class that has the property with several instances of the other class |
|
Interchange one concept that has a relation with another concept with a cardinality of "N" and one instance of the concept that has a relation instance with several instances of the other concept |
| In57 | Interchange two classes, one object property with domain one class and range the other class, and several instances of one class that have the property with the same instance of the other class |
|
Interchange one concept that has a relation with another concept with a cardinality of "N" and several instances of the concept that have a relation instance with an instance of the other concept |
| Instance and datatype property benchmarks | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In58 | Interchange one class and one instance of the class that has a datatype property with a literal, without the property definition |
|
- |
| In59 | Interchange one class, one datatype property with domain the class and range "String", and one instance of the class that has the property with a literal |
|
Interchange one concept that has an instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with a string value in the instance attribute |
| In60 | Interchange one class, one datatype property with domain the class and range "String", and one instance of the class that has the property with several literals |
|
Interchange one concept that has an instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with several string values in the instance attribute |
| In61 | Interchange one class, one datatype property with domain the class and range the XML Schema datatype "string", and one instance of the class that has the property with a value |
|
Interchange one concept that has an instance attribute of type "xsd:string" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with a string value in the instance attribute |
| In62 | Interchange one class, one datatype property with domain the class and range the XML Schema datatype "integer", and one instance of the class that has the property with several integer values |
|
Interchange one concept that has an instance attributes of type "xsd:integer" with a minimum cardinality of "1" and a maximum cardinality of "N" and one instance of the concept with several integer values in the instance attribute |
| Concepts and properties whose names start with a character that is not a letter or _ | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In63 | Interchange an ontology containing two classes named "1class" and "2class", each with one datatype property of type String named "-datatypeProperty1" and "-datatypeProperty2" respectively, and an object property between the classes named ".objectProperty" | Interchange an ontology containing two concepts named "1concept" and "2concept", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "-datatypeProperty1" and "-datatypeProperty2" respectively, and a relation between the concepts named ".objectProperty" with a cardinality of "N" | |
| Concepts and properties with spaces in their names | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In64 | Interchange an ontology containing two classes named "class 1" and "class 2", each with one datatype property of type String named "datatype property 1" and "datatype property 2" respectively, and an object property between the classes named "object property" | Interchange an ontology containing two concepts named "concept 1" and "concept 2", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "datatype property 1" and "datatype property 2" respectively, and a relation between the concepts named "object property" with a cardinality of "N" | |
| Concepts and properties with URI reserved characters in their names (;, /, ?, :, @, &, =, +, $, ,) | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In65 | Interchange an ontology containing two classes named "concept/1" and "concept:1", each with one datatype property of type String named "datatype/property/1" and "datatype=property=2" respectively, and an object property between the classes named "object$property" | Interchange an ontology containing two concepts named "concept/1" and "concept:1", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "datatype/property/1" and "datatype=property=2" respectively, and a relation between the concepts named "object$property" with a cardinality of "N" | |
| Concepts and properties with XML delimiter characters in their names ('<', '>', '#', '%', '"') | |||
| Id | Description | Graphical representation | WebODE's instantiation |
| In66 | Interchange an ontology containing two classes named "class<1" and "class>1", each with one datatype property of type String named "datatype#property#1" and "datatype%property%2" respectively, and an object property between the classes named "object"property" | Interchange an ontology containing two concepts named "concept<1" and "concept>1", each with one instance attribute of type "String" with a minimum cardinality of "1" and a maximum cardinality of "N" named "datatype#property#1" and "datatype%property%2" respectively, and a relation a relation between the concepts named "object"property" with a cardinality of "N" | |
