The RDF(S) Import Benchmark Suite is composed of the different benchmarks that have to be performed in each ontology development tool to evaluate their RDF(S) importers.
This benchmark suite has been built regarding the components of the knowledge model of RDF(S) commonly used by ontology development tools (classes, properties and instances) and the combinations that can be obtained with these components. It also considers the different ways of writing these components in the RDF/XML syntax.
In order to diminish the effort of executing the benchmark suites, a template for the RDF(S) Import Benchmark Suite results is provided.
Each benchmark imports one specific combination of elements of the knowledge model of RDF(S) into the tool and checks if the elements were correctly imported into the ontology development tool. Some of these benchmarks (like I03) may not have a correct meaning in some tools.
The description of a benchmark includes the benchmark identifier, a description of the benchmark in natural language, a graphical representation of the graph, and files with the RDF(S) graph that must be imported both in the Notation3 and RDF/XML formats (the RDF/XML file has been generated from the Notation3 file using cwm).
This benchmark suite was updated according to the comments and suggestions identified by the benchmarking participants while performing the experimentation and during the Interoperability Working Days. The old benchmark suite can be found here.
The results of the Evaluation stage 1 of the benchmarking are available, which include the results of the execution of the RDF(S) Import Benchmark Suite in several tools.
The benchmarks are grouped according to the main component that they manage, and these are: classes, properties or instances.
All the files composing the benchmark suite can be downloaded here.
| Legend:
|
| Class benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I01 | Import just one class |
|
graph01.n3 | graph01.rdfs |
| I02 | Import several classes with no properties between them |
|
graph02.n3 | graph02.rdfs |
| Metaclass benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I03 | Import one class that is instance of another class, being this last class instance of a third one |
|
graph03.n3 | graph03.rdfs |
| I04 | Import one class that is instance of several classes |
|
graph04.n3 | graph04.rdfs |
| I05 | Import several classes that are instance of the same class |
|
graph05.n3 | graph05.rdfs |
| I06 | Import one class that is instance of another class and viceversa |
|
graph06.n3 | graph06.rdfs |
| I07 | Import just one class that is instance of himself |
|
graph07.n3 | graph07.rdfs |
| Subclass benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I08 | Import one class that is subclass of another class, being this last class subclass of a third one |
|
graph08.n3 | graph08.rdfs |
| I09 | Import one class that is subclass of several classes |
|
graph09.n3 | graph09.rdfs |
| I10 | Import several classes that are subclass of the same class |
|
graph10.n3 | graph10.rdfs |
| I11 | Import one class that is subclass of another class and viceversa, forming a cycle |
|
graph11.n3 | graph11.rdfs |
| I12 | Import just one class that is subclass of himself, forming a cycle |
|
graph12.n3 | graph12.rdfs |
| Class and property benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I13 | Import one class that has a property with another class. The property is supposed to be defined with a domain and a range of some metaclass of the classes (such as rdfs:Class) |
|
graph13.n3 | graph13.rdfs |
| I14 | Import one class that has the same property with several classes. The property is supposed to be defined with a domain and a range of some metaclass of the classes (such as rdfs:Class) |
|
graph14.n3 | graph14.rdfs |
| I15 | Import several classes that have the same property with the same class. The property is supposed to be defined with a domain and a range of some metaclass of the classes (such as rdfs:Class) |
|
graph15.n3 | graph15.rdfs |
| I16 | Import just one class that has a property with itself. The property is supposed to be defined with a domain and a range of some metaclass of the classes (such as rdfs:Class) |
|
graph16.n3 | graph16.rdfs |
| I17 | Import just one class that has a property with a literal. The property is supposed to be defined with a domain and a range of some metaclass of the classes (such as rdfs:Class) |
|
graph17.n3 | graph17.rdfs |
| I18 | Import just one class that has the same property with several literals. The property is supposed to be defined with a domain and a range of some metaclass of the classes (such as rdfs:Class) |
|
graph18.n3 | graph18.rdfs |
| Single property benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I19 | Import just one property |
|
graph19.n3 | graph19.rdfs |
| I20 | Import several properties |
|
graph20.n3 | graph20.rdfs |
| Subproperty benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I21 | Import one property that is subproperty of another property that is subproperty of a third one |
|
graph21.n3 | graph21.rdfs |
| I22 | Import one property that is subproperty of several properties |
|
graph22.n3 | graph22.rdfs |
| I23 | Import several properties that are subproperty of the same property |
|
graph23.n3 | graph23.rdfs |
| I24 | Import one property that is subproperty of another property and viceversa |
|
graph24.n3 | graph24.rdfs |
| I25 | Import just one property that is subproperty of himself |
|
graph25.n3 | graph25.rdfs |
| Property with domain and range benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I26 | Import just one property that has as domain a resource and as range another resource, without the resource definitions |
|
graph26.n3 | graph26.rdfs |
| I27 | Import just one property that has as domain a class, with the class defined in the ontology |
|
graph27.n3 | graph27.rdfs |
| I28 | Import just one property that has as domain several classes, with the classes defined in the ontology |
|
graph28.n3 | graph28.rdfs |
| I29 | Import several properties that have as domain the same class, with the class defined in the ontology |
|
graph29.n3 | graph29.rdfs |
| I30 | Import just one property that has as domain rdfs:Class |
|
graph30.n3 | graph30.rdfs |
| I31 | Import several properties that have as domain rdfs:Class |
|
graph31.n3 | graph31.rdfs |
| I32 | Import just one property that has as range a class, with the class defined in the ontology |
|
graph32.n3 | graph32.rdfs |
| I33 | Import just one property that has as range several classes, with the classes defined in the ontology |
|
graph33.n3 | graph33.rdfs |
| I34 | Import several properties that have as range the same class, with the class defined in the ontology |
|
graph34.n3 | graph34.rdfs |
| I35 | Import just one property that has as range rdfs:Class |
|
graph35.n3 | graph35.rdfs |
| I36 | Import several properties that have as range rdfs:Class |
|
graph36.n3 | graph36.rdfs |
| I37 | Import just one property that has as range rdfs:Literal |
|
graph37.n3 | graph37.rdfs |
| I38 | Import several properties that have as range rdfs:Literal |
|
graph38.n3 | graph38.rdfs |
| I39 | Import just one property that has as domain a class and as range another class, with the classes defined in the ontology |
|
graph39.n3 | graph39.rdfs |
| I40 | Import just one property that has as domain a class and as range several classes, with the classes defined in the ontology |
|
graph40.n3 | graph40.rdfs |
| I41 | Import just one property that has as domain several classes and as range a class, with the classes defined in the ontology |
|
graph41.n3 | graph41.rdfs |
| I42 | Import just one property that has as domain several classes and as range several classes, with the classes defined in the ontology |
|
graph42.n3 | graph42.rdfs |
| I43 | Import just one property that has as domain and range the same class, with the class defined in the ontology |
|
graph43.n3 | graph43.rdfs |
| I44 | Import just one property that has as domain a class and as range rdfs:Literal, with the class defined in the ontology |
|
graph44.n3 | graph44.rdfs |
| I45 | Import just one property that has as domain several classes and as range rdfs:Literal, with the classes defined in the ontology |
|
graph45.n3 | graph45.rdfs |
| I46 | Import just one property that has as domain a class and as range the XML Schema datatype "string", with the class defined in the ontology |
|
graph46.n3 | graph46.rdfs |
| I47 | Import just one property that has as domain several classes and as range the XML Schema datatype "integer", with the classes defined in the ontology |
|
graph47.n3 | graph47.rdfs |
| I48 | Import just one property that has as domain rdfs:Class and as range rdfs:Class |
|
graph48.n3 | graph48.rdfs |
| I49 | Import just one property that has as domain rdfs:Class and as range rdfs:Literal |
|
graph49.n3 | graph49.rdfs |
| Instance benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I50 | Import just one instance of a resource, without the resource definition |
|
graph50.n3 | graph50.rdfs |
| I51 | Import one class and one instance of the class |
|
graph51.n3 | graph51.rdfs |
| I52 | Import several classes and one instance of all of them |
|
graph52.n3 | graph52.rdfs |
| I53 | Import one class and several instances of the class |
|
graph53.n3 | graph53.rdfs |
| Instance and property benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I54 | Import one class and one instance of the class that has a property with another instance of the same class, without the property definition |
|
graph54.n3 | graph54.rdfs |
| I55 | Import two classes and one instance of one class that has a property with an instance of the other class, without the property definition |
|
graph55.n3 | graph55.rdfs |
| I56 | Import one class and one instance of the class that has a property with a literal, without the property definition |
|
graph56.n3 | graph56.rdfs |
| I57 | Import one class, one property with domain and range the class, and one instance of the class that has the property with another instance of the same class |
|
graph57.n3 | graph57.rdfs |
| I58 | Import one class, one property with domain and range the class, and one instance of the class that has the property with several instances of the class |
|
graph58.n3 | graph58.rdfs |
| I59 | Import one class, one property with domain and range the class, and several instances of the class that have the property with the same instance of the class |
|
graph59.n3 | graph59.rdfs |
| I60 | Import one class, one property with domain and range the class, and one instance of the class that has the property with himself |
|
graph60.n3 | graph60.rdfs |
| I61 | Import two classes, one property with domain one class and range the other class, and one instance of one class that has the property with an instance of the other class |
|
graph61.n3 | graph61.rdfs |
| I62 | Import two classes, one property with domain one class and range the other class, and one instance of one class that has the property with several instances of the other class |
|
graph62.n3 | graph62.rdfs |
| I63 | Import two classes, one property with domain one class and range the other class, and several instances of one class that have the property with the same instance of the other class |
|
graph63.n3 | graph63.rdfs |
| I64 | Import one class, one property with domain the class and range rdfs:Literal, and one instance of the class that has the property with a literal |
|
graph64.n3 | graph64.rdfs |
| I65 | Import one class, one property with domain the class and range rdfs:Literal, and one instance of the class that has the property with several literals |
|
graph65.n3 | graph65.rdfs |
| I66 | Import one class, one property with domain the class and range the XML Schema datatype "string", and one instance of the class that has the property with a value |
|
graph66.n3 | graph66.rdfs |
| I67 | Import one class, one property with domain the class and range the XML Schema datatype "integer", and one instance of the class that has the property with several integer values |
|
graph67.n3 | graph67.rdfs |
| URI reference benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I68 | Import several resources with absolute URI references | graph68.rdfs | ||
| I69 | Import several resources with URI references relative to a base URI | graph69.rdfs | ||
| I70 | Import several resources with URI references transformed from rdf:ID attribute values | graph70.rdfs | ||
| I71 | Import several resources with URI references relative to an ENTITY declaration | graph71.rdfs | ||
| Empty node benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I72 | Import several resources with empty nodes | graph72.rdfs | ||
| I73 | Import several resources with empty nodes shortened | graph73.rdfs | ||
| Multiple properties benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I74 | Import several resources with multiple properties | graph74.rdfs | ||
| I75 | Import several resources with multiple properties shortened | graph75.rdfs | ||
| Typed node benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I76 | Import several resources with typed nodes | graph76.rdfs | ||
| I77 | Import several resources with typed nodes shortened | graph77.rdfs | ||
| String literal benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I78 | Import several resources with properties with string literals | graph78.rdfs | ||
| I79 | Import several resources with properties with string literals as XML attributes | graph79.rdfs | ||
| Blank node benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I80 | Import several resources with blank nodes with identifier | graph80.rdfs | ||
| I81 | Import several resources with blank nodes shortened | graph81.rdfs | ||
| Language identification benchmarks | ||||
| Id | Description | Graphical representation | N3 | RDF/XML |
| I82 | Import several resources with properties with xml:lang attributes | graph82.rdfs | ||
