Research Results: The EERQI Prototype Framework

PDF versionPrinter-friendly version

 

The EERQI Prototype Framework consists of several tools which were developed in the course of the project. Those tools assist the researcher or peer-reviewer in the process of quality detection:

  • search and query engine that enables the determined finding and identification of education research documents in the WWW. Additionally it harvest a content base consisting of educational texts provided by the EERQI publishers. The search and query engine was extended with multilingual functions in order to provide search results in the four EERQI language (English, German, French, Swedish). Thus the final product we call the multilingual search engine.

    • Sources and fingerprints for the classifier part of the EERQI subject focussed search engine (TAR-archive).

  • The tool aMeasure, capable of delivering information about the ‘extrinsic’ quality indicators (such as citations, webmentions) of a publication or author.
  • Automatic semantic analysis for the highlighting and thus fast detection of key sentences in a text which assists the reading process. The method is applicable to educational research publications (in at least) the four EERQI languages.

  • A set of intrinsic (= text-immanent) indicators for the detection of quality in educational research publications that has been presented to the research community and was positively evaluated.

  • An accompanying Peer Review Questionnaire with an operationalization of the intrinsic indicators in the form of 16 items that was tested for reliability and practicality.

  • First attempts to detect interrelations between ‘extrinsic’ and ‘intrinsic’ quality indicators.

  • A set of use case-scenarios that advice how to use an intelligent combination of the above mentioned tools appropriate for a given assessment task or process.

  • First tests of a citation analysis (scientometrics) revealing other reasons for citation than scientific excellence.

 

 

The EERQI Prototype Framework is a set of tools that can be used during the process of quality detection.

  • Part 1 is the detection of potential quality via identification of relevant educational research texts in different sources with the help of the EERQI multilingual search and query engine.

  • Part 2 is the application of ‘aMeasure’ (developed by EERQI partner HU-Berlin). ‘aMeasure’ is a stack of tools and programs to measure extrinsic characteristics of research publications (such as citations, webmentions) by using Google Scholar, Google Web Search, MetaGer, LibraryThing, Connotea, Mendeley, and citeulike. In the context of the EERQI project ‘aMeasure' was used to collect information about extrinsic indicators of quality of educational research publications.

  • Part 3 is the application of linguistic technology in order to provide automatic support for evaluating the quality of a text. The method developed in EERQI allows for the automatic and fast identification of key sentences to which the reader should pay particular attention (Automated semantic analysis).

  • Part 4 leads to a final quality judgement through the application of a Peer Review Questionnaire that contains a tested version of operationalization of the intrinsic indicators that were developed by the EERQI project.

The elements of the EERQI Prototype Framework can either be applied as single methods for specific parts of an assessment process; or they can be applied consecutively, leading to a final judgment on the basis of intense reading of selected texts. In the latter case, the parts of the framework can take over filter (or selection) functions in the assessment process.

 

The EERQI-project developed another relevant tool for supporting the process of quality detection: the EERQI peer review questionnaire. This instrument comprises of operationalized items that indicate internal features of the quality of a text. The reliability and acceptance of this questionnaire was tested with a positive result. Here you find the test version of the questionnaire >>>