oa Meten is weten? Over de waarde van de leesbaarheidsvoorspellingen van drie geautomatiseerde Nederlandse meetinstrumenten

In 2011, Kraf, Lentz & Pander Maat published a study in this journal into three automated tools for measuring the readability of texts written in Dutch: Texamen, Klinkende Taal and Accessibility Leesniveau Tool. Among other things, Kraf et al. (2011) showed that the outcomes of the three tools often differed from each other. It was not clear yet, however, how the predictions from these measuring instruments would compare to actual comprehensibility scores. Our study aimed to shed more light on the validity of the predictions from the three tools. We used the same 19 texts of which Kraf et al. (2011) published the scores from the three instruments for readability prediction. As measuring instrument for actual text comprehensibility the cloze procedure was applied. In order to ensure that each word would have an equal chance of being deleted of being deleted, all in all 95 cloze tests were constructed. In 19 of these tests, the first, sixth, eleventh, etcetera words were deleted; in 19 other tests, the second, seventh, etcetera words were deleted, etcetera. Each of the 125 participants in our study was presented with three of four of the resulting cloze tests. Possibly interfering effects of the subjects of the texts and the order of presentation were controlled for. Between the outcomes of the cloze tests (exact scoring method) and the predictions from Accessibility Leesniveau Tool and Klinkende Taal statistically significant but moderate correlations were found. No significant correlation was found between the outcomes of the cloze tests and the predictions from Texamen. Furthermore, possible relationships between the outcomes of the cloze tests and seven text characteristics were investigated. It was found that average word length and average sentence length, rather than other characteristics such as proportion of frequently used words or type-token ratio, contributed to explained variance in the cloze scores.