Tijdschrift voor Taalbeheersing - Volume 41, Issue 2, 2019

Text features and quality of learner text: an annotation study

Manually annotated corpora of writing products may greatly contribute to writing research: they offer detailed insights in the quality of these texts, in the text features actually attended to by human text raters, in possibilities and difficulties for the use of automatic writing analytics and writing tools, and in the relations between different text quality dimensions. This paper presents the Utrecht System for Annotation of Learner text (USALT), that covers both general features (orthography, punctuation, wording, coherence) and genre-specific elements (such as openings, endings, structuring devices and politeness). The annotations contain up to three items (annotation unit; problem type; part-of-speech tag). USALT reflects various text quality dimensions, notably correctness, comprehensibility and appropriateness (both stylistically and in terms of genre conventions).

We present an USALT analysis of 371 texts produced by Dutch students from grades 7-9 (aged 12-15 years), taken from the so-called Schrijfmeters-corpus. The assignment concerned a letter about ‘typically Dutch things’ to a Swedish girl about to emigrate to The Netherlands. USALT reliabilities were adequate. In terms of problem frequency, we were struck by the pervasiveness of punctuation problems. Furthermore, the orthography and punctuation problems together present considerable difficulties for automatic analysis of original learner texts at this level. A remarkable result regarding relations between various text quality dimensions is that the frequency of orthography problems correlates higher with genre convention problems than with lexico-grammatical problems. We also used the annotations as predictors of the holistic scores assigned to the texts by human raters. Standardized annotation frequencies by themselves may account for 45% of the score variance, with a prominent role for annotations regarding genre elements; text length by itself explains 52%. The best model includes both text length and annotations (65% explained variance). In ongoing work, USALT is being extended to handle argumentative writing assignments.

Oh (/o/) as information receipt in Dutch talk-in-interaction

Recently there has been an increased interest cross-linguistically in how speakers use interjections in everyday interaction. A particularly productive line of inquiry deals with what are known as change-of-state tokens, interjections with which speakers claim that there has been a shift in their cognitive state such as their knowledge, understanding, attention, etc. In this paper I explore the variability of the Dutch interjection oh /o/. Focusing on its use in response to informing turns, I argue that as a free-standing particle speakers use it to claim that the information in that prior turn was in some way unexpected, either because it contradicted what the speaker claimed he or she knew, or because it contradicted some presupposition that was encoded in an earlier question. I subsequently discuss the most frequent ways in which oh is combined with other turn components, showing how it is used to respond to announcements of valenced news, to do now-remembering, and to make claims of now-understanding. In closing I show that when oh prefaces additional turn components such as oké (‘okay’), each component deals with a different action-implication of the ongoing sequence and that oh is used to receive the information being conveyed.

Webcare via public and private social media. A corpus study on the effect of Conversational Human Voice in the PT sector

This study investigates the use of Conversational Human Voice (CHV) by a Dutch Public Transport (PT) operator in reactive webcare conversations with travelers on private and public social media channels and the effect of this use on the traveler’s sentiment during the conversation. In this study, CHV is unraveled into eight aspects. 244 conversations were analyzed, selected from the PT-operator’s public and private social media channels. Messages sent by the PT companies were coded for CHV; messages of travelers were coded for sentiment. The aspects organization responds as an individual and informal language were used the most often by the PT operator. Use of CHV is similar on private social networks and public social networks, with the exception of organization responds as an individual (used more on private social networks) and language to compensate the lack of non-verbal communication (used more on public social networks). Furthermore, results show that the use of sympathy has a positive effect on the traveler’s sentiment.