Development of the Handwriting Legibility Scale (HLS): A preliminary examination of Reliability and Validity
Introduction
As children progress through school, there is a growing demand for the production of legible handwriting at reasonable speed (DfE, 2013). In the early school years the focus is on letter formation and joining, to produce handwriting that is neat and legible, both to the writer and to those trying to read the written product. The child must develop flexibility in their handwriting skill, to produce their very neatest writing when required (as for a wall display) and to be able to write quickly when in an examination. The development of fluent and legible handwriting becomes more and more important as demands increase through the school years. Children are expected to keep up with class work, to produce a greater volume of writing and to work in time-pressured situations. Handwriting has been described as a ‘low-level’ skill in the broader writing process (Berninger et al., 2002). With a limited working memory capacity, if lower-level skills are not automated, they can impact on ‘higher-level’ compositional processes (Connelly, Campbell, MacLean, & Barnes, 2006). This emphasizes the importance of identifying and supporting those with handwriting difficulties to ensure they are able to achieve their potential.
Teacher reports indicate that significant proportions of children in school classrooms have handwriting difficulties. For example, 12–13% has been reported in the UK (Barnett, Stainthorp, Henderson, & Scheib, 2006) and 23% in the USA (Graham et al., 2008). These difficulties include having slow and/or untidy writing that is difficult to read. Children may have handwriting difficulties for a variety of reasons. Some have neurological or physical disabilities affecting the muscles or joints. Others have difficulties with motor control and coordination in the absence of a neurological or physical disorder, for example children with Developmental Coordination Disorder (DCD) (Prunty, Barnett, Wilmut, & Plumb, 2014; Rosenblum & Livneh-Zirinski, 2008). Indeed poor handwriting is mentioned in the formal diagnostic criteria for this condition (APA, 2013). Handwriting difficulties are also common in other developmental disorders, including Attention Deficit Hyperactivity Disorder, Specific Language Impairment and Dyslexia (Dockrell, Lindsay, & Connelly, 2011; Rosenblum, Epsztein, & Josman, 2008; Sumner, Connelly, & Barnett, 2014). Another group of children seem to have isolated handwriting difficulties in the absence of a more general motor difficulty or other developmental disorder. Sometimes referred to as ‘dysgraphia’, this may be associated more with inadequacies in teaching and practice opportunities than a limitation within the child (Berninger & Dagmar, 2003).
Whatever the underlying cause, handwriting difficulties may lead to reluctance to write, underachievement and low self-esteem (Feder & Majnemer, 2007). It is hardly surprisingly therefore that they are often a cause for concern to parents, teachers and to children themselves (Hammerschmidt & Sudsawad, 2004) and are the most common reason for referral to Occupational Therapy (Feder, Majnemer, & Synnes, 2000). Despite the recognized importance of handwriting skill and the considerable number of children faced with difficulties in this area, there has been very little research to describe and quantify handwriting difficulties in children. This has in part been due to the lack of robust assessment instruments. Screening scales for identifying handwriting difficulties have been developed for capturing the teacher’s impression from the child's handwriting production in class (Rosenblum, 2008) or the child’s perception about his/her own handwriting production (Rosenblum & Gafni-Lachter, 2015). These questionnaires provide a useful overview of various aspects of handwriting (including the motivation to write, speed of production, legibility and fatigue) but do not involve the examination of a specific sample of handwriting, Tests of handwriting speed (e.g. Barnett, Henderson, Scheib, & Schulz, 2007; Smits-Engelsman, van Bommel-Rutgers, & van Waelvelde, 2015; Wallen, Bonney & Lennox, 1996) and digitizing tablets (with various specialized software programs) have been used to detail temporal and spatial aspects of performance (e.g. Prunty et al., 2014, Rosenblum and Livneh-Zirinski, 2008). These allow for an examination of both general and specific aspects of fluency and speed of production across a range of handwriting tasks (differing in length and cognitive demands). However, there is a lack of robust practical tools to assess the quality and legibility of the written product. Some scales have been developed to assess in great detail different components of handwriting legibility (for example the extent to which letters match a standard, the consistency of letter size or spacing between letters) (e.g. Reisman, 1993, Smits-Engelsman et al., 2015). Such assessments require scrutiny and measurement of single letters, can be time consuming to undertake and are therefore not practical for teachers to use in the classroom setting. Furthermore, the requirement to match the written sample to criterion letter ‘standards’ prevents their application to different written languages, handwriting styles or scripts.
Specially designed computerized assessments (e.g. Falk, Tam, Schellnus, & Chau, 2011) allow for a faster and more objective measurement of specific criteria of handwriting quality (e.g. letter size, spacing, alignment). However, the relationships between such criteria and the overall legibility of the written text as captured by a global impression are weak (see Rosenblum,Weiss, & Parush, 2003). Furthermore, unfortunately such technology is not commonly available and using it is not a trivial or intuitive task for class teachers. In fact teachers need a standardized practical tool for the measurement of legibility to help them identify those with difficulties and determine whether the child needs support to develop their skills (Hammerschmidt & Sudsawad, 2004).
The aim of the current study was therefore to develop a quick and easy tool to provide a reliable and valid assessment of handwriting legibility. The focus was on the assessment of a naturalistic (and therefore ecologically valid) handwriting task, which is easy to gather from large numbers of children in the usual classroom setting. The assessment is based on an overall impression of the handwriting (rather than a detailed examination of individual letters), with a view to application in different languages and scripts. The initial development of the tool focuses on children in the UK from the age of 8 years and upwards, who by this age will have received several years of instruction and should be developing effective skills of written communication (DfE, 2013). The analysis of construct validity focused on the ability of the test scores to discriminate between two groups of children, those with and without DCD. As mentioned above, handwriting difficulties seem to be a hallmark of DCD, frequently mentioned by parents and teachers and even included in the formal DSM-5 (APA, 2013) description of the condition. The ability of the new tool to discriminate performance of these two groups is therefore important. A reliable and valid assessment of handwriting legibility is much needed in order to easily and quickly identify those with difficulties, provide effective support and thereby avoid academic underachievement.
Section snippets
Development of the HLS items and an examination of content and expert validity
Phase One was based on professional experience of the authors (which included work on teacher’s impressions of handwriting legibility), validated by a review of the literature. Five legibility criteria were established to form the basis of the new assessment tool, the Handwriting Legibility Scale (HLS). The five criteria were: global legibility, effort required to read the script, layout on the page, letter formation and alterations to the writing. Instructions for scoring emphasized gaining an
Data analysis
A PCA was conducted to determine the factor structure of the HLS and t-tests were used to examine gender effects in the item and total scores. The discriminative analysis was undertaken in three ways; firstly by comparing the HLS scores of the children with and without DCD using a Mann Whitney U test. Secondly a Chi-squared analysis was undertaken on the HLS category scores (low, medium and high) to determine whether the number of children falling into each category was significantly different
Content and expert validity
Independent feedback was received from each of the 12 experts, based on their knowledge of handwriting, use of other assessment tools and practical experience of working with children. After rating three scripts using the HLS the experts completed a feedback sheet. All considered the criteria to be clear, although some commented that it would be helpful to have further examples where the HLS had been applied to different scripts, to help those new to the scale understand the separate criteria.
Discussion
Some children have difficulty using a pen to accurately form letters and words, making their handwriting slow to produce, untidy and hard to read. This may be related to a physical difficulty, poor motor control and coordination or other learning disorders. Alternatively it may be the case that the child has had inadequate teaching of the basics of writing and/or lack of dedicated handwriting practice. Whatever the underlying reason, poor legibility may lead to reduced motivation to write, low
Conclusions
The HLS provides an easy to use global score of handwriting legibility. Initial examination suggests good internal consistency and construct validity of the tool. However, further refinement of instructions is needed to improve the reliability of scoring and additional data collection and analysis to confirm a cut off point to accurately identify those with poor legibility. The HLS may then be a useful tool to identify those with poor legibility, to quantify levels of performance and to help
Funding
The collection of normative data on the DASH was supported by Harcourt Assessment (now Pearson). Other aspects of the work were supported by a studentship from Oxford Brookes University to the second author.
Acknowledgement
We are grateful to the children and families who took part in this research and to the experts and raters who assisted in the development of the new tool.
References (30)
- et al.
On the development of a computer-based handwriting assessment tool to objectively quantify handwriting proficiency in children
Computer Methods and Programs in Biomedicine
(2011) - et al.
An examination of writing pauses in the handwriting of children with Developmental Coordination Disorder
Research in Developmental Disabilities
(2014) - et al.
Handwriting process and product characteristics of children diagnosed with developmental coordination disorder
Human Movement Science
(2008) DSM-5. Diagnostic and Statistical Manual of Mental disorders
(2013)- et al.
Handwriting policy and practice in English primary schools
(2006) - et al.
Detailed Assessment of Speed of Handwriting
(2007) - et al.
Preventing written expression disabilities through early and continuing assessment and intervention for handwriting and/or spelling problems: Research into practice
- et al.
Teaching spelling and composition alone and together: Implications for the simple view of writing
Journal of Educational Psychology
(2002) - et al.
European Academy of Childhood Disability (EACD): Recommendations on the definition, diagnosis and intervention of developmental coordination disorder (long version)
Developmental Medicine and Child Neurology
(2012) - et al.
Setting cutoff scores: Legal, psychometric, and professional issues and guidelines
Personnel Psychology
(1988)