Examining Writing: Research and Practice in Assessing Second Language Writing

Stuart Shaw and Cyril Weir. Cambridge: Cambridge University Press, 2007. Pp. xiv + 344.

Reviewed by Dr. Ruth Breeze
Institute of Modern Languages, University of Navarra
Pamplona, Spain

For professionals interested in the principles and technicalities of assessing second language writing, Examining Writing offers a key to best practice in this complex area. Intended as a high-level academic statement, it explains the thinking behind the way writing is tested by one of the most prestigious English language examining boards, Cambridge ESOL. Although the volume focuses exclusively on Cambridge exams, its theoretical discussions and ample bibliography are also of interest to anyone involved in testing or researching writing.
Examining Writing begins with an account of the socio-cognitive validity framework. To understand recent developments in testing writing, the text helps readers to comprehend the fundamental underlying construct, which is intended to take cognizance of the interaction between writer traits (communicative language ability, cognitive processes) and context of use (i.e. tasks), rather than proceeding directly from traits to score. Expressed in more general terms, this means that tests have to take in the social dimension of writing, which has been one of the main areas of interest in writing research and pedagogy over the last 30 years. For examination purposes, social aspects, such as writer-reader relations, genre and cultural conventions, have to be built into the task itself. After a detailed explanation of this model, the authors examine each of its components in turn. Chapter 2, on test-taker characteristics, explains Cambridge ESOL’s approach to fairness and discusses policies for dealing with some types of permanent disabilities that might affect candidates’ performance. Chapter 3 centers on cognitive validity, that is how closely an examination writing task mirrors the cognitive processing involved in writing in real life. The authors show how they incorporate social issues of readership, goals and genre, as part of the writing process at levels from KET (A2) to CPE (C2).
Chapter 4 turns to context validity, understood as the linguistic and content demands that have to be met in order to write the text successfully. In the authors’ view, tests should resemble the performance conditions of real-world contexts. Chapter 5 examines the construct of scoring validity, providing an overview of holistic and analytic rating scales, examiner training, and performance. Chapter 6 looks at the washback effects of testing writing, but focuses mainly on IELTS and CPE and says little about the effects of mainstream examinations such as PET and FCE on classroom practice.
The final two chapters discuss comparisons and correlations between IELTS, Cambridge ESOL, and the Common European Framework of Reference for Languages and underline the nature of the book as a theoretically-based explanation of Cambridge ESOL’s practice.
From the perspective of a teacher interested in the theory and practice of second language writing, the book offers an abundance of stimulating material. Among other aspects, it contains a series of detailed case studies (unfortunately not listed on the contents page) which focus on specific issues within the area of testing writing such as the development of a common rating scale (to take in all the CEFR levels) and a lexical analysis of the exams from KET to CPE. At times, however, the book’s identity as an account of Cambridge practice leads to curious incongruities such as the chapter on context validity which takes us from Roman Jacobson’s theory of linguistics and poetics through to storage of exam packets in reinforced metal cabinets. The persistent references throughout the book to the shortcomings of the CEFR, such as “there is nothing in the CEFR about response format” (p. 66) or the complaint that the CEFR only defines texts as short or long, without giving actual word length (p. 81), also seem somewhat out of place. Nevertheless, despite these irregularities Examining Writing is surely essential reading for anyone involved in serious language assessment.