Abstract
In the
spring of 2004, as part of the rejuvenation and restructuring of Inje
University, the idea of an English Campus, in which the University (a
private university in southeastern Korea) would become a campus devoted
to the instruction and use of English throughout the curriculum, was
created. The idea, like many ideas for growth and development in tertiary
institutions was put on hold until an opportune moment for implementation
arose. In August of that year, several committees were established with
goals designed to create a fully functioning English University in a
non-native environment. Of these committees, there was one entitled,
The High Class English Committee whose function was to increase the
students TOEFL® TSE and TWE scores by a factor of one. The results
of this five-week trial project for this committee are hereby recounted.
Overall, trends showed an increase of 1.5 points for the written component
with the speech component scores increased by one.
Keywords: TOEFL, TSE, TWE, Writing, Conversation, Testing
1 Introduction
Upon his induction in early 2004, the new university president established
the goal of making Inje University a functioning English campus in the
truest meaning of the words. His goal was to take a mid-sized local
private university and make it equal in stature to larger national universities
as a means of improving university visibility, thereby attracting a
larger number of students to the school from an already dwindling national
student population. To do this all English language faculty, were gathered
and requested to offer suggestions, which the president could act on
resulting in reaching goal achievement. Later that fall, at the end
of the semester break, the president returned to the university and
with the assistance of a newly appointed head of the English Campus
project, created seven committees, which would form the nucleus for
this objective. The seven committees were; 1) the TOEIC® Project,
2) The TOEFL®/GRE Project, 3) The High Class English Project, 4)
The Mileage Project, 5) the Department Centered English Project, 6)
The English Events Project and 7) the International exchange committee.
While each committee had the overall objective of increasing English
proficiency throughout the university, they also had their individual
objectives. The objective of this report is to describe the establishment,
development, and results of the High Class English Committee.
2 Methods
2.1) Student Selection
Upon receiving the directives for the High Class English Committee,
departmental secretaries were contacted and asked to obtain the TOEIC®
scores for all Inje university students. As per the requirements of
the directive, all students with scores below 700 on their TOEIC®
score were excluded from the initial data list, producing a list of
365 students across all majors.
Since Inje
university is a private medical university, there is a preponderance
of high TOEIC® scores in medically associated fields. These medical
students; however, were not selected for the trial project and therefore
were excluded from the trial project. This resulted in a final list
of 211 students with TOEIC® scores in excess of 700 for the trial
project.
These students
were contacted through their departments and requested to come for an
interview, during which they were presented with an outline of the project
goals, and their role in the project. Due to the nature of the project,
students were then asked to volunteer for the project. Students, who
chose to volunteer, were asked to sign a letter of consent for the purpose
of research in order to release their data to the researcher for research
purposes.
Within
the sample group, students came from a cross section of all university
departments. No one major was found to dominate the study group, possibly
indicating a large degree of student interest in increasing their scores
on standardized language tests presently used in EFL in Korea and also
in the project goals.
2.2)
Materials Selection
The primary researcher then compiled a list of materials for the project,
which were designed for increasing students speaking and writing proficiencies
on the TOEFL® test.
Project
duration was established at five weeks, with each volunteer student
being required to attend two conversation classes and one writing class
beyond their normal allotment of English classes. Topics for the conversation
classes were taken from a series of TOEFL® test guides, based on
applicability and relevance to the TOEFL® test and to any life situation
that the students might possibly encounter. Comparatively, topics for
the writing class were also taken from a series of TOEFL® texts,
with their subjectivity based on test and/or daily relevance.
Each student
was required to attend a minimum of two conversation lectures a week
and suggestions were made such that they attend lectures by more than
one instructor with the express purpose of increasing the variability
the student encounters with respect to language patterns.
Alternatively,
students were assigned to a specific instructor for the writing assignments.
This was to ensure consistency in grading and evaluation of skill development
throughout the project.
2.3)
Evaluation
The evaluation of the students utilized the following methodologies.
2.3.1)
Ongoing skill Evaluation
This took two individual forms, each based on the respective skill group
being observed and measured.
2.3.1.1) Conversation
Conversation when compared to grammar and writing is a more flexible
medium and as such can only be measured subjectively, while the others
are more capable of being measured objectively. Therefore, the conversation
classes utilized ongoing, anecdotal comments as a means of measuring
an individuals' progress through the designed conversational curricula.
The comments
consisted of notes on every students' miscues, language misuse, inaccurate
structures, and sentence developments. Students were monitored throughout
the conversation and recasts were utilized to correct the construct
errors. Recasts were given at the end of each conversation class, to
not impede the flow of the discourse event. The instructor also functioned
in keeping the conversation focused and wrote notes throughout the conversation
class.
2.3.2.2) Writing
Correcting the weekly timed essays was the means for evaluating the
students writing skill development. As a means of measuring skill development,
each assignment was corrected and evaluated for progress relative to
the previous weeks work. As well, writing was evaluated for clarity,
conciseness, accuracy, logic, and argument structure.
Corrected
essays were photocopied and then returned to the students for evaluation
and assimilation of corrected information in order to improve writing
skills over the duration of the project. As with the conversation classes,
all data obtained from the written essays was collated by the primary
researcher and retained for future purposes.
2.4)
Logistical control
Students, after volunteering were randomly allocated to one of five
instructing professors for the writing class. They, along with the instructor,
chose a time during which to meet as a group for the writing class.
Each writing class was of one hour's duration.
The first
thirty minutes of each writing class were designated for essay writing,
while the second thirty were set aside for discussion of the corrected
essays. There was a one-week delay between essay completion and essay
discussion so the instructor could mark the essays.
Students
were required to attend two hours of conversation every week. The choice
of instructor was left to the student as long as they participated in
the mandated two hours per week. To determine accuracy of the attendance
for this phase, sign-in sheets were utilized in every conversation class.
These sheets were then submitted to the primary researcher at the end
of every week to assess attendance to the conversation aspect of the
project.
If a student
missed a conversation period, they were contacted and warned regarding
the delinquency. However, if they missed a writing class they had to
report to the primary researcher and present a valid reason for the
lack of attendance. If the reason was accepted, they would then be required
to make it up at that point. However, if the reason was not valid the
student would be presented with the option of discontinuing with the
study and then designated as a DNC (Did not complete) variable for the
purpose of project evaluation.
2.5)
Curricula and texts
The materials used for the project came from an assortment of TOEFL®
texts. Topics used for both the conversation and timed essays primarily
came from the following texts; TOEFL® 185 essay topics 2003 edition,
the Nexus Introduction to TOEFL® writing (with accompanying 185
essay topic guide), The CBT TOEFL® start up text and Interactions
2 - writing texts. Information used for the conversation classes, with
respect to grammatical structures and vocabulary used, came from, CBT
TOEFL® Grammar, and the Nexus TOEFL® Vocabulary and Grammar
texts.
2.5.1)
Writing
The TOEFL® 185 essay topics, 2003 edition is a series of 185 graded
essays organized and ranked on level of difficulty and occurrence in
the TOEFL® exam. Essay topics are presented as a question followed
by a sample answer and the Korean translation of the question and answer.
Students were only given the question; all other information was deleted
for the writing class. The instructor, as a method for correcting the
timed essays, used the sample essays from the text.
In the
Nexus Introduction to TOEFL® writing, topics are presented in a
linear method. This method is based on essay construction. Each unit
is designed around the construction of a section of the essay, hence
focusing on the actual development and construction of the argument.
This text is accompanied by a smaller guide, which contains 185 selected
essay topics.
The CBT
TOEFL® startup text is a computer based TOEFL® test for the
measurement of listening, structure, and reading skills. However, some
of the topics used in the text were used for conversation and writing
topics.
Lastly,
Interactions 2 - writing, a text for writing instruction was used to
obtain a more varied group of writing and conversation topics, that
were not in the TOEFL® list, for the purpose of obtaining variety
in the project and giving the students the chance to explore the use
of the language skills they were being given from the other texts.
2.5.2)
Conversation
Currently, the TSE (Test of Spoken English) is not a major component
of the TOEFL® exam; however, future guidelines indicate that this
will change. Therefore, in anticipation of this change the students
were required to show an ability to converse freely in English. To do
this the following texts were source materials. The CBT TOEFL® grammar
text was used as a source to assess grammatical structures in speech.
This was accompanied by the use of the NEXUS TOEFL® vocabulary text
to facilitate increased vocabulary development. The Korean system has
built an inherent dependency upon the TOEIC® test; however, the
TOEFL® approach measures a different vocabulary and hence having
a spoken, contextual understanding of this alternative vocabulary was
deemed necessary for the project.
3 Results
Initial student placement encountered some logistical trouble when trying
to match student schedules with professors' schedules. However, once
this difficulty was rectified, students were required to attend a writing
development class before beginning writing. This class was used to instruct
the students on the basics of writing a structured essay. Upon completion
of this class, students were assigned their first timed essay topic,
which varied from professor to professor.
Through
the duration of the project, attendance attrition resulted in seven
students withdrawing from the project due to overloaded student schedules.
This form of attrition was expected and was factored in during project
development. In the final count for students, fifty-five students partook
in the project for its duration. However, again due to the nature of
project timing this attendance was not perfect. For those students who
did attend the following results were obtained.
3.1.1
Writing
In the course of the abbreviated duration of the project, students were
required to write five timed essays. These essays were corrected and
returned to the students one week later. Upon the return of the essays,
students were given the chance to ask questions regarding the corrections
and the reasons for the errors, as a means to increase their performance
for the following week.
Scores
for the assigned essays in the first week ranged from a low of 0.5 to
a high of 4, on a scale of 6 as accepted by the TOEFL® marking methodology.
Over the duration of the trial program scores improved by 1.5 points
on average, using the 6-point scale. In the final week of the trial,
project the scores ranged from 1.5 to 4.5 on the 6-point scale.
For the
first week, the mean of the test scores was 1 with a distribution from
0.5-4 and a standard deviation of 0.5. Upon termination of the project,
the mean score was 2 with a distribution from 1.5 to 4.5 and a standard
deviation of 1.0.
The trial
project goal was to increase the students' average score by 1 point
on the TOEFL® exam. Results show that the average score for the
trial group increased by a score of 1.5, thus surpassing the objective
of the project. This level of increase while meeting the project goals
is only partly indicative of the actual results.
3.1.2
Speaking
As previously mentioned, development of the speaking skills is harder
and more subjectively measured. This creates problems in objectifying
a score for students. Hence, the speaking component was measured by
observations of each student's individual development and how well they
could discuss the topics and answer the questions presented to them
by the professor and the other students.
This was
achieved by getting the students to deal with each separate component
of the speaking component for the TOEFL® exam. The exam has four
major components. A discussion component, whereby the student presents
a fact based opinion. A slide component in which the student is presented
with a series of pictures for which they have to create a plausible
story. A graph component in which they describe a graph and lastly,
a flow chart component in which they must describe the processes outlined
by the flow chart.
Each component,
therefore, is a function of descriptive speech. The goal then was to
develop student's ability to describe accurately what they see or think
when presented with a topic. Obtaining materials that would develop
each student's descriptive skills achieved this.
Grading
them towards this achievement; however, was not as easily achieved.
Students were graded on how well they did on each assignment, but this
grading scheme was far more subjective than objective.
4 Discussion
Students on average increased their scores; however, as the test grade
point increases, the marking criteria become far stricter in the requirements
to obtain a given score. Therefore, any increases made by the students
were in the lower range of the TOEFL® criteria guidelines.
Increases
in students writing scores were more readily identifiable. Those increases
ranged from 1.5 to 4.5 with an average of 2, thus identifying an average
increase of 1.5. On the TOEFL® scale of 6 this is a relative increase
of 25%, which, while it might not appear to be a great increase it is
substantial in light of the initial levels of the students. This is
especially true when looked at with respect to the 5-week trial period.
While increases
in the students speaking skills, were harder to measure, development
of the students' relative descriptive abilities did improve. Anecdotal
results showed that students' abilities went from the basic description
to partial analytical description over the 5-week period as well. This;
however, did require greater interaction between the instructors and
students in order to be realized.
For the
speaking component students began by describing their responses to written
situations in a discussion format. They went on to look at and describe
short packets of information obtained from storyboards, graphs, and
flowcharts. Most students at the end of the trial period were managing
to do this in a functional manner, while some did not manage to do so.
In light of this, the project in the future will be reworked to provide
more reinforcement to develop students' verbal skills.
While the
overall results do not show great progress, relative to the required
goals of the TOEFL® exam, they do indicate that measurable progress
is possible over the short term. Progress achieved; however, is only
at the initial level. To go beyond this would require a far greater
level of commitment on the part of the students and the professors who
contributed. Since the project guidelines only called for a limited
increase of 1 point on the TOEFL® grading criteria, it was possible
to reach the goal; however, if the goal were to reach one of the higher
grade categories according to the TOEFL® guidelines, it would be
a far more tedious goal to reach.
To reach
the higher skill levels designated by the TOEFL® testing criteria,
would require, as previously mentioned, a far greater level of commitment
from both the students and the professors involved. As well, this would
require that the students be taught the skill sets necessary to reach
those higher levels, namely grammar, vocabulary, and comprehension skills.
The current project, due to the severe time constraint, did not incorporate
the instruction of these skills, not to mention, that the instruction
of these skills, was under the auspices of another committee.
Those students
who took part in the project fell into the upper strata of students
at the University. This was by design, having been preselected by the
committee as having a TOEIC® score exceeding 700. While this limits
the veracity of the project with respect to bias, it does identify those
students who are most likely to take the TOEFL® exam as a means
of furthering their academic careers.
5 Conclusions
The approach used to develop Korean students' writing skills worked
to limited degree. Students made progress in skill development; however,
the attained levels were only at the lower to middle level of the TOEFL®
grading criterion. This does not mean students are incapable of attaining
the higher levels, just that to do so they need a more balanced approach
with respect to contiguous instruction in order to reach a higher level
of understanding.
This trial
project while it did reach the prescribed trial goals was not given
the necessary time (in terms of instruction) to grant the students the
ability for full development. To facilitate students reaching higher
TOEFL® criterion levels, greater instruction time is necessary on
the part of both the instructors and the students.
Comparatively,
writing development was not as easily definable, partly due to the nature
of instruction of speech skills. A more advanced level of skill development
is necessary to obtain these TOEFL® criterions. Reaching the advanced
levels (again a level of 6 is highest) is much harder for the average
student. While no statistical data are available to accurately define
the veracity of the anecdotal observations, speech and descriptive skills
did improve through the duration of the trial project; they were just
not as easily scored.
Overall
students who participated in the trial project would receive better
scores on the TOEFL® after working in the project. Increases; however,
that would be offset by missing information which is needed to do well
in the whole test; grammar, comprehension, reading, and listening. Students
who took part in this committee did not necessarily take part in the
other TOEFL® committee project, which was designed to develop grammar
and comprehension skills. If students, who took part in one committee,
took part in the other, then their overall TOEFL® scores would increase
making it far more valuable as a project course, since by having all
component scores increase is better than having only one component score
increase.
To accurately
measure a students progress it would have been better to have the students
take the TOEFL® test, as was originally designed into the project;
however, due to financial constraints, both on the part of the students
and the school, this option was removed from the project goals. For
the future, it has been requested that the students be given the capability
to take the TOEFL® exam as a means of determining the accuracy of
the project results. At the time of writing, this condition had not
been met for the current project duration.
While a
preselected data source, in this case, students, is effective in obtaining
results, it does mask any overall trends by creating bias in the data
set. To accurately verify this trial a more cross-sectional approach
is deemed viable in the future.
Lastly,
while teaching to a test is helpful, it is not always the best method.
That is, teaching towards a test, helps a student pass the test, but
it does not help them learn what they are doing in order to pass the
test. Teaching towards the test can have both positive and negative
washback effects (Lynch 2003). Lynch cites Messick (1996) regarding
the negative washback effect in which teaching to the test stresses
inauthentic language performance. Lynch (2003) also goes on to cite
the positive washback effect, of teaching to the test, as approximating
genuine communication identified by Berhardt (1991), Kitao and Kitao
(1996), O'Malley and Valdez Pierce (1996). Messick (1996), also cited
in Lynch (2003), goes on to clarify that the link between testing and
washback is not that clear. Good teachers can offset this problem of
formatted instruction by modeling their instruction in a way that creates
communicative learning, but that is not always possible depending upon
the material required for the student to do well on the test.
6 References
Bernhardt,
E. B. (1991). Reading development in a second language: theoretical,
empirical, & classroom perspectives. Norwood, New Jersey: Ablex
Publishing Corporation.
Ji, Eun-Ee,
and Ju, So-Yeon. 2002. Bak Jeong CBT TOEFL® Start-up Series.
Eul Yoo Moon Hwa sa. Seoul. Korea 291pp.
Ji, Eun-Ee,
Min Sang-Hung, Pyeo, Naen-Ee, Im, Jun-Hyun, and Pyeo, Naen-Gut. 2004.
Strategies for TOEFL® Writing. Nexus Books. Seoul Korea.
315pp.
Ji, Eun-Ee,
Min Sang-Hung, Pyeo, Naen-Ee, Im, Jun-Hyun, and Pyeo, Naen-Gut. 2004.
TOEFL® Vocabulary: the Successful Toeflers' Choice. Nexus
Books. Seoul Korea. 384pp.
Ji, Eun-Ee,
Min Sang-Hung, Pyeo, Naen-Ee, Im, Jun-Hyun, and Pyeo, Naen-Gut. 2004.
TOEFL® Grammar. Nexus Books. Seoul Korea. 400pp.
Kim, Song-Su.
2003. Lingua TOEFL® 185 Essay Topics. Lingua Forma. Seoul
Korea. 449pp.
Kitao,
S. K. & Kitao, K. (1996). Testing communicative competence. The
Internet TESL Journal, 2(5), [On-Line]. Retrieved August 17, 2001,
from http://aitech.ac.jp/~iteslj
Lynch,
Richard (2003). Authentic, Performance-Based Assessment in ESL/EFL Reading
Instruction, The Asian EFL Journal, [on-line]. Retrieved Mar
28, 2005, from http://www.asian-efl-journal.com/dec_03_sub.rl.php
Messick,
S. (1996). Validity and washback in language testing. Language Testing,
13(3), 241- 256.
O'Malley,
J. M. & Valdez Pierce, L. (1996). Authentic Assessment for English
Language Learners. New York: Addison Wesley.
Phillips,
Deborah. 2001. Longman Complete Course for the TOEFL® Test: Preparation
for the Computer and Paper Tests (With answer key). Pearson Education
NY, NY. 676pp.
Yoon, Ji-Won,
Byun, Gi-Seung, and Jeon, Hong-Seok. 2001. Bak Jeong CBT TOEFL®
Start-up Series. Eul Yoo Moon Hwa sa. Seoul Korea 605pp.
