The Internet TESL Journal

Student-directed Assement in ESL/EFL: Designing Scoring Rubrics with Students

David Litz
UAE University (Al-Ain, United Arab Emirates)
davidralitz(at)yahoo.com

Introduction

This paper will discuss and provide a useful example of "negotiable contracting" (Stix, 1997), an innovative assessment approach in ESL whereby students are actively involved in the construction or creation of rubrics prior to undertaking more complex language learning tasks. It will be demonstrated that the art of negotiable contracting not only empowers students by giving them shared ownership of their own learning and assessment but it also provides them with additional opportunities to use the target language as they analyze and discuss the rubric construction process, examine the rubrics and their different levels, generate descriptive language from the rubric's criteria, and suggest revisions for improvement based on criteria in the rubric.

What Are Rubrics?

Scoring rubrics are becoming one of the most popular forms of authentic assessment and they are used by ESL/EFL practitioners in a variety of educational contexts. Essentially, a rubric consists of a fixed measurement scale and a set of criteria that are used to discriminate among different degrees of quality or levels of proficiency. They are intended as communication devices that precisely convey to students what their learning target is and what they need to do to reach that target (Maryland Assessment Consortium, 1995, p.1).

Rubrics can be either holistic or analytic.  Holistic rubrics provide an overall impression of the elements of quality in a student's work. They are utilized when minor errors in part of the process can be tolerated and in instances when tasks require students to create some sort of response but there is really no definitive, correct answer (Mertler, 2001; Nitko, 2001). Moreover, the score reported using a holistic rubric is on the overall quality, proficiency, or understanding of the content as this type of rubric involves assessment on a uni-dimensional level (Mertler, 2001).

Analytic rubrics indicate the level of performance of a student's work on two or more separate elements of quality. Nitko (2001) suggests they are preferred when a focused response is required and individualistic creativity is not expected to be a major feature of the students' responses.  They also result initially in several scores, followed by a summed total score. Their use represents assessment on a multi-dimensional level (Mertler, 2001).

Designing Scoring Rubrics with Students

For many years now, ESL/EFL teachers have been designing rubrics and providing them to their students and their widespread use is certainly a testament to their importance, relevance and effectiveness in modern ESL/EFL assessment. Nevertheless, authors such as Wiggins (1993), Seeley (1994), and Stix (1997) argue that constructing or drawing up rubrics jointly with students prior to them undertaking the related lesson or task provides an even richer, more authentic, learner-centered and empowering authentic assessment experience than merely providing them with a teacher-designed rubric. Stix (1997) refers to this rubric co-construction process as negotiable contracting and his research shows that students who are given a role in the assessment process and provided with the appropriate direction by their teachers are able to accurately evaluate their strengths and weaknesses and better pinpoint areas where they need to focus their efforts for improvement. As a result, students typically perform at higher levels and gradually come to view assessment not as an arbitrary form of reward or humiliation, but rather as a positive tool for educational enrichment and growth.

Additional research by Litz & Smith (2004) with adult ESL/EFL learners at a military training institution has shown that students who engaged in negotiable contracting also typically reported that they enjoyed having the opportunity to use and generate the target language in order to analyze and discuss the rubric construction process, examine the rubrics and their different levels, and suggest revisions for improvement based on criteria in the rubrics. Moreover, students were able to develop a clearer picture of the task and their teacher's expectations while the teachers reported that they had clearer instructional goals.

Empowering Students

The art of negotiable contracting or rubric co-construction in ESL/EFL requires the teacher to be responsible for grading but he/she must also act as a facilitator of a whole-class discussion on the assessment process. In this way the teacher presents his or her own expectations for the assigned work but also asks the students their opinion of what they think would constitute quality work. Essentially, the teacher and class negotiate with one another in an attempt to arrive at an acceptable mutual consensus. The end result is that the teacher has created an atmosphere of openness and accomplishment where students have had increased opportunities to use the target language in a communicative setting. In addition, students have been made to feel empowered for they are now valued participants in the assessment process and they can take pride in having created their own assessment tool.

Steps Involved in Negotiable Contracting

The act of rubric construction with ESL/EFL students is probably better suited to teachers and students that are familiar with the use of scoring rubrics in general.  It can be done with both lower level and higher level students but the degree of teacher facilitation and student involvement and the complexity of descriptive language that is produced for the descriptive criteria will obviously depend on the respective levels of the students and their familiarity with cooperative learning techniques (Litz & Smith, 2004).

A step-by-step guide to the process of designing rubrics in tandem with students is described below. Information was compiled from various sources such as Wiggins (1993), Mertler (2001), and Stix (1997). The steps will be discussed in the context of a real-world example of an ESL teacher who has assigned their high-beginner/low-intermediate class the task of writing an autobiographical essay of at least fifteen sentences.

Step 1:  Students Examine and Discuss Possible Samples of Work that Exemplify Each Level for the Task at Hand

Before the students begin any work on the task the teacher arranges the class into cooperative learning groups. As a warm-up the teacher provides an example of an excellent sample of work as well as a good one, fair one, and a poor one. The teacher then has each group of students decide what they thought were excellent, good, fair, and poor about each one.

A very good example might elicit responses from the students such as: A poor example, on the other hand, might elicit responses such as:
After each group submits their suggestions the teacher records them on chart-paper and posts them around the room for later use in the overall process.

Step 2:  Students Brainstorm Observable Attributes and Task Outcomes (Skills, Characteristics, and Behaviors That Will Be Expected)

During the second step the students are told they will decide on how the task (autobiographical essay) will be graded, and the procedure can be presented to the students in the following manner:

"I want you to be the teacher. What things would you look for in this assignment that would receive the best grade? Working in your groups, make a list of criteria that you as a teacher would use for grading purposes. We will examine them together and make a list that we will all agree on as the criteria of assessment for this assignment."

After allowing time for discussion, the teacher calls on one speaker at a time from each group. The spokesperson for the cooperative group submits one criterion to be listed on the board. The teacher utilizes active listening for this phase of the activity, meaning that students are required to listen to one another and check off each item if it has already been mentioned. This avoids wasting time and ensures that the same ideas are not being repeated. The teacher charts the various groups' criteria on the board. Examples for a short autobiographical essay might include: vocabulary and spelling, organization, design and layout, style, coherence and cohesion, grammar, originality, content, task fulfillment i.e. complete/incomplete etc. After all of the groups have submitted their ideas, the students can discuss them. If the teacher has a criterion that was not posted on the board, but is essential to a fair assessment of the project being assigned, the teacher should detail the item being added to the list, and explain why it is so important. After the students have exhausted all their ideas, they work in their cooperative groups and prioritize their top-five favorites. Once again, the teacher calls on the speaker for each group, having the student submit his or her group's first choice. The teacher charts the information, and again, utilizes active listening. As each item is posted on the board, the teacher asks how many other groups had that particular criterion on their top five listing. The teacher makes a check mark next to the criterion to note its priority among all the cooperative groups. Once the list is completed, the teacher checks which criteria were heavily weighted and which were not.

In this particular example the students and teacher selected:
It is recommended that only four or five criteria be selected to formulate the rubrics because the students should not feel threatened by a seemingly insurmountable task. If a teacher insists on using more than that number, it might overwhelm the students.

Step 3a:  Lower Level Students Examine Characteristics and Criteria that Describe Each of the Selected Attributes or Task Outcomes

During the third phase the teacher could use a 'Pre-existing Rubrics Bank for Teachers'. If time is short or if he/she has lower level students, the criteria can be easily cut and pasted to form the rubrics and descriptive levels and the students can be asked to complete the rubrics as part of a communicative jigsaw activity. In this type of activity the students are placed in groups and each group is provided with the different criteria and descriptive levels. Both the criteria as well as the descriptive levels are mixed up and the students work together to put them in the correct places in the rubric.

Step 3b:  Higher Level Students Brainstorm, Describe, and Develop Thorough Descriptive Criteria for All of the Observable Attributes and Task Outcomes

If time permits and the teacher feels as though the students are of the right ability, he/she can submit one criterion to each cooperative group so the students can help design the wording for the descriptive levels of the rubric. As an alternative the teacher could have each group circulate to different tables around the classroom and have the students work on criteria descriptors for each level. At this point in the lesson the students would be reminded of their descriptions of previous work samples and encouraged to use some or all of this language while developing their own level descriptors. Higher level students might be permitted the freedom to use a great deal of their own vocabulary for the linguistically-based descriptive scale listed from left to right across the top of the rubric chart. Lower level students might have less autonomy in this regard and might benefit more from increased assistance and elicitation from the teacher.

Conclusion

The use of negotiable contracting is an effective way of motivating and empowering our students through authentic or alternative assessment practices. Giving ESL/EFL learners a voice in their grading not only provides them with a sense of ownership and a clear understanding of the task, but it also enables them to practice the target language in a way that moves beyond the parameters of a typical classroom lesson or unit of study; and the importance and relevance of this cannot be stressed enough.

Figure:1 Student-generated Rubric

 

CRITERIA

LEVEL 1

LEVEL 2

LEVEL 3

LEVE

TASK
COMPLETION

 

Information was either missing, not needed or off task.

Task was completed, but could be better. Some information is needed and some was on-task.

Task was completed well.

Most of the information was needed and on-task.

Task was completed very well. All of the information was needed and on-task.

ORGANIZATION

Poor organization of ideas. No paragraphs and sentence marker errors.

Little organization of ideas. Paragraphs and sentence markers were used, but with some errors.

Good organization of ideas. Good use of paragraphs and sentence markers.

Excellent organization of ideas. Excellent use of paragraphs and sentence markers.

VOCABULARY & SPELLING

Poor use or range of vocabulary. Many spelling errors.

Adequate (fair) use of vocabulary. Some spelling errors persist.

Good use of vocabulary. A few spelling errors.

Excellent use of spelling. Very few spelling errors.

GRAMMAR & PUNCTUATUION

Poor use of grammar and punctuation. Many errors.

Adequate (fair) use of grammar and punctuation. Some errors still present.

Good use of grammar and punctuation. A few errors still present.

Excellent use of grammar and punctuation. Very few errors are present

References

The Internet TESL Journal, Vol. XIII, No. 11, November 2007
http://iteslj.org/
http://iteslj.org/Lessons/Litz-StudentDirectedAssessment.html