Make the criteria explicit when several people must judge alike

Where more than one person evaluates writing against the same standard, their evaluations should be consistent, and shared strategies for evaluating and shared characteristics of a good paper serve that end.

Why consistency matters

Evaluations should be consistent, so that each paper is judged fairly against similar papers. ¹ Consistency is especially important at the Massachusetts Institute of Technology (MIT), where a course may have more than 600 students and a dozen members of the writing staff. ¹

Dividing the grading

In some of MIT's technical courses in science or engineering, the technical staff of professors and teaching assistants assign the papers and grade them for technical content, and the writing staff grade them for use of language and adherence to standard formats and styles. ¹

Strategies for evaluating consistently

The writing staff at MIT use various combinations of six strategies to evaluate the writing in student papers. ² The strategies aim to make commenting on and grading the papers more efficient, and to make many graders more consistent, within large courses and from year to year. ²

1. Teach students how to write a good paper before they start writing; the better their first drafts are, the easier the papers are to comment on and the more the students learn from revising.
2. Educate the writing staff about what each letter grade means. Define the grades by describing the extent of revision required before a paper is publishable, by describing the characteristics of typical papers at each grade, by analysing the strengths and weaknesses of model papers at each grade, and by sorting mistakes into those that lower the grade only slightly, such as from a B to a B-, and those that lower it by a full letter, such as from a B to a C.
3. Review one another's comments and grades on individual papers; having new staff members read comments from experienced staff members is especially useful.
4. Use a course-specific form, either a general rubric or a more specific, detailed checklist; for some courses, using such a form rather than writing individualised comments on papers can save time and standardise the responses.
5. Communicate with one another. It helps for the writing staff to communicate about typical comments and the requirements for each letter grade before, during and after grading, through staff meetings and email.
6. Hold communal grading sessions, in which papers are graded independently by two graders and significant discrepancies are resolved by a senior staff person.

^{2, 3}

Characteristics of a good paper

Besides defining the letter grades, the writing staff at MIT consider ten characteristics in grading papers. ⁴ The list is not exhaustive, but it addresses the most common and most serious problems. ⁴

1. Clarity. The content must be expressed clearly. It should also be true, but truth is determined by the professors and teaching assistants teaching the course.
2. Coherence. The argument of the paper must be well developed and convincing, and the content must be presented logically from old to new information with no appreciable gaps. "Splat prose", in which information is present but disorganised, is not acceptable.
3. Concision. The prose must be reasonably concise and efficient, with sufficient density of information.
4. Correct grammar. The prose must be written in standard, formal English with correct syntax, diction, punctuation, spelling, capitalisation and use of standard idioms.
5. Professional tone. The prose must be formal and professional, not chatty, casual or slangy. It must be understandable by someone who is not familiar with MIT, a particular course or a particular laboratory project, and it must focus on information, not on the student's personal experience. The paper must be complete as submitted.
6. Self-contained graphics. Tables and figures must be legible, prepared in ink, adequately labelled and understandable without the text.
7. Required content. Each section of the paper must contain the required content, as defined by professional styles and course assignments. For example, the introduction of a full technical report must contain a problem statement that is clear, concise and explicit.
8. Standard format. The paper must follow the appropriate standardised format for its type, such as a proposal, primary research report or full technical report.
9. Adherence to style guide. The paper must follow the style guide designated by the professor in charge of the course.
10. Citations. Students are expected to know what constitutes plagiarism and to avoid it. They must give credit for everything they did not generate themselves, including graphics they copy from the course handout.

^{4, 5}

Using the process beyond the classroom

Reviewing or editing professional manuscripts for publication has similarities with evaluating student papers for pedagogic purposes, so some strategies and criteria are useful in both contexts. ¹ MIT's writing program reports that the six strategies and ten characteristics have been useful to it, and they may also be useful in editing or reviewing professional papers for publication. ⁶ The writing staff's process can be used as is or modified as appropriate for various settings. ¹ The ten characteristics can inform self-editing and the development of a personal style sheet. ⁶ The whole process can be used during peer review or collaborative writing, with reviewers or some of the collaborators commenting on and "grading" a paper to provide the author with specific feedback about the most commonly problematic issues. ⁶ The goal may be to improve the quality of the paper, to teach the author, whether oneself or another, how to write better, to teach everyone involved in the process how to write better, or any combination of these. ⁶