John Myers
Ontario Institute for Studies in Education of the University of Toronto
Even the best teacher education programs are notoriously short and barely adequate for preparing students to teach in today's schools. At the Ontario Institute for Studies in Education of the University of Toronto (OISE/UT) several projects have been undertaken in the use of sources in history classrooms (see Ruth Sandwell's article in this issue). This article describes the origins, features and challenges from an ongoing project exploring the use of sources in the history classroom.
Over the past few years, students in my history education methods class have been looking at the use and interpretation of sources in an effort to influence history teaching that relies far too much on textbooks and regurgitation and too little on critical interpretation of the evidence. I shall not go over the ground Ruth Sandwell examines in her article about the authentic nature of source analysis. Instead I wish to add two additional reasons for engaging in this work.
In many jurisdictions such as the U.K. and Ontario literacy is being seen as a neglected goal of the curriculum. As a result there have been a number of initiatives to improve student reading, writing, and oral levels, including the increasing use of provincial and national tests.
History content presents many challenges in comprehension. Since this area of the curriculum is highly literate and concept rich, it may be that student difficulties arise from the nature of its language demands, especially those reading demands made upon students by teachers, textbooks, tests and examinations (Myers, 1999). Reading critically and for meaning is seen as increasingly basic, not just in history but across the curriculum (Wineburg and Martin, 2004).
It seemed that more emphasis on using documents in the classroom would assist students in improving literacy and provide further justification for the place of history in the school curriculum, in addition to promoting critical thinking and all of those exciting things we try to do.
Assessment and evaluation are contentious issues these days. In many cases, the assessment tail wags the curriculum dog. Assessment shadows everything we do: from the student question, "Does this count?" to the editorials decrying our failure to educate the young.
As more traditional views of assessment such as standardized tests on a provincial or state level are re-emerging and as teachers find newer ideas harder to implement than first assumed, contrasting trends in assessment and evaluation now compete for prominence. These include:
As a result this project developed as a way to introduce our teacher-candidates to literacy, the nature of history as a discipline, and both traditional and new forms of classroom assessment as an efficient and effective combination.
From the primary grades on we have always used primary sources: pictures, artifacts, maps, written and oral accounts. In the context of sound teaching the use of documents can help students consider multiple perspectives, reconcile different positions, evaluate the strength of competing arguments, and promote deeper levels of thinking through the development of critical skills and sound habits of mind.
We have been less successful in using these sources in assessments and evaluations. In North America, document-based questions (DBQs) used to be considered appropriate only for senior high students in IB or AP programs, though the British have been using "sources" for decades. Now there is a move to use them to bring more authenticity to instruction and assessment. The Begbie Contest in British Columbia has used document-based questions since 1994 (see http://www.begbiecontestsociety.org/ for sample items) and a number of states such as New York have brought them into prominence in their Regents Exams (see http://www.emsc.nysed.gov/ciai/dbq/ssindex.html for two short courses for teachers wishing to use documents for assessment purposes). Canadian provinces such as Québec are using DBQs in their provincial examinations.
What follows is the assignment instructions undertaken by 56 teacher-candidates in two history curriculum methods classes at OISE/UT from November, 2003 to February, 2004.
Rationale: One of the current trends in assessment and evaluation in history is a revival of the use of traditional test items combined with the use of primary source documents. Well-designed DBQs have the potential to promote higher order thinking and enhanced literacy combined with rigor and interest for students.
Components:
Evaluation Criteria
Weighting: 15% of the course grade
Additional Notes to Teacher-Candidates:
- All components will be modeled in class
- There will be time in class for working on this assignment
- The teams you form now may be used in the major assignment of the second term, so learning to work together now will be advantageous
- Scoring your essays using a rubric will be done in January
- Please hand in two copies. I shall mark one and have the other for use by your classmates in either class. This way, we shall develop a bank of quality DBQs
- While students in applied level classes or grades 7-8 may work with fewer documents, the choice of appropriate sources for an essay item at this level may be more challenging than the design of "academic" or senior DBQs using more documents.
Here are samples of the work based on the curriculum in our grade 10 Canadian History: 1900-2000 course. The multiple choice samples were designed for a quiz on the 1920s. The essay sample come from a World War One unit for this same course.
Sample #1
"Do you feel justified in holding a job which could be filled by a man who has not only himself to support, but a wife and family as well? Think it over."
Based on the quote, the moral code of the 1920s could be best described as:
Sample #2
Based on the quote by the Superintendent-General of Indian Affairs to his agents in 1921, what best describes Canada's policy towards First Nations peoples?
"I have…to direct you to use your utmost endeavors to dissuade the Indians from excessive indulgence in the practice of dancing. You should suppress any dances which cause waste of time, interference with the occupations of the Indians, unsettle them for serious work, injure their health, or engorge them to sloth or idleness."
Sample #3
"Did the events of World War 1 help to unify Canada and contribute to a sense of Canadian identity?"
Document-based essay question:
"After returning from a trip to Europe, Prime Minister Borden became concerned that a system of voluntary enlistment would not be sufficient for a victory in Europe. Evaluate Robert Borden's decision to implement conscription. Was it a wise decision on his part? Support your answer by making references from the documentation provided."
(The teacher candidates then offered six documents including quotes, texts, photographs and charts.)
In addition to the discussions we had in our classes we received feedback on the project from Dr. Ruth Sandwell at OISE/UT, a social studies professor at Niagara University familiar with the New York state use of DBQs, and three teachers from Quebec, Ontario and British Columbia who had experience in using documents in their classrooms. The B.C. teacher was Charles Hou, the originator of the Begbie Contest. The teacher-candidates found the assignment to be as interesting as it was challenging. Their own feedback matched that of the outside reviewers.
Our goal in designing the Multiple Choice Questions was simple: to design items that required students to read the documents in order to determine the correct responses. This was a challenge and even with major revisions we had many multiple choice questions that still needed better wording or did not require examination of the document to be answered. The outside feedback complimented us on our efforts while pointing out the difficulties of sound multiple choice question design. The teacher-candidates commented on the surprising challenges of the multiple-choice format: surprising given the surface simplicity of a multiple choice question. Several groups had begun by looking at questions they had used during the recent practicum experiences in schools, but quickly found that the items they designed asked for recall only.
Designing DBQ essays requiring an analysis and interpretation based at least in part on the specific documents was easier. Still the timelines were pretty tight and most of the teacher-candidates went to the internet for sources. Some of these were not adequately checked and a number of historical errors crept in to the DBQs. In other cases documents were selected that were simply too long, did not provide a good fit with the question or were attached to questions that were vague or wordy. In addition, teacher-candidates found the search for appropriate documents to be frustrating given the tight timelines. While they also recognized the value of working in teams, some teams were more cohesive than others.
The construction of the rubric was also a challenge though the work of the Begbie Contest helped us in our thinking a great deal.
(adapted from the work of the Begbie Contest, op cit.)
"The purpose of a DBQ essay response is to test your ability to analyze and interpret historical documents and to write an essay. To complete this task successfully you should consider the following steps:
Use information from as many of the documents as possible in order to provide evidence for your position. While you must also hand in your outlines and notes, only the essay will be marked."
We chose to use four levels to match current rubric-based assessment practice in Ontario. To present to students an image of quality from the outset, we began our leveling at the highest level by asking, "What does a quality response look like?"
The Ontario Achievement Charts are the bases for determining final grades for a course. These are designed to broaden what teachers assess based on:
K/U= knowledge and understanding
T/I= thinking and inquiry
C= communication
A= application
Thus, each criterion in our DBQ Essay Rubric was matched with one or more of the achievement chart categories. We did not attach a grade range to our levels though the inference is that these fall in line with Ontario practice.
Based on our work we designed the following rubric:
| Criteria | level 4 outstanding |
level 3 good |
level 2 needs improvement |
level 1 poor |
|---|---|---|---|---|
| Clarity of thesis and conclusion How well is the position stated and summarized?(C, T/I) |
Both the thesis and conclusions are clearly stated with key criteria for judgment stated | Both the thesis and conclusions are clearly stated. Criteria for judgement is implied but not explicitly stated. | Thesis and conclusion are clearly evident though only one of these is clearly stated. Criteria implied. | Thesis and conclusion are evident but neither is clearly stated. Criteria for judgment is implied |
| Organization of argument How well are the thesis, body of the argument and conclusion linked?(C, T/I) |
The essay is easy to read as the argument is clearly linked to thesis and conclusion through consistent and appropriate use of connectives; e.g., "but," "and," , "because of", "moreover", "yet", "since" "therefore". | The argument is linked to the thesis and conclusion though the reader needed to read carefully to see the links as connectives not always used consistently or appropriately. | The essay is not easy to follow and needed at least a good second or third reading to see the connections among data supporting an argument. | Either the thesis or the conclusion did not clearly link to the arguments made. Spelling and grammar errors too few to detract further from the ease of understanding of the argument. |
| Use of evidence How is data from the documents used as evidence?(K/U, T/I, A) |
Data from required documents incorporated into interpretation along with other data. Data from competing interpretations is weighed and conclusions drawn based on criteria. | Data from required documents incorporated into interpretation along with other data. There is an attempt to consider data from competinginterpretations. | Data from required documents incorporated into interpretation along with other data. Competing interpretations not acknowledged. | Data from required documents interpreted though little data from outside sources evidenced and there is no attempt to acknowledge data supporting competing interpretations. |
| Additional criterion based on task. |
None of the criticisms expressed during the DBQ Project took away from the quality of the work or the view held by the candidates and the outside reviewers that this project was and is worthwhile. A number of candidates modified this work during their second practicum period and were pleased with the results. The school associates were also very interested in this work and wanted copies of the entire project. A colleague from another university, Dr. Marianne Larsen from Trent, is currently doing a similar project based on this original work.
It is important yet challenging for history teachers entering the field to have a sense of what is possible and to strive to be the best they can be while recognizing that there is so much to learn.
We (and this definitely includes myself) have learned a great deal from the first year of this work.. A number of students and professors doing similar work attested to the challenges of question design (Niagara University teacher candidates, personal communication on Portfolio Night, Dec. 6, 2004; Larsen, personal communication, Jan. 26, 2005).
This year we have just finished doing this assignment for the second time. The examples and experiences from the first year result in better work judging from the lower level of anxiety by teacher candidates and the higher grades I gave.
While the title of this article refers to the concept of "authenticity", we have some reservations about how this has applied to our project to date: reservations shared recently by Grant, Gradwell, and Cimbricz (2004). In response to these reservations we shall be incorporating the use of documents into our work on performance assessment based on the Critical Challenges model of inquiry (Case and Wright, 1997). Perhaps in a year we shall be able to report on this phase of the project.
Case, R., and Wright, I. 1999. Taking seriously the teaching of critical thinking. In The Canadian Anthology of Social Studies, edited by R. Case and P. Clark, 179-193. Vancouver: Pacific Educational Press.
Grant, S. G., Gradwell, J. M., and Cimbricz, S.K. 2004. A Question of Authenticity: The Document-Based Question as an Assessment of Students' Knowledge of History. Journal of Curriculum and Supervision 19 (4/Summer): 309-337.
Myers, J. 2004. Assessment and Evaluation in Social Studies Classrooms: A Question of Balance. In Challenges and Prospects for Canadian Social Studies, edited by A. Sears and I. Wright, 290-301. Vancouver: Pacific Educational Press.
Myers, J. 1999. Literacy Is Everyone's Business. Rapport 20 (3/Summer): 17-21.
Sandwell, R. in press. The Great Unsolved Mysteries of Canadian History: Using a web-based archives to teach history. (submitted to Canadian Social Studies).
Wineburg, S. S. and Martin, D. 2004. Reading and Rewriting History. Educational Leadership 62 (1/September): 42-45.
John Myers is a Curriculum Instructor in the Teacher Education Program at the Ontario Institute for Studies in Education of the University of Toronto. He can be reached by email at jmyers@oise.utoronto.ca.