25 Years: What (Good) Teaching Looks Like

Editors’ note: This is the second of a series of blog posts on the state and history of the University of Michigan’s undergraduate mathematics program. Calculus reform came to Michigan in the early 1990s—closing in on 25 years ago. So this is another installment in a “25 year retrospective.”

“…Don’t you realize now, what you see is me
Tell me what you see.”
–The Beatles

Observing Teaching

Observations of teaching may occur in many contexts: in a new instructor training program, in support of reference letter preparation, in the evaluation or review of an instructor, to provide other instructors insight on different teaching styles, and more. In our Introductory Program courses, by which we mean here our course before calculus, calculus I, and calculus II, we have regularly done classroom observations for new instructors as part of our instructor training and support program. The manner in which these have been done has changed slightly through the years, but overall retains the goals and general structure that it had when Michigan started with “reform” calculus in the 1990s.

Our observations of teaching include class observations and feedback visits. In both cases the primary goal is to provide formative evaluation, that is, to promote the instructor’s improvement as a teacher. We describe each of these below, provide some of the historical background of how they have been used, and give some of the logistical and administrative details connected with their use.

Types of Visits

In both class observations and feedback visits an observer visits an instructor’s classroom. In the former they observe only, while in the latter they both observe and solicit feedback from the students in the class.

Class Observations

Our classroom observations are initiated by the observer, who suggests a couple of possible days for the observation to the instructor, and they agree then on a mutually agreeable class for the observer to visit. Who is doing the observation has changed over time; currently, observers are course coordinators, program directors, graduate student co-coordinators, and/or experienced graduate students with experience equivalent to that of a graduate student course co-coordinator. On the day of the observation, the observer arrives and sits somewhere in the classroom that is ostensibly out of the way, and observes the full class period. The observer takes notes on what the instructor and students are doing at different times in the class period, where the instructor is, who is asking questions and who is answering them, what the instructor writes on the board, significant points or events in the class, etc.

Frequently, the observation notes include a column (or page; in Figure 1, the all caps text) showing what the instructor wrote on the blackboard; a column (or page) highlighting what they say and interactions with and between students (in Figure 1, the lower-case writing); and a map showing where students are in the class, which of them are contributing in different manners, and the instructor’s movement between them. An example of (part of) a set of observation notes is shown in Figure 1, and an example of a map of an instructor’s movement in the classroom during an observation is shown in Figure 2. Of particular interest in the observation are characteristics of the class such as: who is doing the mathematics (is the instructor lecturing, or are students actively engaged with the material?), who is asking questions (the instructor, a handful of students, or many of them?), and who is answering them (the instructor, one or two students, many other students, or group members?).

Following the observed class, the observer will review their notes and thoughts on the observation, and formulate a short summary to be given to the instructor. This summary will include information about the class that was observed, specific aspects of the course that were positive (“what is going well”), and comments on how the lesson could have been improved (“suggestions”). Formulating this summary generally takes the observer on the order of an hour. The observer then meets with the instructor to review how the class went, go over the summary and other observations from the observer, and answer the instructor’s specific questions about that class and teaching in general. This meeting also usually takes on the order of an hour.

Feedback Visits

Feedback visits were developed by Beverly Black at the University of Michigan[1], who at the time worked with the University’s Center for Research on Learning and Teaching (CRLT)[2]. The initial portion of the visit is similar to that of a classroom observation, with the observer arranging a time to visit and taking notes while attending the class. However, in the last 20 minutes or so of class the instructor leaves the room and the observer solicits specific student feedback from the class about what is going well and what could be improved. This is done in a structured manner.

First, the observer introduces themself, notes that all comments that are gathered will be anonymous (the observer will type up responses), and that the point of the visit is for the instructor to gain students’ feedback on what they can do to improve students’ learning for the rest of the semester.

Following the introduction, a feedback form (Figure 3) is distributed to groups of four students. The instructions are to have a recorder in the group write down comments that all members of the group agree on, in two categories: major strengths of the course; and changes that could be made to improve the learning environment. Students are encouraged to be as specific and practical as possible, providing concrete examples where they can.

After students have had the opportunity to formulate their thoughts, the observer moderates a full-class discussion and writes on the board suggestions in both categories on which there is a consensus. In some cases, a comment may be written on the board with an indication of the degree to which there was some disagreement with the statement.

At the end of class, the observer records (or takes a picture) of the full-class comments, and erases the board. After the visit, the observer prepares a summary document similar to that created for a classroom observation, indicating all of the comments (both positive and constructive) on which there was consensus. Where appropriate, and with annotation, comments that did not have uniform support may be included as well. As with the class observation, formulating this summary generally takes on the order of an hour.

Once the summary is created, the observer meets with the instructor and discusses the summary, answers questions, and provides closure on the visit. This meeting also usually takes about an hour.

History

Pat Shure, who directed the Introductory Program as it was reformed in the 1990s, reports that new instructors were observed as they started teaching from the start of our calculus reform, and possibly before[3]. Starting with the implementation of calculus reform, observers were both faculty in the department and staff from CRLT, and the observations were feedback visits which were tied in part to the evaluation of the revised courses[4]. CRLT did a large number of these visits until about 2004, when the workload associated with supporting them proved to be in excess of their resources (though they continue to have a program that provides a class observation for any faculty member requesting one). Since then, the Department has done all class observations for new instructors in mathematics.

In the mid-2000s, all new instructors in the Introductory Program had two class visits from the course coordinators, co-coordinators, and supporting faculty in the Program[5]. The author’s recollection is that one of these was a classroom observation while the other was a feedback visit (this is consistent with CRLT having provided feedback visits as our reform calculus was implemented). In 2007, for example, there were 38 new graduate student instructors and 5–6 new post-doctoral and visiting faculty who were each visited twice, for a total of about 90 visits. These were done by seven observers, who were course coordinators and co-coordinators, and three other faculty associated with the Introductory Program.

Providing this number of visits became unsustainable without additional resources, however: the visits in 2007 required an average of about 20 hours a semester from each of the observers for the class visits alone (our Introductory Program courses meet for 80 minute class periods)—and a total of on the order of 40 hours once time to generate summaries and meet with instructors was included. For lack of additional observers, by the end of the first decade of the 2000s new instructors were getting a single classroom observation visit, a practice that continues to the present. In cases where the classroom observation or other data suggest that there are challenges in a new instructor’s classroom a second observation will be done, and when there are issues in an instructor’s first semester of teaching another classroom observation is done early in their second semester of teaching.

Since 2015, all Introductory Program mathematics courses have been taught in sections of 18 students, which has significantly increased the number of new instructors requiring class observations. To be able to address the additional observations that are required, observers now include the course coordinators and co-coordinators (eight faculty and graduate students in the fall semester) and several additional experienced graduate students who are hired to assist with the observations.

As noted above, CRLT still supports feedback visits for instructors who request them, which are available in addition to these Department observations.

The Logistics of Observations

Managing the logistics of the classroom observations is a task unto itself. Historically the Director of our Introductory Program has assigned observers to instructors and determined when observations should start. The observations are done early in the semester—so that the feedback from the observation can be used by the instructor to improve their teaching. The Director additionally works with our administrative office to hire the non-coordinator observers, and is responsible for training all observers. The training of new observers involves the Director meeting individually with each observer, explaining the logistics and expectations for the visits, summaries and instructor meetings. As these observers have been observed themselves when they were starting their teaching in the Department, they know about the general structure of the program from an instructor’s perspective.

The observers are then responsible for scheduling their visits, writing up their summary, and meeting with the instructors to discuss the observation. When an observer determines that there are potential issues in a classroom, they follow-up with the Course Coordinator and Director to ensure that a second visit or other appropriate action is taken. The Department also maintains a simple database system to keep track of the outcome of the observations. This allows observers to enter general descriptive information about the visit (how well logistics were managed—board work, class control, etc.), as well as comments and suggestions that the observer provided the instructor. This is primarily to allow the Director access to these data without having to find a given observer.

Practical Considerations

There are a number of aspects to this type of large-scale observation program that do not fall in the category of logistics, per se, but are practical considerations that merit note. These include the management of instructors’ concerns and students’ expectations.

In many parts of our instructor support, we find that our goal of formative evaluation, providing feedback to (allow instructors to) improve, is often clearer to those who run the program than it is to those who are new instructors in the program. Throughout the training program there is a tendency for instructors to view the evaluation as summative, grading them to possible ill effect. Accordingly, we work to communicate to instructors that the observations are not an evaluation of their teaching, and that they instead serve to help them improve their teaching. This is bolstered by the fact that instructors uniformly find the observation process helpful.

Management of expectations is also important for feedback visits, in which students are being asked for their opinion on the instruction in their section. In this case there are many things that are specific characteristics of the course which students may otherwise feel the instructor is imposing on them. For example, our Introductory Program courses include team homework that is worked on by groups of four students, and the courses are conceptual and require significant work and thought. These are things at which students frequently bridle, but are beyond the instructor’s control. Therefore, we often begin the discussion of the feedback portion of such a visit with a framing of the course and what instructors do and do not have the authority to change.

Other Viewpoints

In addition to the classroom observations that we do for all new instructors in our Introductory Program, we provide a second feedback mechanism that provides some of the same information afforded by the feedback visits. At about the midpoint of the semester, after the first midterm exam, we provide to each class-section an on-line survey that includes the same prompts and questions as the University administered teaching evaluation that is given at the end of the semester. The feedback from this is, for each instructor, reviewed by the coordinator of the instructor’s course before being released to the instructor. In those rare cases in which the student feedback suggests significant issues in a class section, the coordinator will meet with the instructor to provide insight and feedback on the nature of the comments and suggestions for changes the instructor should make. Similarly, the responses to the midterm evaluations may suggest an additional classroom observation should be scheduled for the instructor.

What Good Teaching Looks Like

It has been clear from the start of our “reformed” calculus program that the training and support we provide for our (new) instructors are essential for its continued success. Our new instructor training is a topic unto itself, and runs for the week before our classes start in the fall. However, it is clear that training before the semester without additional support is insufficient to sustain the forward instructional momentum that we aim to initiate when training.

What does good teaching look like? In our classrooms, it is active and noisy, with an instructor who is providing their students with similar guidance to that which we seek to provide the instructors. Good teaching looks like good learning: participants are actively engaged with their subject (or subjects), doing work to understand it (or them), and struggling productively with what is necessarily an ongoing and difficult task. In the vernacular of our School of Education, this struggle with teaching and learning what we need to know to teach better is also mathematics.

References

1. Black, B. 2015. Personal communication.
2. CRLT. http://crlt.umich.edu/. Accessed 31 May, 2018.
3. Shure, P. 2018. Personal communication.
4. Brown, M., A. Taylor, & P. Shure. 1992. A New Calculus Program at the University of Michigan. NSF Proposal. DUE-9252503.
5. Personal recollection: the author has a spreadsheet scheduling two visits for each new instructor in 2007.