The Effect(iveness) of Change – Michigan Math In Action

Over the course of last summer we revised our standard “sophomore-level” differential equations course, taken by most engineering students. This is the course which traditionally has been a recipe course in which one learns to categorize all of the different types of differential equations which can be solved by hand while glossing over the fact that most differential equations can’t. The revision was to change it to be a more conceptual and more relevant course; in the following we look at some of the history of the course, the goals of the revision, and the outcome of our efforts.

History

Our version of this course in the past appears to have been rooted in the traditional “recipe-driven” course, with evolutionary changes moving in the direction of a more conceptual, more demanding course. Resource constraints require that it be taught in a lecture of on the order of 100 students. The lecture meets three times a week for a standard 50 minute period. For very many years—since at least the mid-90s, and possibly earlier—there has been an extra hour a week associated with the course which has been in some weeks a computer lab and in others a standard recitation. The students in this course are 75–85% from the College of Engineering; the Department offers other courses in differential equations for other clientele (e.g., a course that assumes that students have had a course in linear algebra and introduction to proof writing).

$[image of classroom style computer lab]$
Computer Lab, pre-revision

The original computer labs for the course were developed in the early to mid-90s and centered around Euler’s method and some applications; they underwent a major redesign in the late 1990s.[1] Those were the basis for the labs that were in use for the ensuing 10–15 years—let it not be said that our work does not have an ongoing impact! Between then and 2016, the labs evolved slowly, and largely in the direction of requiring less student knowledge of Matlab (the package used for the labs, as all engineering students at the University learn and use it) and, arguably, less conceptual engagement. Students had five sessions they spent working on the (5) labs, and the remaining weeks met in recitation, in which expected recitation type activities (question answering, material clarification) took place.

Common wisdom is that the course as it was before revision was perceived by students as easy and, as it was straightforward, a “good course.” An unscientific survey of student comments on ratemyprofessor.com[2] bears this out, finding it characterized as “the easiest of the required engineering math classes.”[3] Feedback on the labs in teaching evaluations suggested that students saw them as little more than hoops to jump through that didn’t connect to the course as a whole. Recitations, not surprisingly, were regarded as much more helpful. One might suppose that it’s difficult not to like a space where one is told the answers one seeks.

Goals and Revision

At the end of the day, there is an increasing and persuasive body of evidence that indicates that if we are going to make a (mathematics) course effective, in the sense of engendering student learning, we must get students actively engaged with the material.[4,5,6] It is clear from the preceding discussion that the course was not necessarily doing a good job of this before its revision. Active engagement is difficult when the primary instruction takes place in a large lecture, and even the labs and recitations were not well structured to ensure active learning.

We had some advantages and some disadvantages when planning the revision to this course. It is coordinated by a course coordinator, who is able to set the textbook, syllabus, course materials and assignments, and exams. This allowed us to redefine the course as we would like—subject to maintaining support for the material needed by the engineering students who are the vast majority of those taking the course—which is a significant advantage. The two significant disadvantages are in its tradition of allowing instructors independence in the classroom and its format—logistically and politically we could not change it to a small-section course. The net impact of these constraints was that the lectures were likely to remain lectures, except insofar as individual instructors sought to bend them to be more interactive or engaging.

Accordingly, our work on the course focused on the course materials and lab sessions. Expectations may help define what students learn, and the lab sessions/recitation are the place where we could push greater interactivity and engagement.

$[image of collaborative computer lab]$
Computer Lab, post-revision

Further, we have one, very positive, data point supporting an approach to the course on this path. In 2014 we revised our Calculus III (multivariable and vector calculus; Stewart[7] chapters 12–16) course. This course is taught in the same format as this differential equations course, and our revision involved updating the computer labs in which the students worked on their labs from the static classroom rows shown above to an interactive environment shown to the right. At the same time the lab materials themselves were rewritten to be more substantial and conceptual and to require greater student collaboration. To assess the impact of this change we did pre-/post-testing with an internally written test measuring students’ understanding of the course material. Students’ pre- to post-test improvement in the course after we had implemented the revision was dramatically better than before. We will be the first to caution you not to believe any study with $n = 1$ , especially one that isn’t published, but, still, $n = 1 > 0$ .

Objectives of the Revision

Our goals in revising the course were:

to increase students’ engagement in the course,
to extend the material, and to make better connections between different course material within the course and between the the course material and that from other courses,
to improve students’ understanding of the connection between the labs and the course as a whole, and
to update the course to have a more “modern” approach to differential equations and have a greater emphasis on conceptual understanding.

The Revision: an Overview

These objectives, and the changes described below, were evaluated and approved by the College of Engineering’s Director of First Year Programs and their Undergraduate Program. To accomplish them, we revised the course syllabus and adopted a new textbook[8], and updated the assignments and labs. The book itself we found to be stronger mathematically than that which we had been using, and it takes a “systems first” approach, introducing systems of two first-order equations immediately after treating first-order differential equations. Completely new labs were written in the course of the summer of 2016 by a post-doc funded by a gift from MathWorks. The new labs were more demanding, strongly application based, and written to require that students work in pairs and fours to complete them. Other homework and exams were also revised to be more demanding and conceptual.

These changes were all implemented in fall 2016, with admittedly somewhat rough (“beta”) versions of the labs. As part of the implementation, beginning mid-semester we held a weekly meeting with the graduate students teaching the lab sections of the course, which proved invaluable for figuring out what was really going on there, what was working well, and where there were issues.

The New Labs

Each of the new labs has an application around which the work revolves.

Lab	Model	Mathematical Goals
1	Gompertz model for the size of a cancer tumor [e.g., 8, p.27]	Introduce series approximations to solutions of differential equations and the linearization of nonlinear equations by Taylor expansions.
2	The van der Pol oscillator, an active RLC circuit [e.g., 8, p.500–504]	Introduce systems of differential equations and phase plane analysis, linearization of nonlinear systems, and the differences between linear and nonlinear behavior.
3	A model for a ruby laser [9]	Introduce systems of more than two equations, examine the behavior of second-order equations and response to sinusoidal forcing.
4	A chemical equations model [10]	Explore numerical methods for the approximation of solutions and the effect of stiffness on approximation error.
5	The Lorenz equations [11]	Examine linearization and nonlinear behavior, explore bifurcations and see chaos.

Because it’s fun to see the equations that these involve, they are included below:

Lab	Model
1	$y' = r y \ln(K/y)$
2	$x' = y$ , $y' = -x + \mu y(1 - x^2)$
3	$N' = \gamma(A - N(1 + P))$ , $P' = P(N - 1)$
4	$x' = -ax + cyz$ , $y' = ax - cyz - by^2$ , $z' = by^2$
5	$x' = c(-x + y)$ , $y' = rx - y - xz$ , $z' = -bz + xy$

$[image of students working in lab]$
Work in Computer Lab

Each of the labs is structured to have a strong emphasis on collaborative, engaged work. They are completed on a two-week cycle. In the first week, students have a pre-lab due that requires them to work through some basic mathematics related to the lab. The goal of the pre-lab is to introduce the lab and give a context for it. In the lab period, students in pairs from a team of four to work on Matlab exercises that explore the concepts in the lab. The exercises worked on by the pairs are similar, but address slightly different, complementary, aspects of the material or problem that is being considered. The following week, students use the work that both pairs did as they continue with a second set of Matlab exercises. At the beginning of the lab period after the second workday the team submits a lab writeup that explores the mathematics and meaning of the work they did in the previous two weeks.

Other Changes

The biggest change to the course was in the restructuring of the labs. However, there were other changes, the most significant of which were in the course material and its organization and in the assignments and exams.

The course material pre- and post-revision is summarized below.

#	Topic (Pre)	Topic (Post)
1	first-order equations	first-order equations
2	higher-order linear, constant-coefficient equations	systems of two linear first-order equations, phase plane analysis
3	systems of linear first-order equations, phase plane analysis	second-order linear, constant-coefficient equations
4	systems of two autonomous non-linear first-order equations	Laplace transform techniques
5	Laplace transform techniques	systems of two autonomous non-linear first-order equations

The effect of the changes is really two-fold: it emphasizes phase plane analysis and a dynamical systems perspective when considering differential equations, and it means that material students find more difficult is more evenly distributed throughout the semester. The more difficult topics in the syllabus are 3–5 in the old syllabus, and 2 and 4 in the new one. The treatment of systems and the phase plane early in the new syllabus makes the analysis of nonlinear systems much less formidable. Both of the effects of these changes are very valuable, though the effect of the rearrangement of the material wasn’t fully appreciated until after we got into the first semester teaching with the new syllabus!

The second change, in assignments and exams, resulted in them being more conceptual and, demonstrably, more challenging. The number of written homework assignments was decreased (from 10–11 to 5), but the problems were made significantly more substantial. Problems on the two midterms and final reflected the more conceptual emphasis in the course, and were also significantly harder (the median of exam scores over the two semesters preceding revision was 80%; in the first semester post-revision, 61%).

Effect(iveness)

…[It] was like looking both ways before crossing the street and then getting hit from behind by an airplane.

Student evaluation

Perhaps the most plainly obvious result of the revision was that students didn’t like the course becoming more challenging (which it did). Instructors’ teaching evaluations almost certainly reflected this, and many comments on evaluations expressed concern about the demands of the course (now “…by far the most difficult” of required math classes for engineering[3]).

That said, while course expectations were significantly higher, students largely rose to them. Students’ perceptions of the lab materials, as measured in a survey at the end of the semester, were not positive—but we have no comparative data with the labs they replaced, and we suspect those would have been evaluated similarly or worse at helping students learn the material and having connection with the course material. That said, students reported that the labs improved their ability to use Matlab (41 responses out of 124 total), demonstrated connections to real-world applications (18 responses), aided in visualization of the studied mathematics (17 responses) and improved their ability to work in teams (7 responses). The graduate students teaching the labs felt that students generally understood the labs and the mathematics in them, though some students got hung up on the difficulty inherent to programming in a computer environment like Matlab.

We do not have a direct way of measuring student learning before and after implementation of the course revision, because the timeline of the revision didn’t allow doing any sort of assessment before we implemented the changes. However, we do have some measures by which we may assess the impact of these changes. There is no doubt that the new lab materials substantially increase students’ engagement with the material, as well as the amount of time they spend on collaborative work, both of which are essential for improved learning.[4,5] There are substantially better connections between the materials and the course, even if these need greater emphasis for students to see them. And it is clear that the use of applications and overall conceptual focus of the course increased.

Conclusions

So, was it all effective? It’s hard to determine the actual impact on student learning, which is the real goal of all that we are doing here. There was clear and loudly voiced concern from students that the course changed suddenly and for the worse (one student wrote on the final exam “You’ve ruined this course. I hope you’re happy!”).

But at the same time I think we have to have some faith in our understanding, as educators, of what we are doing. The inclusion of more engaged learning in any course must be a good thing, and an increasing focus on conceptual understanding is at the end of the day aligned with our sense of what is important.

Effect? We have effected more engaged learning and a better overall course focus. Is that effective? In the long run it must be.

References

[1] Charles Doering, personal communication 1/4/2017.
[2] Rate My Professors. <http://www.ratemyprofessors.com/>, accessed 2/8/2017.
[3] Peter Gavin LaRose at University of Michigan. (Rating post date 11/29/2016) <http://www.ratemyprofessors.com/ShowRatings.jsp?tid=156282>, accessed 2/8/2017.
[4] Freeman, S., et al. (2014). Active Learning Increases Student Performance in Science, Engineering and Mathematics. Proceedings of the National Academies of Sciences, 111(23):8410–8415.
[5] Kogan, M. & S.L. Laursen (2014). Assessing Long-term Effects of Inquiry-based Learning: A Case Study from College Mathematics. Innovative Higher Education, 39(3), 183-199 (published online August 13, 2013). DOI 10.1007/s10755-013-9269-9.
[6] Conference Board of the Mathematical Sciences (2016). Active Learning in Post-Secondary Mathematics Education. <http://www.cbmsweb.org/Statements/Active_Learning_Statement.pdf> (July 15, 2016), accessed 2/8/2017.
[7] Stewart, J. (2012) Calculus, 7th ed. Brooks/Cole/Cengage, Belmont, CA. ISBN 978-0-538-49787-9.
[8] Brannan, J.R. & W.E. Boyce (2015). Differential Equations, An Introduction to Modern Methods and Applications, 3rd ed. Wiley, Hoboken, NJ. ISBN 978-1-118-53177-8.
[9] Erneux, T. & P. Glorieux (2010). Laser Dynamics. Cambridge University
Press, Cambridge.
[10] Hairer, E. & G. Wanner (1996), Solving Ordinary Differential Equations II. Springer.
[11] Sparrow, C (1982). The Lorenz Equations: Bifurcations, Chaos, And Strange Attractors. Springer-Verlag, New York. ISBN 978-0-387-90775-8.