Problem Roulette

try-it-out-ps

An awesome exam study service for students.

Problem Roulette (PR) is a cloud-based service offering topical access to a library of past exam problems at the University of Michigan.  Since 2013, PR has served over 4 million problems to more than 10,000 students across 8 introductory courses.

Self-directed study and assessment in a realistic framework.

Timed examinations are a mainstay of learning assessment in large enrollment courses.  Exam day is like Judgment Day, a source of major stress for students.  The set of problems prepared by the instructor typically probe multiple facets of learning in the discipline.  Students must therefore juggle a variety of principles, techniques and methodologies to construct a correct answer, all within a fixed time limit.

Problem Roulette helps students prepare for exams by providing random access to past multiple-choice exam problems in a topically-structured environment.  PR provides instant feedback to students, allowing them to assess their problem solving accuracy and speed relative to their peers.

Equal access for all students.

A traditional mode of preparation is for instructors to post a small number of past exams to a course management system such as Canvas.  Over time, the scarce resource of old exams become prized commodities, motivating certain student groups to develop rich libraries of past exams for added advantage.  By providing access to thousands of old exam problems in a particular discipline to all students, PR serves to level these social imbalances.

Student testimonials

The refreshing thing about PR was that I could actually practice all the formulas and concepts I had learned in class on new problems; no longer relying on memorization, the concepts actually stuck with me at exam time.


The features I enjoyed most which I think made Problem Roulette superior to other forms of studying were the results on the average time students took to answer the problem and the percentage of students who got it correct. That allowed me to better evaluate my readiness for the exam and as someone who tends to take too long on each individual problem it allowed me to work more on my timing so that by the time the exam came around I could finish in the allotted time.


It allowed for a different approach to studying that didn’t involve a specific number of questions to complete, or time constrains. In this way, a lot of the pressure was relieved, which allowed me to move slightly faster along the problems.

How it works

A student authenticates to a U-M server (1) and, after choosing a course/topic pair, receives a random problem pulled from the target library (2-4). Student submission of an answer (5) triggers updates to the problem and student databases (6). The server provides feedback to the student (7-8).


Grade points earned in Physics 240 are shown separately for regular (blue circles) and for light/non (red triangles) users of the PR service. Students are partitioned into four bins based on beginning of term GPA. Points are plotted at the mean GPA value of each subset, and error bars show the standard deviation of the mean grade. Linear fits to the mean measures are shown. Regular PR users are offset at a level of approximately 0.23 in final grade relative to light/non users.

Grade points earned in Physics 240 are shown separately for regular (blue circles) and for light/non (red triangles) users of the PR service. Students are partitioned into four bins based on beginning of term GPA. Points are plotted at the mean GPA value of each subset, and error bars show the standard deviation of the mean grade. Linear fits to the mean measures are shown. Regular PR users are offset at a level of approximately 0.23 in final grade relative to light/non users.


engaging-problems-PR_screenshotThis image shows the view a student would have when working on Problem Roulette. After working out the problem, the student selects one of the answers shown on the top of the page. Upon submission, the student receives feedback on the problem.


This image shows the view a student would have after submitting a problem. After submitting an answer, the student receives feedback including the correct answer, solve time, average user solve time, and a distribution of the response rates for each answer choice. This image shows the view a student would have after submitting a problem. After submitting an answer, the student receives feedback including the correct answer, solve time, average user solve time, and a distribution of the response rates for each answer choice.