Make It Stick Action Plan

How to put Make It Stick into action, by Peter Brown (April 2019).

  1. The science of learning:
    1. Retrieving learning from memory helps make it stick; reviewing does not.
      1. Why? The mental effort to recall strengthens the new material’s connections in the brain and makes it easier to recall again later.
      2. Re-reading and other forms of review do not help learning stick.
    2. Desirable difficulties in learning are ones that help to...
      1. Create understanding of new material
      2. Strengthen connections in memory
      3. Create cues to recall it later

      Examples:

      1. Trying before being taught how (to solve a problem; answer a question; etc.), then getting corrective feedback.
      2. Elaborating to create meaning. (“Why?” “What if?” “How does it fit what I already know?”).
      3. Spacing out learning & retrieval attempts.
      4. Mixing up practice of problem types.
    3. Intuition leads students to low-value strategies that feel productive but are not.
      1. Re-reading, massed practice, and practice that’s blocked by problem type create illusions of mastery.
      2. Spaced and mixed practice at recalling and applying learning are more effective but less often chosen because the added difficulty is interpreted as “I’m not getting it.”
      3. Students should be asked to demonstrate learning. Frequent low-stakes quizzing helps students discover what they do and do not know, and helps lock-in learning and carry it forward.
  2. What to do:
    1. Teach students the science of learning.
    2. Model effective strategies in class and homework:
      1. Coach students about illusions of knowing, and how to use elaboration and self-quizzing to become stronger learners and to reveal what they don’t yet know.
      2. Assign active-learning exercises that help students create their own understanding of new material (get ideas from peers, and from Peter Brown’s PowerPoint slides).
      3. Space topics over the term of the course (i.e. circle back as new material is covered, so that older material is refreshed and connected).
      4. Mix up practice problems in class and in homework instead of blocking by type.
      5. Quiz often (reach back to earlier material to help students carry it forward).
      6. Make exams cumulative.
    3. Be transparent about why you are using these strategies, and encourage students to adopt them in all of their courses.

Helping Students Make it Stick

Article by Peter C. Brown, Author of Make it Stick: The Science of Successful Learning from UND newsletter On Teaching, Fall 2015, Vol. 25 (3)

The most effective strategies for making learning stick are far from what most of us do as professors and students. Professors commonly focus on lecturing and explaining. Students lean on underlining, highlighting and rereading notes and texts (the preferred study strategies of more than 80% of college students). In other words, intuition tells us to try to drive new learning into the brain, but a large body of empirical evidence shows that trying to get new learning out of the brain is what builds the robust neural pathways that make the learning stick. Mental effort builds mental ability.

Cognitive psychologists Robert and Elizabeth Bjork at UCLA have coined the phrase “desirable difficulties” to characterize strategies that require the kinds of active mental engagement that make learning deeper and more durable. Among these are retrieving learning from memory; spacing and mixing practice sessions when studying (mixing up topics rather than cramming); testing oneself to practice retrieving information; trying to generate the answer to an unfamiliar problem before being taught the solution, followed by corrective feedback and instruction; and elaborating on new material to give it meaning by putting it in one’s own words, connecting it to what one already knows, and perhaps constructing mental cues to help recall the knowledge later.

How can such strategies be brought into the classroom without creating untenable demands on professors? Henry Roediger at Washington University at St. Louis, whose research helps to inform the new science of learning, says that for starters professors can trade off some of their lecture time to engage students in grappling with class material. Below are short recaps of how three professors have made room for desirable difficulties in their classrooms.

Mary Pat Wenderoth, biology, University of Washington, Seattle

In a Biology 220 course, Wenderoth and a colleague switched from what she calls a “low-structure” format (midterms and a final) to a “high-structure” format, with daily reading quizzes and weekly practice exams.

Wenderoth says this change was most challenging in the first year because of the extra time to write the practice exams. The best source was old exam questions. Wenderoth asks students to grade their own answers (to increase metacognition) and then either the instructor, lab coordinator or lead TA checks the answers. This approach gives the instructor insight into how students are thinking about topics and gives students practice in writing out answers.

Wenderoth says that her biggest change to class structure has been to prioritize the information. She cites Grant Wiggins’ and Jay McTighe’s book Understanding by Design. “They talk about 3 levels of information: enduring understanding, important to know and do, worth being familiar with.” To make time in her classes for students to actually work with and solve problems, Wenderoth has moved the information “worth being familiar with” either to the reading quiz or greatly minimized it in class. “Isolated pieces of information that fit into this latter category are seldom retained, and Google is a ready source.”

Wenderoth says, “The biggest change to my teaching is that I approach each class with the mantra ‘Ask. Don’t tell.’ I basically use the same class notes/outlines and just use them in a different way. I figure out how to ask students questions to guide them into constructing their own understanding of the topic.

“So time-wise, creating reading quizzes takes time the first time through, weekly practice exams are just old exam questions. Reviewing student answers can take time if the instructor does it rather than the lead TA. Prep for class just means I have to think about what students struggle with and how to ask the right questions to guide their understanding. This last one takes time to get good at, but with practice even the instructor gets better and teaching becomes much more fun.”

Andrew Sobel, political economics, Washington University at St. Louis

Sobel teaches a big lecture course in international political economics to freshmen and sophomores. The class meets 26 times over 13 weeks. When he learned about the benefits of retrieval practice as a learning strategy, Sobel instituted 9 quizzes spaced across the semester. He tells the students which days the quizzes will be given, so there are no surprises.

(Sobel had tried a regime of pop quizzes some years earlier, thinking it might create an incentive for better attendance, but had to abandon them. When students did poorly in a quiz they dropped the course rather than risk a bad grade, and class reviews and enrollment plummeted.)

To accommodate the quizzes without sacrificing more lecture time than he was willing to give up, Sobel dropped the midterm and final exams. Every quiz covers recent material and also reaches back to help lock-in material from earlier in the semester and connect it to subsequent learning. The quizzes count for 90% of a student’s grade, and the last 10% is at Sobel’s discretion.

Five years into this new format he said, “The quality of discussions in class has gone way up. I see that big a difference in their written work, just by going from three exams to nine quizzes.” By the end of the semester student mastery is comparable to what he’s seeing in his upper division classes. Meanwhile, the predictable quizzing schedule has been accepted and even embraced by students -- class enrollment has grown from 165 to 185 and counting.

“The interesting thing about adopting this strategy is I now recognize that as good a teacher as I might think I am, my teaching is only a component of their learning, and how I structure it has a lot to do with it, maybe even more.”

Kathleen McDermott, psychology, Washington University at St. Louis

McDermott administers daily low-stakes quizzes in a university class on human learning and memory. It’s a class of 25 students that meets twice a week for 14 weeks, minus midterms and a final exam.

McDermott gives a 4-item quiz in the last 3-5 minutes of every class. The questions hit the high points of the lecture and/or the readings. If students have understood the material, they will get all 4 answers right, but they’ll have to think in order to do it. Anything covered in the course to-date is fair game for a quiz, and she will sometimes draw from past material that she feels the students haven’t fully grasped and need to review.

To accommodate unexpected absences and keep demands on McDermott’s time manageable, students are allowed to drop four quizzes across the semester. In exchange, absences need not be justified, and no missed quizzes will be made up.

By the end of the semester, her students say that the quizzes have helped them keep up with the course and discover when they are getting off track and need to bone up. “The key with quizzes is to establish very clear ground rules for the student, and make them manageable for the professor,” McDermott says. “As a student, you’re either there and you take it, or you’re not. For the professor, no hassling over makeup tests.”

The quizzes in totality count for 20% of a student’s grade in the course. In addition, she gives two midterm exams and a final. The last two exams are cumulative, reinforcing learning by requiring students to engage in spaced review.

[Adapted from Make it Stick, The Science of Successful Learning, by Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel, Harvard University Press.]

“Man Sentence” Elaboration Demonstration

Demo for Elaborative Interrogation, from Mark McDaniel.

(This explanation is for the instructor. It can be discussed with the students after the demonstration. The purpose of this exercise is for students to experience the difficulty of recalling arbitrary information (rote study), and to show how elaboration that gives meaning to new information makes it easier to recall later.)

  1. First, introduce the exercise to students:

    This is a test of your ability to remember a dozen simple actions. For example:

              “The cold man bought grass seed.”
              “The pale man changed his shirt.”

    I will read 12 such statements, and then give a memory quiz. Do not take notes. Just listen and memorize. 

  2. Present the rote-study sentences.

    (Read one sentence at a time, lingering for a few seconds, and instruct students to try to learn the sentences without making written notes. If you want a more striking effect you could instruct students to repeat the sentences to themselves.)

    1. The short man bought the broom

    2. The brave man gave the money to the robber

    3. The fat man read the sign

    4. The tall man bought the crackers

    5. The thin man found the scissors

    6. The rich man picked up the chair

    7. The dying man used a feather

    8. The kind man ate dinner

    9. The bald man used the phone

    10. The frightened man ironed the sheet

    11. The dishonest man looked closely at the wrapper

    12. The smart man went to work

  3. Next, give the test (without a delay).

    (Ask students to take a blank sheet of paper and make a list numbered 1-12, where they will write their answers as you ask the test questions. NOTE: The test questions below are in random order.)

    1. Who found the scissors?

    2. Who used the phone?

    3. Who bought the broom?

    4. Who read the sign?

    5. Who looked closely at the wrapper?

    6. Who ironed the sheet?

    7. Who gave the money to the robber?

    8. Who used a feather?

    9. Who went to work?

    10. Who ate dinner?

    11. Who picked up the chair?

    12. Who bought the crackers?

  4. Read the students the answers and ask them to score their responses:

    1. The thin man found the scissors

    2. The bald man used the phone

    3. The short man bought the broom.

    4. The fat man read the sign

    5. The dishonest man looked closely at the wrapper

    6. The frightened man ironed the sheet

    7. The brave man gave the money to the robber

    8. The dying man used a feather

    9. The smart man went to work

    10. The kind man ate dinner

    11. The rich man picked up the chair

    12. The tall man bought the crackers

  5. Move on to the elaboration-study test.

    Explain that there will be a new list of arbitrary actions. For each of these actions, ask the students to elaborate by imagining a reason for the action. For example:

              The hungry man got into the car ...
                   to go to the restaurant.

              The brave man ran into the house ...
                   to save the boy from the fire.

  6. Present the elaboration-study sentences: 

    1. The sad man looked at his new boat

    2. The artistic man put down the knife

    3. The sleepy man bought the mug

    4. The evil man wound up the clock

    5. The blind man hit the flea

    6. The bearded man threw out the coupon

    7. The one-legged man flicked the switch

    8. The religious man used the saw

    9. The long-haired man looked for the pole

    10. The Irish man counted the leaves

    11. The weak man thanked the check-out girl

    12. The patriotic man memorized the words

  7. Give the next test, without delay: 

    1. Who used the saw?

    2. Who put down the knife?

    3. Who memorized the words?

    4. Who bought the mug?

    5. Who threw out the coupon?

    6. Who looked at his new boat?

    7. Who hit the flea?

    8. Who counted the leaves?

    9. Who looked for the pole?

    10. Who flicked the switch?

    11. Who thanked the check-out girl?

    12. Who wound up the clock?

  8. Read the students the answers and ask them to score their responses:

    1. The religious man used the saw

    2. The artistic man put down the knife

    3. The patriotic man memorized the words

    4. The sleepy man bought the mug

    5. The bearded man threw out the coupon

    6. The sad man looked at his new boat

    7. The blind man hit the flea

    8. The Irish man counted the leaves

    9. The long-haired man looked for the pole

    10. The one-legged man flicked the switch

    11. The weak man thanked the check-out girl

    12. The evil man wound up the clock

  9. Discuss the demonstration.

    Poll the students’ scores. Did they perform better in one test than the other?

    Using elaboration during study (for example, creating meaning or associating the new material with retrieval clues like visual images) is shown to help later recall. Did they experience this themselves?

SUGGESTED READING

Pressley, M., McDaniel, M. A., Turnure, J. E., Wood, E., & Ahmad, M. (1987). Generation and precision of elaboration: Effects on intentional and incidental learning. Journal of Experimental Psychology: Learning, Memory, and Cognition13, 291-300.