Formative Feedback

I've been thinking a lot about feedback lately. It started with this tweet:

@MrAbend I would be really curious as to what the Math learning/cognitive science research says about error types. Does @mpershan know?

— Anna Blinstein (@Borschtwithanna) April 15, 2017

@mpershan was kind enough to respond with an email and sent me down a rabbit hole of articles and blog posts about the usefulness of feedback. Since Michael was the inspiration for this journey, it's only fitting that I try to imitate his style of writing out loud to try to organize my thoughts on this topic (sorry, Michael - reading this back after I've finished the blog post has shown me that you are inimitable. Also, I should probably avoid writing blog posts at 11 pm in the future).

The central question we discussed was: what is the purpose of feedback? Clearly, it is only useful if it changes a person's thinking. Does pointing out a mistake do this? Does categorizing the mistake do this? Does indicating a student's level of understanding of a topic ala Standards-Based-Grading do this? Do questions do this better than statements? Do students need to reflect on the feedback or do another problem related to the feedback received or implement it in some other way in order to get more benefit from it? Written vs. oral? Immediate vs. delayed?

Feedback while kids are working in class:

This is the type of feedback I think I know how to do the best. When kids are working on a task, either on their own or with someone else, I am usually able to ask questions, point out features of their work, or connect them with other students' thinking so they can make progress, identify and correct errors, and clarify their own ideas. The one blind spot that I think I still have in this area is when a student thinks about a problem in a way that is really, really different from methods I understand or have seen and thought about before. This doesn't happen very often, but when it does, I'm really stumped. I can help them verify that their answer is incorrect. I can ignore their method and show them a way to think about the problem correctly or point them to another student in the class with a different approach. But if I don't understand it, I can't help them resolve the cognitive dissonance of their incorrect approach, which means that my work is not complete.

But in general, this is the type of feedback that seems to pay the most dividends. The kid is right there with their work, we can have a conversation, I can see if they are able to implement my feedback and give more or of a different kind, as needed, or ask them to work on a related problem. This is really the best case scenario in feedback world for me.

Feedback on homework:

Things start to get real hazy real quick when I'm looking at a kid's work outside of class and my feedback is now provided in written form or via a conversation with them the next day. Will they have time/inclination to do anything about my feedback? Without the option of a conversation, I have to make a guess, which I suspect is often not great, about their thinking and the amount/level of information to provide back and how to do that in a way that opens thinking rather than closes it. Honestly, I don't have any evidence that students get a ton out of the written feedback on their homework assignments. I've thought about building in class time to have students read the feedback on their assignment from the previous day and do something with it (since homework is turned in digitally and feedback is provided digitally, I have no idea how thoroughly students would be reading my feedback otherwise), but it seems like I could just use this time to talk to students of concern about their work or have the class do a problem related to an issue that I saw on many papers. We already go over homework questions in class before it's turned in and the answers are provided in advance, so presumably, they know if they are understanding the material. If I'm very concerned, I would rather email a student or talk to them in class or ask them to work with me outside of class. Spending lots of time writing comments and then flinging them into a black hole of ??? doesn't seem like the best use of my limited time. But not providing feedback on homework also seems wrong. So I'm at a bit of an impasse here. I've moved some of my homework grading (especially for bigger projects) to in-person conversation and in an ideal world, I would be able to do that for all of my grading, but time with students is a precious commodity.

Feedback on assessments:

This type of written feedback seems to go better than homework. I think that there are a few components that have made it more successful:

1. Students perceive assessments to be more summative and take feedback on them more seriously. They know it's a check of their understanding that will more directly be reflected in their grade (grades as motivation.... laaaaaame, but I'm not sure how to get around this... I have to produce some sort of grade at the end, and I like homework to be purely for feedback so that leaves assessments for grading). As a result, they read comments more carefully and are more motivated to figure out their mistakes and learn from them so that they can show more understanding on the reassessment.


2. I separate the feedback and grading parts to help students focus more on the feedback initially. When I grade assessments, I only write comments/questions (and try not to say too much since I know I'll be there in person to continue the conversation). I record their SBG grades on the assessment in the online gradebook only a day or so later, based on the research that showed that when students receive written comments and a grade on an assessment, they basically ignore the comments and only look at the grade, and that this is not helpful for learning. Getting back their assessments with comments only helps to keep the conversations on concepts and learning only, not on grades, as well as encourages students to work together with less comparison to others. 


3. We spend class time correcting quizzes, usually in groups that are either assigned randomly or by common error types. The quiz corrections are an assignment that is collected, they are not for "earning back points" (I don't actually understand what that means), but they are required in order to reassess. I ask students to analyze their error (did they misunderstand an aspect of the concept? execute a procedure incorrectly? make a careless mechanical error?), as well as redo the problems on which they made errors. Based on my thinking around this issue, going forward, I'd like them to also state what they plan to do to make progress on the issue identified. Michael seemed to think that identifying the type of error is not particularly helpful to students, but I think that when followed up with a "next step," it is maybe more useful?


4. I think that more students actually know what they should do to make progress with assessment feedback. They've done a lot of work with the concepts being assessed. They can talk to peers to understand other approaches, they can talk to me, we can schedule a meeting outside of class to work together, they can refer to online resources organized by content topic to review a concept or procedure, they can do practice problems from homework assignments and previous reviews related to this concept so the feedback is both more closely connected to a concrete goal and to ways of reaching that goal. 

So my main questions right now are:

1. How can I make feedback on homework more useful in helping students change their thinking?
2. Are there ways to improve both my in-class and assessment feedback?
3. How can I move more of my feedback to conversation and away from enigmatic notes that try to strike just the right balance of tantalizing hint/information-giving and hook to motivate kids to want to look at their homework again and rethink their approach, but that mostly get ignored or scanned quickly and not attended to? Did I mention that writing tons of feedback on homework assignments takes a lot of time???
4. Are there aspects of feedback that I'm not considering?

Reflecting on homework

Last week, I posted on Twitter asking for help with homework structure and routines to limit the amount of time we were spending in class going over questions, and boy, did I get some great responses.

Hey #MTBoS, teach me how to go over hw. I give answer keys, but it still takes like half the period to answer questions & make connections
— Anna Blinstein (@Borschtwithanna) April 24, 2015

First of all, I really appreciated everyone who took the time to give ideas and feedback. This is what makes the #MTBoS so amazing. I've gone through all of the suggestions to see which ones make sense for me and my classes, and here is my attempt to summarize and make a plan for myself:

1. Spiral homework so that it lags classwork (great idea from @hpicciotto, @cheesemonkeysf, and @pegcagle) - questions that relate to classwork from a few days ago give students time to process more deeply, have metacognition about their learning, and make stronger connections to the material. I would like to structure my homework into Review (questions that relate to content from a few days ago), Reflect (processing current content and making connections), and Reach (more challenging problems and questions to preview upcoming content) sections.
2. Provide answer keys in advance (I try to do this, but it doesn't always happen due to time constraints with developing an "emerging" curriculum... I need to remember that when I don't do it, it means I lose a ton of class time) and upload detailed solutions after we have gone over assignments.
3. Remind students that they can ask each other questions on Google classroom. I used to use a more bloggy class blog so students naturally commented and discussed online, but after switching to Google classroom, it has totally faded as a tool for student discussion outside of class. I'm hoping that with some reminders and maybe an assignment to comment or respond to a comment, I can jumpstart this type of interaction.
4. After students finish homework, they give feedback regarding their understanding. @z_cress shared an awesome Google Form for doing this, which looks like this and will help me have a better idea in advance of how much time homework will need to take and how much support students need with this content:





5. Start class by having students work in groups to ask each other questions and clarify problems for a few minutes; at the same time, ask various students to put up specific problems that many are confused on or that will be useful to discuss as a class on whiteboards. I still need to think through this a bit more - do I want everyone putting up work and circulating around the room and discussing (as suggested by @dandersod here) or more focused on working in groups and having only a few group questions put up on the board? I would like to play around with these and see what works for me. 

A few other blog posts on how others are handling homework:

• Henri's explanation of lagging homework
• E's plan for homework for next year
• Daniel's switch from homework to exit tickets

If you have other suggestions or blog posts to share about homework structures, I'd love to see 'em!

Thoughts on homework

Julie put out a call for posts about homework differentiation, which came at a perfect time for me since this is something I've been thinking about recently. I'll get to differentiation in a sec, but first, some general issues that I'm having with homework:

It has sort of taken over my teaching life. We spend a lot of time on it in class, and I spend a lot of time on it outside of class. I am not sure if this is good or bad. The way that I do homework, students have all of the answers and must check their assignment and identify problems they have done incorrectly or that they don't know how to do. In class, students discuss their questions with each other in groups for about 5 minutes, then we make a list of questions to go over as a class. Students volunteer to present their solutions to the identified questions and answer any follow up questions from classmates. Those who got those questions wrong or didn't complete them are expected to make corrections to their work. The whole thing takes about 20 minutes, on average, which is almost half of the 45 minute class period. For my end, I look at each problem for each kid and write feedback on each paper. If there are errors or incomplete work or misunderstandings demonstrated, credit is reduced and students are expected to correct those assignments and turn them back in to me. Once the assignment meets my criteria for "goodness," full credit is given. Students are also required to write up corrections and explanations of their errors for tests and complete a journal entry reflecting on their progress each chapter. For my four classes, which have four homework assignments each week, the whole grading process takes me on average about 2 hours every day. That's a lot of time!!!

On the plus side, I think doing problems independently, discussing their thinking with other students, presenting their work and having to make sure that it adequately communicates their process, and getting regular, frequent feedback from me are all really helpful for their learning of both the math content and how to be a better student. On the minus side, it means that there isn't much time left for group explorations or as thorough grounding of each lesson as I would like, both in terms of how much class time is available and how much time I have to plan lessons outside of class.

I use this system not because I think it's super duper awesome, but because I haven't seen any other way that accomplishes the student learning piece better. A few things that have helped me be slightly more efficient:

1. If I see a bunch of errors, say three or more, I stop grading the paper and write a note to the student reminding them that they need to check their answers and make corrections, either on their own or with help from me. The assignment doesn't get any credit until that happens. I don't want them relying on me to check their answers for them, and I want them to come in and get help.
2. As the year continues, I start restricting the amount of time given before corrections can be turned back in to me. Eventually, they are not allowed to correct any more and must come in and get help before turning in the work in the first place. This helps them transition to their future math classes, none of which allow corrections on homework assignments. 
3. I try not to assign too many problems and assign ones that I think are meaningful and helpful to students' learning. Kids do see the connection between putting effort into their homework and their learning and progress in the class.

But really, this is a crazy inefficient system, especially in terms of my time use. I don't know about everyone else, but grading homework for me is somewhere up there between having hot pokers shoved into my eyes and being trapped in an airless mine shaft. It's seriously obnoxious. But I hate the idea of just grading based on completion because then there's no feedback to the student and much less of a connection between homework and learning. So, I'm stuck with my current system unless someone has any suggestions.

Okay, on to differentiation!!

So I don't currently actively differentiate a lot of homework, other than test corrections and journal entries. There is a good deal of differentiation that happens as a natural outcome of how I structure homework though because students who didn't get full credit on assignments have to correct them, which means that they are addressing their own specific misunderstandings. Also, students who score less than 70% on a test are required to complete extra work as well as meet with me.

Here are some other ideas of things I'd like to try:

1. After a quiz, breaking up the class into groups based on the types of mistakes they made. Each group will have all the kids that need to work on a particular concept together. Then, I can work with each group to reteach/address misunderstandings on that topic and their homework will be problems targeting that specific concept. The kids that rocked the quiz will work together on a challenge problem. Their homework will be to complete it and do a write-up of their process.
2. In my regular-paced class, which is completing Algebra 1 over two years, we often spend two days on a learning target. For the first day, I'd like to assign everyone the same basic homework assignment. Then, for the second night's homework assignment, students will have the option of either doing more basic practice OR to do a more challenging, shorter set.
3. I loved this post by Bob Lochel that I recently read about differentiating homework assignments in which different problems are worth different point amounts (depending on difficulty) and the students complete enough problems of their choice to earn the assigned total point value.

Looking forward to reading everyone else's differentiation strategies and ideas.

Homework Folders - Made 4 Math #5

I think I skipped a few weeks there, but hey, that's just how I roll. Anyways, today's Made 4 Math post is basically the only original thing that I've ever thought of for teaching. I mean, literally ever. I take the "beg, borrow, and steal" mentality of teachers everywhere to a new level, which is why I am so proud of this one idea that is my very own. It came about because my biggest difficulty in teaching (and life) is organization. Loose papers are my kryptonite. So this is my system for keeping kids' homework organized. I buy 4 sets of colored folders ($0.10 each at Office Depot), one set for each of my classes, 7 folders per set.

Each folder will hold the work of one group. I generally have between 4 and 6 groups of 4 students per class, but I buy a few extra folders in case one rips or a kid loses one (more on that later). The groups are based on the kids' seats - the desks are usually arranged in groups of 4. 

The folder is labeled with the kids' names and the class and period number. When we change seats, I just stick a new label on top of the old one and write the names of the new group on it.

The folder has two parts: "In" on the left side is where kids turn in their homework and "Out" on the right side is where I place graded homework. One of my group roles is a Materials Manager. This student's role every day is to pick up his or her group folder from the magazine stands that hold all of the folders.

He or she passes out the graded homework from the right side of the folder to each member of the group and after the group or class goes over homework questions, the Materials Manager inserts everyone's homework papers into the left side in numerical order (each kid in the group is assigned the numbers 1 through 4). My gradebook is set up so that the kids are listed in the order of their groups and their assigned numbers so entering grades is a snap, straight from the group folder. It also means that I never ever ever lose homework - it just moves from the left side of the folder to the right side. I don't have to use any class time to collect or pass out assignments, and if I want to know if a kid turned in their homework that day, I can find out very quickly. I also like that it turns over responsibility to the students.

This also allows me to easily keep track of work for kids who are absent. When I pass out any worksheets or problem sets in class, I pass out a copy for the absent kid too and the Materials Manager puts his or her name on the sheet and places it in the right side of the folder so that it's ready for the student when they return to school. 

The best part is that almost all of the other MS math teachers at my school have adopted this system so now I don't even need to take the 5 minutes I used to explain how it works or remind students to not put any papers on my desk or try to hand them to me (I literally tell students that if they hand me a piece of paper, there is about a 90% chance that this paper will never be seen again by human eyes).

One more logistic thing - I added a responsibility for the class leader (a job that rotates each week) to check at the end of class that all group folders have been returned to their container so that no one walks off with one. Before I added that, I had a group folder disappear for good about once every two years, which is still probably a better statistic than how often I lost homework before I started using this system. Hopefully, this is useful to some new teachers out there, and I'd love to hear how you handle homework organization in your class.

Thoughts on homework

Some recent discussions on twitter and on google plus got me thinking about my own homework policies. An Alfie Kohn article that I had read back in grad school and now reread as a result of these discussions posits that homework at any level is at best, useless, and at worst, actually detrimental to learning. In my Algebra classes (which is primarily what I've been teaching the last two years), I give about a page's worth of homework four times per week. Students are expected to check their odd answers at home, mark the problems that are incorrect, and fix them if they can. At the start of the next class, students check any even problems (usually only one or two of these), and discuss with their group the problems that they did incorrectly, but that they couldn't fix on their own. Then, as a class, we discuss any questions that the group couldn't resolve, as well as problems that I just thought were interesting or that bridge to the next lesson. Students turn their homework into group folders and get back their graded homework from the previous day.

The first thing that I do with a homework assignment is determine whether every problem was attempted, the answers checked, and incorrect problems were corrected in a way that demonstrates understanding. I look at the work and write notes if the thinking demonstrated was creative or elegant, or on the other hand, unnecessarily laborious. If there are skipped steps in work or problems that are incomplete, I circle the problem number. If there are errors, either of understanding or careless in nature, I circle and make a slash through the problem number. Each homework assignment is worth 3 points, and I grade based on how well I believe the student understood the lesson. A score that is less than 3 means that I want the student to correct that assignment and turn it back in to me (they will need to fix the circled and the circled & slashed problems). A student can correct and turn in work as many times as they want until the unit is over. They can ask me or other students questions in class to get help with an assignment or they can see me outside of class (there are designated times in the schedule for students to meet with teachers). Almost every assignment, whether it scored a 3 or not, has comments on it that I want the student to read and think about.

Homework is about 20% of a student's grade in my class, but I find that there is a very high correlation between homework grades and grades on quizzes, tests, and projects for most students. (Yes, there are exceptions... for those students that legitimately don't benefit from the homework because their in-class work is sufficient, I handle homework differently.) At the beginning of the year, students are not great at checking and correcting their work on their own. I emphasize homework corrections as a way to understand their errors and deepen their understanding, but I push them in the direction of checking their work and learning to ask for help before they turn their assignment in so that they ultimately are not dependent on me for going through their work and giving them feedback.

One of the pros of my system is that students take homework seriously. They know that I look at every problem and that I use it to assess their understanding of the content. I think that the detailed feedback they get on assignments and the emphasis on correcting and understanding, rather than just completing, is helpful for their learning. The main con of my system is that it takes so darn long for me. I do spend about 30 minutes per class per assignment (this varies between 45 min at the start of the year and 15 min at the end of the year). I also wonder if it's a helpful system if their future Math teachers treat homework very differently (most of the high school teachers either spot check a few problems a few times a week or collect homework at the end of the unit and grade on completion or sometimes, don't count homework at all), but my goal is that by the end of the year, it is internalized and seen as helpful for their learning so that they will continue to do their best work regardless of how it is graded. And isn't that the point of most techniques and habits of mind that we teach?