Instructor-Led ECF Survey: A Step-by-Step Guide
Asking students for feedback early in a course can be one of the most effective steps toward improving your teaching. It can provide more nuanced and targeted information than final course evaluations, allows you to respond and potentially make changes while the course is still in progress, and shows students that you care about their learning and take their perspectives seriously. Both students and instructors have reported improved course experiences after the early course feedback process.
The Eberly Center offers multiple ways to collect and act on early course feedback from students, with the most commonly used option being Eberly-run focus groups. The steps below will help you through the entire process of a “DIY” early course feedback.
Running a survey yourself is a good fit if you meet all of the following criteria:
- You have gotten early course feedback before and want to check the impact of the changes you have made since
- Your goal is formative feedback, not an evaluation of your teaching
- You are open to making changes to the course (and have the authority to do so)
- You are comfortable seeing unfiltered, anonymous student feedback
- You are comfortable creating a Google survey and performing an initial analysis of qualitative responses
- You can run the survey after students have already gotten grades back on a major course deliverable and with multiple weeks left in the semester
If you want more support in any of these areas or are interested in end-of-semester options for feedback, come talk to us!
Choose a date during weeks 4-8 of the semester, or 3-4 of a mini, when you can dedicate roughly 15 minutes of class time to having students fill out the survey (doing this in class is essential to getting reasonable completion rates). Consider what will be happening immediately before and after this – you will get broader and more balanced feedback if students have had time to process any grades and are not anxious about an exam. Most importantly, you will need to set aside time to look at the feedback and debrief with students in a reasonable timeframe, if not the next class period then within the following week.
QUICK TIP: Schedule your post-survey meeting with an Eberly consultant in advance!Next, make a copy of the survey template Google Form (if you prefer to collect and analyze data electronically) or document (if you would rather work with hard copies; note that this is not appropriate for small classes where student handwriting is likely to be identifiable). If you are using the Google Form, you can change the framing language to be in your own voice but do not change the template settings, which should look like the screenshot below.
There are two core questions which are included in every survey, one asking students which strengths of the course help them learn and another asking for suggestions. If you have reasons to ask about something specific that students might not comment on without prompting, you can add questions from our bank of common additional questions. However, we recommend adding no more than 3 short-answer questions or 5 Likert scale questions to ensure you can get high-quality responses in a short time.
Student engagement is usually highest if the survey is done at the beginning of class, not the end.
Begin with some framing to make sure that students understand 1) their feedback will be anonymous, 2) this is formative feedback you are voluntarily collecting because you want to hear about their experiences in the course, and 3) responses will be most helpful if they are specific and actionable.
Once all students have access to the survey, the instructor and any TAs should step out of the room for the rest of the survey time.
We recommend using the protocol provided below for analyzing data from your survey:
4.1. Read through all of the student worksheets and identify significant themes or topics (e.g., class discussion, lecture slides, grading, etc.) that students commented on.
Here, “significant themes” refer to categories of feedback that each arise multiple times across students. Write down the conspicuous themes.
- Try to identify at least three themes for “strengths” and three themes for “suggestions.” These themes should be value-neutral, rather than focused on students’ opinions about them. For instance, if students say “no one speaks in my small breakout room discussions,” the topic would be “small group discussions” or “breakout rooms”
- Focus on themes that are within your control. For instance, if many students say “I don’t like the time of the class,” that’s likely something that you cannot change in the middle of the semester. Make a note of it, but look for themes that are within your control.
4.2. Review the student worksheets and highlight/mark/color code comments that pertain to each of the themes that you identified.
Comments could be listed under “strengths” or “suggestions.” For example, if you identified “class discussion” as a theme, then student comments like “Zoom breakout rooms are great because I can discuss course concepts with my classmates” and “I’d like more opportunities for classroom discussions” would both get flagged for this theme. The first comment is a strength. The second comment is a suggestion.
4.3. In addition to highlighting these comments, record a + or - sign beside the comment and track how many of each sign the theme has.
This step will help to identify if there is some consensus about how effective a particular aspect of your course is. For example, if “class discussion” has 10 plus signs and 1 minus sign, then you can feel good that class discussion is working well. This step can be challenging if students have not worded their feedback in a constructive manner or if they have only listed complaints instead of providing suggestions. Try to focus on the essence of the issue that the student is identifying and not the way the student has expressed it.
4.4. Dig deeper into the specific strengths or suggestions that students identified.
- If the “grading” theme has a lot of minus signs, what are the particular, actionable suggestions that students have made (e.g., needing more clarity, needing grades/feedback quicker, etc.)?
- If students have focused on making complaints rather than suggestions, consider bringing up that theme to the whole class to discuss avenues for solutions. This conversation could be open-ended or you could prepare a few solutions that you would be ready to implement for the rest of the semester and let them decide which one addresses their concerns the best.
- Some themes might have similar numbers of plus and minus signs. In those cases, return to the student comments to see if you can reconcile this difference. For example, maybe students find breakout rooms beneficial for learning, but they are suggesting that they spend more/less time in them.
4.5. Return to unmarked student comments and identify new themes as time permits.
If you identify only the top three themes/topics for “strengths” and the top three for “suggestions,” then you’ll likely still have student comments that you didn’t mark or group together. If time permits, return to these student comments and see if there are additional themes or recommendations that you can glean from them. Sometimes, suggestions from a minority of students can be the most useful for all learners. However, one can also easily get hung up on one or two strongly worded comments. Ask yourself, will implementing this suggestion improve student learning or engagement?
Questions you should be able to answer based on your analysis:
- What were the major strengths (3-5 things)? Which strengths resonate with you?
- What were the major suggestions (3-5 things)? Which suggestions resonate with you?
- Which specific comments are causing you to feel stuck or unsure about what to do next?
After you have completed your preliminary survey analysis, we recommend you meet with an Eberly consultant, ideally within one week of deploying the survey. Email eberly-assist@andrew.cmu.edu to request a consultation; put “Instructor-led ECF survey” in the subject line. Please submit requests at least one week in advance of your survey deployment. As with all of our services, this meeting would be confidential and collaborative: our goal is to help you strategize and move forward with your course based on the student feedback.
How we can help you:
- Identify and prioritize the most practical suggestions and feedback
- Strategize around the major strengths or suggestions, as well as interpret specific feedback
- Brainstorm how to respond to students
- Get technology-enhanced learning (TEL) support (e.g., learn about options for polling software, improve Canvas course functionality)
How this meeting differs from the Focus Group service:
- One post-survey meeting only
- You must do an initial analysis of the data first (we will not review raw data sets)
- No formal report (but we will send you a brief summary of strategies over email)
Please note! We require that you send your analysis summary to your Eberly consultant in advance of this meeting. The summary can be informal (e.g., list of main strengths/suggestions, plus additional comments you would like help interpreting). See “Questions you should be able to answer based on your analysis” above in Step 4.5.
A critical part of the early course feedback process is:
- Discussing the feedback you received with your students
- Thanking students for their effort and input
- Attempting to respond to their feedback to set a positive tone for the class and show fundamental respect for students' role in making the class work
It is important to respond to your students’ feedback in a timely manner, as soon as you have had a chance to analyze your results and (recommended) meet with an Eberly consultant.
How you address the feedback is up to you. At the very least, we recommend thanking students for taking the time to thoughtfully fill out the survey. If you plan on making changes based on student feedback, we encourage you to address both strengths (i.e., things you will continue to do or do more often) and students’ suggestions that you are able/willing to implement. If there are specific changes you are making or considering, we encourage you to name them and link them to student feedback. Otherwise, students may not feel that their feedback was heard.
- We recommend that you respond to student feedback verbally, during class, rather than by email, to minimize misunderstandings or misinterpretations.
- If you feel that some of the suggestions were not specific, or some themes didn’t yield specific ways to improve, or if student feedback was divided evenly (e.g., half the students said they enjoyed small group discussions and the other half said small group discussions do not help with their learning), then consider having a class discussion or administering a more targeted survey to dig deeper into the issue or what potential tweaks could improve the student experience or learning.
- This discussion with students can also be an opportunity to explain your choices and teaching philosophy. For instance, “We have small group discussions to give you a chance to think about the issues in a lower stakes situation.”
- If a few students requested a change you don't plan to make, help them to understand why it isn't possible.
QUICK TIP: Frame your discussion so that you maintain a positive tone.
It is important not to come across as defensive, angry, or overly apologetic because these reactions can undermine students' trust and respect for you. Sometimes just a few negative comments can make it difficult to stay positive, so feel free to contact the Eberly Center if you want to talk about the feedback with someone before responding to the class.
QUICK TIP: Consider ways in which you can enlist students' support for your efforts to improve the course.
For example, if students report that you talk too fast or too softly, ask them to let you know nicely (perhaps via a single hand signal) so that you can adjust quickly. Or if students report difficulty understanding the material, you might propose some ideas such as pausing longer, summarizing the main points more often, or encouraging students to ask more questions and discuss what seems most helpful.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.