Session 09: Review and Feedback

@feb22a-cohort We are now entering the week for Session 9: Review and Feedback. Here are the materials to begin to prepare for this week.

1 Like

Discussion 9 Activity: (prompts to respond to…)

  1. How will your team attempt to manage the effects of bias in the review process?
  2. How will your team work to invite a more diverse range of reviews and value a broad range of perspectives?
  3. Are there non-traditional subject matter experts you’d like to work with? (students, community members, etc.)
  4. Are there project specific questions you would like to ask during the review process?

I would really like to engage with pk-12 teachers and non-profit folks who work with families. This type of SME review might be more fruitful than a review from higher ed folks in a peer review process. I can see these folks honing in on the authenticity of case studies or relevant examples. I only worry that this group of folks would value anecdotal experience over research findings- so that’s the bias that I would be looking to balance.

  1. How will your team attempt to manage the effects of bias in the review process?
  • making a priority in the review statement
  • In my role as an Instructional Designer, I believe we have an ingrained and engaged sense of checking for bias, but reviewers and fellow faculty perspectives will help
  1. How will your team work to invite a more diverse range of reviews and value a broad range of perspectives?
  • Besides individuals who have volunteered, I hope my fellow FYE colleagues would like to review to provide feedback. The faculty of these courses come from a wide range of experience, cultures and identities other than my own.
  1. Are there non-traditional subject matter experts you’d like to work with? (students, community members, etc.)
  • My students have been willing to provide feedback on surveys I give. As long as the surveys and feedback are meaningful, as students are willing to complete.
  • Work with student government, student activities, and potentially the student trustee to assess best methods for student outreach.
  • Focus groups on specific topics
  1. Are there project specific questions you would like to ask during the review process?
1 Like

in the field of philosophy, i think reaching out to grad students might be useful and look at successful faculty oer/dei adaption creators whose work reflects the goals of my own in some way. in terms of non-traditional, i’ve worked with local museum folks on art projects so they might be useful when looking at chapter on phil of aesthetics.

Discussion 9 Activity: (prompts to respond to…)

  1. How will your team attempt to manage the effects of bias in the review process?
  • I think having psych faculty (including adjuncts) in our department review the ancillary materials is optimal as they are familiar with students, community & courses. That said, there will be bias as we are members of the department and one of us is dept chair. Therefore, I think recruiting faculty from nearby state institutions would be optimal, perhaps we could get names from OER grant organizers.
  • We need money or something to offer all reviewers with the hopes of mitigating social desirability
  1. How will your team work to invite a more diverse range of reviews and value a broad range of perspectives?

I think it will be important to bring in people are not just psych faculty, but also folks who can examine the activities for feasibility and accessibility without focus on content

  1. Are there non-traditional subject matter experts you’d like to work with? (students, community members, etc.)

I LOVE this question!

  • First year, senior, graduate students who have had a range of experiences in general psychology (e.g., not only students who have earned A grades, in honor societies, etc.).
  • Staff who work with first year students (e.g., first year programs, counselors, student activities, career services)

IMPLEMENTATION – review retreat? Time and place virtual or on campus

Perhaps an expertise exchange to share/exchange consultation time?

@feb22a-cohort

Session 9: Review and Feedback Follow-Up

The ultimate purpose of review is to ensure that your OER is well-structured and ready to be used in the classroom. Review can help you get critical input and suggestions for change that will make your OER even stronger. By sharing your book with subject experts, you can ensure that the content is appropriate, accurate, and adequately covers the material. At heart, review is about bringing more hands on deck to invest and help your resource.

Friday’s session covered the different kinds of review, workflows for these processes, and important considerations for this stage of your projects. We looked at a few central documents and questions that may support you all, and also do a bit of forward thinking about how to share the results of this process!

  1. Peer Review: ‘Peers’ can offer constructive feedback and solutions to improve the quality of educational content. We encourage you to reflect, recognize, and minimize biases in peer review. For instance, consider what types of feedback you need and who can speak to the quality of your content besides another instructor — would an industry expert be able to input? Think back to your SLOs — whose subject matter perspectives are needed to help determine whether the OER is built to help students achieve these outcomes?
  2. Accessibility Review: The accessibility review involves a thorough run through the different output formats of your OER looking specifically at the web accessibility in each format. A specific set of accessibility criteria can guide the people in your team who are tasked with this form of review to ensure that your resource meets the desired accessibility standards. The goal is to make as accessible an OER as you can, knowing that there is always opportunity for improvement down the road.
  3. Classroom Review: This form of review is particularly powerful because it invites feedback from the students which ultimately will help your team to determine necessary improvements for future iterations. Feedback can be gathered both from the instructor using the book to teach as well as the students using the book to learn. Try to identify some academic and non-academic measures as you gather comments from the classroom.

We provided a Review Guide Template that will help you establish review workflows and identify expectations and central guiding questions to better structure your review process and support reviewers. There can be many different lenses/criteria to keep in mind when reviewing the resource, and we suggested coming up with 3-5 central questions to keep things manageable. This is laid out in more detail in the handout for session 9.

In the final part of our session, we asked you to think ahead to how storytelling can be used to communicate the quality of your resource. We prompted you with discussion questions to help you and your teams think of ways to center equity during the review process. We want to compile all your answers here so you and other cohort participants, in this group and in concurrent cohorts can share and learn from one another. Head to the forum and respond to the prompts, located under Session 9: Review and Feedback. (Next week, we can look at and observe other cohorts responses.)

While this stage is fairly straightforward, it’s critical to prepare all the documents and workflows ahead of time to ensure smooth sailing. And remember: along the way, if you have any questions - do not hesitate to lean on each other and the open community, including the Rebus forum, cohort members, and myself.

Next week, we’ll begin looking towards the book’s official launch with a session on formatting and release preparation. This phase is one where your project really begins to take shape as a whole, usable resource.

Thank you for a great session this past week. I look forward to next weeks session on Formatting and Release Preparation.

To minimize bias, I think having faculty from other institutions or experts familiar with the field will be helpful. However, having to do this within their time constraints would be a challenge.
To have a more diverse range of reviews, not just subject matter experts but reviewers who can focus on the accessibility is important to me. Since the main intent is to reduce bias in the content and increase representation, having reviewers who can review the book for these criteria would be helpful. Perhaps faculty from other departments who are familiar with the topic will be able to give an overall perspective on the content.
As for non-traditional subject matter experts, students who take this course will be the main group to work with. I would also like to have a student who has taken my course previously review my book so they can give an opinion on how this book is/not different than the ones we used earlier and if I met my objective.

1 Like