Session 9, Review and Feedback

Textbook Success Program February 2022 C Cohort
1

@feb22c-cohort: Our topic this week is Review and Feedback and here are some useful documents:

See you on Thursday!

1 Like
2

For our session this week, you’ll reply in this thread to these prompts:

  1. How will your team attempt to manage the effects of bias in the review process?
  2. How will your team work to invite a more diverse range of reviews and value a broad range of perspectives?
  3. Are there non-traditional subject matter experts you’d like to work with? (students, community members, etc.)
  4. Are there project specific questions you would like to ask during the review process?
1 Like
3

On number 1 - great question! Like my syllabuses, perhaps I can complete a ‘diversity’ statement and advise on the ‘reading’ process / role of my OER LIT 200: which is to work with both canonical and noncanonical texts and texts from different regions that are equally valued and share aspects and themes significant to many fields, especially literary studies.

4

We will attempt to manage the effects of bias in the review process by including underrepresented groups in our review process. These may include other faculty perspectives as well as students. We should also welcome a further discussion when we observe unconscious bias.

1 Like
5

On number 2, well this is what we have been working on at the level of ‘readings’; we should invite reviewers with experience and knowledge that is quite different than traditional experiences with literature, like relying on Bloom’s reading of Don Quixote for example.

6

On number 3, yes; I should reach out to members in the community whose work is with nontraditional learners and with nontraditional authors, perhaps have them as reviewers; or if they cannot commit to this then, as advisors.

7

On number 4; yes! Too many. But on a serious note I think my approaches in reaching out to you at Rebus who may know of desired experts has helped immensely.

8

  1. How will your team attempt to manage the effects of bias in the review process?
    -Incorporate a small group of diverse individuals (e.g., individuals from different areas in higher education, faculty from different institutions) to get their take on what needs revision

  2. How will your team work to invite a more diverse range of reviews and value a broad range of perspectives?
    -Incorporate faculty and staff members to review within my department (e.g., biology) and outside of my department (e.g., chemistry, psychology) allowing for more diverse reviews and opinions
    -Include students to also have a role in reviewing content since the OERs are dedicated and written for them

  3. Are there non-traditional subject matter experts you’d like to work with? (students, community members, etc.)
    -Students, admins (e.g., department secretary), as well as writing reviewers (e.g., my writing coach in my Ed.D. program). I believe that by having non-traditional subject matter experts the text won’t be written simply from a “biological mind” but can incorporate the views and voices of others outside of the field.

  4. Are there project specific questions you would like to ask during the review process?
    -How does the text flow?
    -Does the content make sense?
    -Does the OER seem user friendly, easy to read and access?

1 Like
9

Hi everyone - For the Lifespan Development Project, here is what I am thinking…

  1. One thing that comes to mind is asking for review support from faculty I don’t know; I think it would be helpful to ask people who don’t know me and don’t know how I teach, so they cannot make assumptions about what I mean.

  2. Again, reaching out to people I don’t currently work with will be important. Also asking for input from different subject areas, to emphasize the idea that developmental psychology is an interdisciplinary field.

  3. I definitely want to include students in this process. I am hoping to have them add content but I also would love to have students serve as reviewers for the project.

  4. Because lifespan covers literally the whole lifespan, there is a LOT of information and ideas that could be included, but not all of them are going to fit. I will definitely be looking to my reviewers to help me pare down the content so that what ends up in the book is essential information, and there are pathways to extra stuff but the main book is relevant and informative, but also succinct and accessible. And I actually think that will really help with the first three questions because that will guide some of those considerations.

1 Like
10

Since creating equitable texts are at the core of the ROTEL grant’s values and process, I’ve no doubt that Larry will also seek out student feedback/review (as an example of ‘less traditional’ reviewers) for his text Conversations with the Earth.

11

Often students take the Guidance course during their first semester. Many have grown up in an area with a high poverty rate and have experienced the effects of structural racism in their earlier education. It is not uncommon that a number of my students are in the process of learning English as a second language. I want reviewers to assess the readability of the material.

1 Like
12

@feb22c-cohort:

Several people weren’t able to make it last week – I understand, this is a busy time of year! – so let me remind you that the slide deck and handout are available, as is a recording!

I’ll give the usual fairly detailed session summary in a second, but just to get a couple of things up near the top of this post:

  • Above in this thread (the second post on the thread) are some discussion prompts to which folks who were able to make the session responded here, in this thread. I strongly recommend that those of you who couldn’t make the live session last week take a moment to skim the slides and handout and then post your own thoughts on those discussion points here, as well!
  • Several teams have seemed embarrassed about slow progress in the sharing we do at the beginning of the sessions (or maybe that’s also why folks aren’t coming to sessions?). But we are indeed in a busy time of year, and many teams have described how they have the very reasonable plan of making a lot of progress in the summer, after the chaos of the spring semester is over. I wanted to point out (again! I think we’ve discussed this in sessions!) that a lot of the things we are doing in these sessions can be thought of as building solid foundations which will enable you to be much more productive in the future (the summer, perhaps). Also, these are things you can possibly work on now, in the midst of semester chaos, in the way that you might not be able to sit down and create a draft of Chapter 4 of your OER, or some other big task. Making a Review Guide, as we talked about in Session 9, or an Author Guide or Style Guide as we’ve talked about in the past, or doing the other things which are in the session handouts, is possible to do now and will help you in the future!

OK, for the session summary:

We talked about review, the ultimate purpose of which is to ensure that your OER is well-structured and ready to be used in the classroom. Review can help you get critical input and suggestions for change that will make your OER even stronger. By sharing your book with subject experts, you can ensure that the content is appropriate, accurate, and adequately covers the material. At heart, review is about bringing more hands on deck to invest and help your resource.

Our session covered the different kinds of review, workflows for these processes, and important considerations for this stage of your projects. We looked at a few central documents and questions that may support you all, and also do a bit of forward thinking about how to share the results of this process!

  1. Peer Review: ‘Peers’ can offer constructive feedback and solutions to improve the quality of educational content. We encourage you to reflect, recognize, and minimize biases in peer review. For instance, consider what types of feedback you need and who can speak to the quality of your content besides another instructor — would an industry expert be able to input? Think back to your SLOs — whose subject matter perspectives are needed to help determine whether the OER is built to help students achieve these outcomes?
  2. Accessibility Review: The accessibility review involves a thorough run through the different output formats of your OER looking specifically at the web accessibility in each format. A specific set of accessibility criteria can guide the people in your team who are tasked with this form of review to ensure that your resource meets the desired accessibility standards. The goal is to make as accessible an OER as you can, knowing that there is always opportunity for improvement down the road.
  3. Classroom Review: This form of review is particularly powerful because it invites feedback from the students which ultimately will help your team to determine necessary improvements for future iterations. Feedback can be gathered both from the instructor using the book to teach as well as the students using the book to learn. Try to identify some academic and non-academic measures as you gather comments from the classroom.

We provided a Review Guide Template that will help you establish review workflows and identify expectations and central guiding questions to better structure your review process and support reviewers. There can be many different lenses/criteria to keep in mind when reviewing the resource, and we suggested coming up with 3-5 central questions to keep things manageable.

In the final part of our session, we asked you to think ahead to how storytelling can be used to communicate the quality of your resource. We prompted you with discussion questions to help you and your teams think of ways to center equity during the review process. We want to compile all your answers here so you and other cohort participants, in this group and in concurrent cohorts can share and learn from one another. Head to the forum and respond to the prompts, located under Session 9: Review and Feedback.

While this stage is fairly straightforward, it’s critical to prepare all the documents and workflows ahead of time to ensure smooth sailing. And remember: along the way, if you have any questions - do not hesitate to lean on each other and the open community, including the Rebus forum, cohort members, and myself.

In our next session, we’ll begin looking towards the book’s official launch with a session on formatting and release preparation. This phase is one where your project really begins to take shape as a whole, usable resource.

1 Like
13

I appreciate this a lot — it’s very true that the work you are all doing now will set you up for calmer and smoother times in the coming months, and let you devote your full energy to creating. Use the small moments (even if only 30 minutes) to adapt the various guides and templates, or decide on workflows as a team. It will certainly save you a lot of time and mental energy as Jonathan notes!