Session 9 - Review and Feedback

Textbook Success Program May 2020 Cohort
1

@may20-cohort Tomorrow, at 3:30pm ET on Zoom, we’ll discuss the different kinds of review and feedback processes to help you produce high-quality OER. We’ll also look at how to manage this process. If necessary we can always revisit some of these concepts and dig a little deeper during our Special Topics session in a few weeks.

We’ll be referring to the following documents:

  1. Handout: Review & Feedback
  2. Slides: Review & Feedback

Looking forward to seeing you all!

2

Hi Arpurva,

We had a question about what authoring tool we are to be using to get the content in the format needed to be hosted by the Rebus system? Was that covered in one of the meetings I missed?

Todd

3

I have a meeting tomorrow afternoon and will only be there for first 45 minutes.

John A. Redmond-Palmer

4

Thanks for your question, Todd. We talked about different publishing/authoring tools during our project scoping session and in a little more detail during our accessibility & inclusive design and authoring & content creation sessions.

@geripalmer has been leaning early on towards using Pressbooks as the authoring and publishing tool — this is the same platform on which the Introduction to Community Psychology book was published. I’ll follow up with you and the team over direct message to get you set up in Rebus’ instance of Pressbooks.

The team at Pressbooks is holding training webinars to help familiarize new users of the tool. You can watch this thread for updates on when the next session will be held: Pressbooks Training Sessions: Getting Started & Advanced Edu Publishing.

5

No worries, thanks for letting me know John. I’ll be sure to ask you and your team for updates first so you can share your updates with the cohort before heading off.

As usual, I’ll post a recap and resources after the call so you can refer to them. Let me know if you have any questions about anything you will miss.

6

Hi Apurva,

I hope you are having a good week. I need to give you a heads up that I might not make it to the session today. Fall kickoff pro dev is taking a lot of time to develop and I have a big push to get ready for next Monday.

I also wanted to thank you for the open pedagogy resources. One of my colleagues that facilitates PD with me is going through the Wiki Scholar training right now. Once she finishes we will begin creating the teaching and learning content repository wiki for faculty and the open pedagogy wiki for student submissions. I am trying to wrap my head around how to manage the student submissions, vet them for accuracy, etc. If you have any thoughts on that process I would love to hear them!

image001.png

7

Thanks Denise. I hope you’re having a good week too. Appreciate the advance notice — I understand that these few weeks are especially busy as many of you are preparing for your Fall terms. As I mentioned to John, if there’s any questions you have upon reading the lesson recap, please let me know.

You’re welcome! I’m glad that you have a colleague to coordinate with. I’ll have a think about best ways to manage student submissions on Wiki, or if there’s anyone who has had experience with this that I could connect you to.

8

I apologize…I have an appointment today during class. I am sorry I will not see everyone today. I look forward to seeing you next week.

9

No problem. Thanks for letting us know. Hope your appointment goes well. We’ll see you next week!

If you have any questions about our topic for the day, please let me know.

10

Hi @may20-cohort — thanks to those of you who could make it yesterday! We had a very good session about review and feedback, with some thoughtful discussions about inclusive review processes. Take a look at the recap below.

Questions and Discussion

  1. @poritzj shared his perspective on the general impression of peer review of commercial textbooks versus OER. We should be more vocal and deliberately highlight when our OER has gone through different kinds of review!
  2. Jonathan also cautioned about whether anonymous review is really completely anonymous — since in the scholarly publishing world, it’s possible to zero in on the identity of authors based on topic, method, keywords, etc. Does this vary with OER?
  3. @mbranson asked about implicit bias and reviewers, which is a concern in the Math field. How can this bias be reduced or removed? Bias can present itself along many lines - race, ethnicity, gender, institution, language, other social/cultural lines. I’ve compiled a list of some of the resources that I’ve come across that explore the issue of bias and peer review (particularly):

Leveraging anonymous review, removing author identifying details from content, training your reviewers, providing etiquette instructions in the review guide proved most common suggestions. Resources may also contain links to other studies on this topic, which you may be interested in further reading.

Chat Transcript and Resources

Most of the discussion took place during the call rather than in the chat. I’ve saved the transcript, and this was what was shared:

Lesson Recap

Feedback and review is a chance for you to share your book with subject experts and ensure that the content is appropriate, accurate, and adequately covers the material. Collaborators will read through content and provide critical suggestions to improve the resource for its intended audience. The presence of review on your OER signals to a prospective adopter that the work has passed through rigorous quality control, and that its content is suitable for use in the classroom. In so doing, it acts as good advertising for the efficacy of your resource.

We’ve identified different types of review processes that can be carried out on your book:

  • Peer review (pre-publication): It is conducted by subject-matter experts and is a common marker that ensures quality of educational content. It can be a prerequisite for adoption or even submission to repositories. We group peer review into anonymous (most traditional, has prestige value to some groups), non-anonymous (lets you credit reviewers and set up a dialogue with authors, and open (most transparent & collaborative, letting others join in).
  • Accessibility review: All creators should ideally find an accessibility practitioner to conduct a set of final checks on the books to see what accessibility standards are met, and to identify areas for improvement.
  • Classroom review: Think of this as a trial run of your resource in the classroom setting, and an opportunity to include student voices in the publishing process. Feedback can be gathered both from the instructor using the book to teach as well as the students using the book to learn.
  • Feedback via project discussion: All resources should contain a pathway for anyone to provide feedback about the book. These openings may even help recruit other collaborators to work on revisions and other post-publication tasks.
  • Post-publication review: These can be collected in repositories, journals, from adopters, and are important to show that the book is being used and having an impact. They are also extremely helpful to promote further improvement of the book and more adoptions, especially if posted on a public site.

With each type of review, you should adequately prepare for the process prior to recruiting reviewers. Collaborate with your existing team to confirm what you would like to get out of each type of review. Once you have your reviewers confirmed, managing the process is fairly straightforward. The review process doesn’t end once reviewers hand in their feedback — we suggest you thank reviewers and keep them engaged and enthusiastic about the resource (take a look at the slides for more details). At heart, review is about bringing more hands on deck to invest in your resource and help it grow with their feedback and input.