Peer observation and review improves university science teaching

The main responsibilities of academics in STEM (science, technology, engineering and mathematics) disciplines are teaching and research. However, many staff have little or no formal training in how to teach. To address this issue, the Peer Observation and Review of Teaching (PORT) program was introduced at the University of Sydney in 2013.

Port programs have been found to be well received and generally effective, but difficult to establish and successfully maintain in higher education. Furthermore, research into these programs has focused on the participants’ experience of the program itself, rather than considering the outcomes relating to teaching practices. To address this research gap, Dr Helen Georgiou and colleagues set out to investigate the effectiveness of this PORT program on teaching practices from the perspective of university teaching staff.

PORT
At the University of Sydney, most participants who have experienced the PORT program have gotten to know it as part of their induction to teaching. Staff can also take part in PORT outside of this induction, and therefore participants include established professors as well as academics at the beginning of their careers.

The program begins with an initial workshop, where staff, collaborating with each other and the workshop facilitators, agree on specific and measurable goals for improving their teaching practice.

Following the workshop, participants are observed whilst teaching students ‘as they normally do’ on a 1:1 basis by another academic (called the reviewer). This reviewer is a more experienced teacher with both expertise in teaching methods and in the broad discipline area. The participant and the reviewer have a follow-up meeting to discuss how the lesson went and to agree upon possible improvements using a new teaching method.

Academics reduced the content of their lectures to allow time for this active learning to take place.

The same reviewer observes the participant give a second lesson using the suggested new teaching methods. The success of the new approach is reviewed in a second follow-up meeting where participants also discuss the future direction of their teaching practice with the reviewer.

PORT participants are also encouraged to learn from their peers by observing at least two teaching sessions selected for observation because they include specific teaching methods. They are directed to critically reflect upon what they have seen using a specially designed form, called the Lecture Activity and Student Engagement (LASE) form.

At the final colloquium, participants meet to share their experiences. This has the added intention of enabling the participants to feel that they are part of the university teaching community.

Research questions
The researchers wanted to evaluate the impact of the PORT program on the teaching practice of members of the Faculty of Science at the University of Sydney. Their specific research questions were:

1. What, if any, are the changes the participants made to their teaching practices as a result of engagement in the PORT program?
2. Were these changes sustained after engagement in the PORT program?
3. Which elements of the program were important in encouraging these changes?

A total of forty-three out of forty-five academic staff who had completed the PORT program between 2014 and 2016 were invited to take part (two could not be contacted). Sixteen academics, with differing levels of seniority within the university consented to participate in the study. The researchers acknowledge that the self-selection of the participants may have biased the study results.

Study methods
The participating academics completed a 35-item online questionnaire. Thirty of these questions were closed and five questions permitted open-ended responses. The questions focused on different aspects of the PORT program as follows:

• Demographics and context (Q1-Q5)
• Initial workshop experience (Q6-Q8)
• Peer observation (Q9-Q18: including two open-ended questions)
• The review process/cycle (Q19-Q28: including two open-ended questions)
• Experience of the final colloquium (Q29-Q30)
• Whether any changes in teaching practice were sustained (Q31-Q33)
• Whether participants would recommend the program and had any suggestions for improvement (Q34 & Q35: including one open-ended question)

In addition, five of the participants engaged in a sixty-minute, semi-structured interview about their experiences of the program. Their responses were transcribed and evaluated using an approach called thematic analysis. This involved: (1) becoming familiar with the data; (2) generating initial codes; (3) searching for themes; (4) reviewing themes; and (5) defining and naming themes.

The findings
Participants considered the PORT program successful at improving both their abilities as teachers and the engagement of their students in class. They appreciated the structured approach taken by the program, and valued the different steps involved, though some participants found the final colloquium to be less useful.

The researchers found that all of the participating academics had adopted new teaching methods as a result of participating in the PORT program. The new teaching approach often involved some form of ‘active learning’ by the students, such as problem solving or taking part in scientific discussions. Academics reduced the content of their lectures to allow time for this active learning to take place. As one participant stated:

I’m teaching next semester [and I’m planning to] put a little more material online and get them to do some pre-readings and those sorts of things and then spend more time going over detailed stuff in the lecture rather than try to cram it all.

These active learning approaches focused on meeting the needs of students. For example, one academic reduced time spent lecturing and instead gave students the opportunity to work through examples.

This change in teaching methods has improved student engagement in learning and a participant noted:

I was sitting at the back of the lecture hall … and then I was observing all the students, like they [had] lecture notes, like [on their] laptop, iPad lecture notes were open and they were not involved in other activities. Everyone was having [a] laptop, but they were not browsing [the] Internet … they were actually paying attention.

The researchers found (from the questionnaire responses and as an interview theme) that the relationship between the academic and the reviewer was an important driving factor for the introduction of new teaching methods. At first, the academics felt that being watched while teaching was a ‘daunting’ prospect. However, they valued the feedback from a ‘second pair of eyes’ and in particular, observations about how the students were responding during class.

The participants had to make decisions about which of the criticisms and suggestions from the reviewer they were going to take on board. They were appreciative when the reviewer had familiarity with their discipline and could gauge how well they were explaining difficult concepts. However, they also found that it was helpful to have a reviewer detached from their immediate work environment. This shifted the focus of the feedback away from the content of their lecture and to strategies that would engage the students with the content.

The researchers observed that the different themes identified in the PORT program all function to enhance the students’ experience of learning.

The academics also benefitted from being given the opportunity to watch their peers teach. Not only did this help them to learn new teaching methods; it also helped to dispel the notion that they were ‘on their own’ in the lecture room. They developed a sense of being part of a university teaching community. In particular, praise from the reviewer and other peers increased their confidence.

Importantly, the PORT program encouraged the participants to develop reflective practice, which is essential in order to sustain improvements in their teaching. Indeed, the participants have persisted in using new teaching methods and furthermore, they have recommended the PORT program to their colleagues.

Significantly, the researchers have observed that the different themes identified in the PORT program all function to enhance the students’ experience of learning.

Overall, the researchers have noted that ‘legitimate and lasting’ changes in teaching are difficult to achieve and they have heralded the improvements in the Faculty of Science at the University of Sydney as a transformation.

Conclusion
Dr Helen Georgiou and co-workers from the University of Sydney have found that the PORT program has improved science teaching and student engagement in higher education.