In this QAA podcast, colleagues explore the institutional context for pass/fail assessment in undergraduate and postgraduate courses at Leeds Arts University, and ways in which the experience of pass/fail assessment can support authentic learning. We hear from students about the role of tutor and peer feedback, and finally, consider its impact on skills development for future employability.
Guest speakers are: – Peter Hughes (Academic developmental manager) – Dr Laura da Costa (Access and participation development manager) – Max Mooney-Walsh (Level 5 student pursuing BA Hons Comics and Concept Art) – Lennox Bruwer (Level 7 student, MA Animation, Interactive Technology, Video Graphics and Special Effects)
Authors: Vikki Hill, Prof Sam Broadhead, Liz Bunting, Dr Neil Currant, Dr Laura da Costa, Dr Marianne Greated, Peter Hughes, Robert Mantho, Dr Emily Salines, Dr Thea Stevens
In February 2021, colleagues from University of the Arts London (UAL), Leeds Arts University (LAU) and Glasgow School of Art (GSA) secured funding for the QAA Collaborative Enhancement Project – Belonging through assessment: Pipelines of compassion. The project began against the backdrop of the Covid-19 pandemic and the team identified a shift in assessment practices across the three participating arts institutions. This offered an opportunity to further our work, in collaboration, to address social justice, belonging and inclusion through compassion.
This project aims to:
Identify areas of enhancement in assessment policies and practices to promote student sense of belonging and tackle issues of social justice.
Link this relational work with attainment gap/awarding differentials agendas in the creative arts.
Develop collaborative, dialogic, polyvocal and affective resources for staff development across the HE sector.
Three research strands emerged from themes relevant to our own institutional priorities, mutually informing the project and institutional practice and policy. These are pass/fail grading, the whole-self and feedback. Initial cross-institutional research and evaluation into pass/fail assessment was taking place at UAL and at LAU in the wake of measures introduced during the pandemic. The whole-self strand developed from academic enhancement work on Fostering Belonging and Compassionate Pedagogy at UAL. The feedback research strand linked to enhancement work in progress at GSA around assessment policies and practice and with UAL work on formative feedback practices and assessment design.
The Belonging through Assessment Symposium was held on Thursday 21 October, 2021. Hosted by colleagues from UAL, LAU and GSA, the day featured interactive discussion, reflection and practice-sharing, exploring perspectives and possibilities of assessment to nurture belonging as a way to address issues of social justice. There were contributions from invited speakers, Dr Jan McArthur, Dr Maha Bali, UAL students Amina Akhmendova and Simbi Juwon-Sulaiman, alongside the project team and academic practitioners, Janine Francois, Mo-Ling Chui, Michael Smith, Nina Spencer and Prof Sheila Gaffney. Colleagues from UAL, GSA, and LAU opened up space to consider humanising, compassionate policies and pedagogies of assessment. Topics included authentic assessment for social good in the creative industries, compassionate feedback, pass/fail grading and the implications of these approaches on policy design and enactment.
In the following section, each of the three research strands (pass/fail grading/ whole-self/ feedback) offer an overview of the work to date and plans for future development and project resolution.
Strand 1: Pass/fail grading
The use of discriminating grades, with letter or numeric representations, for the assessment of student work is embedded in higher education systems internationally. However, it has long been recognised by educationalists that grade-based assessment exerts a substantial influence over students’ study behaviour through extrinsic motivation, and this has led to concerns that a narrow focus on grades can result in overly strategic and superficial approaches to learning (Boud & Falchicov, 2006; Harland et al., 2014; Rust, 2002). Opening up the discussion around pass/fail grading involves questioning some of the fundamental tenets of the neoliberal university (Tannock, 2015).
Countering the dominance of the grade-based approach is a modest history of pass/fail, or gradeless assessment. In the United States, from the 1960s and peaking in the 1970s, the practice was associated with a small number of radical liberal arts colleges (Weller, 1983). More recently there has been a growth of use within professional subjects, particularly medical education and allied subjects where the role of assessment is to judge whether someone is competent in a field (White & Fanlone, 2010; Spring et al., 2011; Ramaswamy, Veremis & Nalliah, 2020). A pedagogic movement associated with “ungrading” (Blum, 2020) has developed. The onset of Covid-19 has seen more than 150 US higher education institutions adopt pass/fail grading as part of a range of measures to offset the impact of the pandemic – however in most cases, given the significance of grade-point average (GPA) in the US system, this has been offered to students only as an option (Burke, 2020; Busken, 2020).
In the UK during the pandemic, many universities implemented ‘no detriment’ policies which increased the use of pass / fail assessment. Specifically at UAL, the ‘no detriment’ policy included making the whole first year pass / fail instead of letter grading. This shift occurred part way through AY 19/20 and was applied for the whole of AY 20/21. At Leeds Arts University pass/fail assessment was introduced for first year undergraduate students during 20/21 as part of a range of regulatory and other measures taken by the University to support students through this situation. Pass/fail assessment was already used within all postgraduate courses at Leeds Arts, so there was familiarity with the approach, but this was the first time it had been used in undergraduate study. Leeds Arts University has reverted to graded assessment from 21/22, but the experiences of the pass/fail intervention continue to be reflected upon and evaluated and the potential use of pass/fail assessment in the future has been kept under review.
This natural experiment in the two institutions gave the team a chance to investigate the impact of pass / fail and investigate staff and students’ attitudes to grading.
What have we been doing?
At UAL in the summer term of 2021, research was conducted with students and staff evaluating the shift from a graded first year to an ungraded (pass /fail) first year.
Number of participants
1st year students (level 4)
Experiences of pass / fail in AY20/21
2nd year students (level 5)
Experiences of pass / fail in AY19/20 and grading in AY 20/21
Experiences of pass / fail in AY19/20 & AY20/21
This interpretivist, qualitative research employed semi structured student and staff interviews. We applied Thematic Analysis (Kiger and Varpio, 2020) to our data. Our initial coding generated a set of codes (Saldana 2013) that can broadly be categorised as:
Affective– codes related to emotional states, e.g., stress, relieved etc.
Social – codes related to other students and comparisons with peers, e.g., awareness of other students’ situations, I work harder (than others) etc.
Feedback – codes related to feedback on their assessed work, e.g., feedback for improvement, lack of feedback etc.
Grading – codes directly about grading, e.g., comparing against other’s grades, comparing grades against criteria, attitude towards grading, etc.
Process – codes about how the change to pass / fail was implemented, e.g., communication to students, staff moderation process etc.
Impact on learning – codes about how different grading regimes shaped the students learning, e.g., focus on grade not learning, able to learn etc.
The research was used to produce an internal evaluation report for Deans, has been disseminated at various events and is being written up for academic journals.
At Leeds Arts University there has been a process of reflection on and evaluation of the experiences of pass/fail grading for first year students during 2020/21, which has led to two internal reports. The first, prepared for May 2021, reviewed the sector evidence base for pass/fail grading to aid the contextualisation of our own experiences. The second, prepared for May 2022, was a follow-up evaluation of the 20/21 experiences, informed by the literature review and drawing upon institutional data sets, a consultation with course leaders and a focus group with students who had experienced pass/fail grading the previous year.
Within the LAU evaluation, various institutional data sets (mitigating circumstances, retention and progression, student satisfaction surveys) were explored to investigate how the outcomes and experiences of L4 20/21 students compare with those of previous years. For the student focus group, a series of question prompts were developed, informed by the pass/fail literature, to explore students’ experiences of grading including:
Previous experiences of grading in education, prior to study at Leeds Arts University.
Recall of reactions and feelings to first learning that the work they produced for their course (20/21) would be evaluated as pass/fail only.
Reflection on their approaches to studying under pass/fail.
Reflection on their experience of receiving their first piece of pass/fail graded work back from their tutors?
Experiences of returning to graded assessment in 2021/22.
In evaluating and learning from the experiences of pass/fail assessment across the two institutions it is important to acknowledge that making sense of 2020/21 is complex, and any attempt to separate out a single phenomenon is problematic. Data and information around overall student performance and experience in 2020/21 will reflect a range of factors, not just the introduction of pass/fail. In both cases it should also be acknowledged that the pass/fail initiatives weren’t planned pieces of innovation or research, they were part of a package of institutional responses developed and implemented at pace in exceptional circumstances.
At Leeds Arts University, from the student focus group, and from the UAL research it has been learned that:
Students have nuanced perspectives on grading.
Students recognise that their previous educational experiences have to a degree conditioned them to work within a graded system, such that removal of that can be destabilising.
Over time, students appreciated that pass/fail grading helped reduce stress and anxiety around their L4 transition into university.
Where students had pass / fail introduced mid-way through an academic year their initial response was more negative. For example, at Leeds Arts University overall student satisfaction with assessment was noticeably lower for 20/21 than for 19/20 or 21/22. At UAL, second year students (those who experienced pass/fail being introduced mid-year) were slightly more negative about the experience than first year students where the pass/fail system was already in place at the start of the academic year.
Pass/fail grading enabled some students to feel free to experiment and take risks in their creative practice.
University grading is for many students mystifying at first and understanding only develops through time. Students acknowledge that staff talk to them about this, but it only begins to have meaning when they have received graded work back and are trying to make sense of it.
Many students are constantly asking themselves the questions: “Am I good enough?” or, about their work, “Is this good enough?” Grading provides one signifier of this, but students recognised that the removal of grading made them pay much more attention to feedback, and they found that that was where the more important information was.
There are some indications that an immersive experience of pass/fail grading has helped students break out of previous mindsets, helped them become more independent learners, and better able to judge the quality of their own work.
There is some indication that the return to grading at L5 has made the learning environment feel more competitive.
From course leaders there remain mixed views over the merits of pass/fail grading, but a shared recognition that in whatever grading system is being used, students need support through transitions between systems and in their development of assessment literacy.
A provisional observation from Leeds Arts University institutional data is that the single discriminatory threshold of pass/fail assessment may act as a clearer line of distinction than in existing assessment frameworks. Something passes, or it doesn’t. The idea of a “marginal fail”, typical for many UK HE assessment frameworks, disappears. This aspect of pass/fail, and the potential consequences of that, will be explored further. From UAL progression data, overall progression rates during AY 19/20 were unchanged from the previous three years but there is some indication that pass rates increased for BAME students during the pass / fail period. At the individual unit (module) level 43% of L4 units saw an increase in BAME pass rates versus only 23% of units having a decrease in pass rates.
At both UAL and Leeds Arts there are discussions about potential continued & wider use of pass / fail within the existing assessment frameworks and regulations, or whether those frameworks need to be changed in light of the research and this project.
In the context of this project our intention is to draw from what we have learned across two institutions to develop educational development resources to support institutions/individuals considering the introduction of pass/fail assessment.
Strand 2: Whole-self
“COVID-19 has been a wake-up call on crisis” (Thompson and Carello, 2021:2). We continue to live in a climate of collective traumas – a global pandemic, war and climate crisis – further compounded by prior trauma histories for some members of our educational communities (e.g., intergenerational, racial, adverse childhood experiences, etc.). This trauma frames educational experiences. As Thompson and Carello (ibid.) assert, this is a pertinent moment for Higher Education to “redress the impact of trauma”. When we speak of trauma, we refer to the definition by Substance Abuse and Mental Health Services Administration (SAMHSA) (2022), “an event, series of events, or set of circumstances that is experienced by an individual as physically or emotionally harmful or threatening and that has lasting adverse effects on the individual’s functioning and physical, social, emotional, or spiritual well-being.” What constitutes as traumatic varies person to person, as trauma is an individual experience.
What we know from neuroscience is that trauma has a direct impact on student learning; it impairs our ability to remember, communicate and learn as our brains are in survival mode (Imad, 2020). Thus, trauma-informed approaches to assessment are a way to help our community to thrive. As Higher Education institutions we can mitigate against these challenges and support all learners by becoming trauma-informed. We are not required to become mental health experts, rather to help our students feel empowered, safe, connected and hopeful (ibid.). This centres our shared humanity (Shevrin Venet, 2021) by acknowledging that we are feeling beings that think, not thinking machines that feel (Damasio, 2006). This also invites us to make a commitment to do no harm, and enact compassionate strategies that nurture students in their learning journeys, both responding to and preventing trauma.
We adopt an equity-centered lens to trauma-informed education, considering how oppression harms students and staff in policies and practices (Shevrin Venet, 2021). Thinking of this as an ecosystem of practice, policies, cultures and norms (ibid.), the aim is to use proactive strategies to address the inequities that cause and worsen trauma (ibid.).
This underpinning scholarship has informed our methodology.
What we have done
This strand analysed assessment regulation from three arts institutions to determine if the related documents were coded to communicate compassion for others, whether this be the students or the members of staff.
This work drew upon trauma-informed education as a lens of “understanding the ways in which crisis and trauma impact students and educators individually and collectively and using that understanding to improve” (Thompson and Carello, 2021:5). The aim was to conceptualise a possible ethical assessment policy based upon doing no harm as a principle of assessment, in a similar spirit to ‘No Detriment’ policies implemented during the Covid-19 pandemic.
The initial work led to a working research question: To what extent do the assessment policies of creative arts institutions acknowledge the interaction between assessment and students’ whole self, representing a compassionate approach to assessment?
The construct of the ‘whole-self’ is taken from behavioural psychology and has been applied by educational practitioners. In order to practise compassionate assessment, an empathic understanding is needed of the student’s wider social and emotional context, rather than understanding students as merely people to be measured and judged.
Two analytical tools were considered, Policy Archaeology (Scheurich, 1994; Huxtable, in press) and trauma-informed care (Hummer, Crosland, and Dollard, 2009). Assessment regulations were identified by the research group from three arts educational institutions. Each member of the research team read and applied the trauma informed care lens to the documents. Four fields were applied: connect, protect, respect and redirect. We adopted these from Hummer et al.’s (2009) principles of trauma informed care as a framework to explore assessment regulation and policies. These are interconnected and collectively provide a basis for trauma informed practices.
Connect – how do they support building and maintaining relationships within university communities?
Protect – how do they support building emotional, cognitive, physical and interpersonal safety?
Respect – how do they empower students and staff through choice and voice, promoting agency, sharing power and decision making?
Redirect – how do they encourage skills building and competence, enabling students to have positive futures?
As these notions originate from another discipline, we interpreted them within an arts educational context.
The research group conducted one stage of coding the regulations using a shared document. Then the resulting themes were subjected to a second stage of coding. This was an iterative process that also enhanced the interpretation of the four fields from an arts education perspective.
Our analysis suggests that although there are variations between the institutions there were some common regulative codes that were developed.
Firstly, the regulations often utilise punitive actions to manage the assessment process. Secondly, decisions about the assessment process are based on a notion of fairness and equality rather than equity. Thirdly, the students’ agency in the assessment process is not facilitated, nor is the student voice apparent in the regulations. This is further complicated by lack of clarity for students. Finally, signposting is lacking for students who need to salvage a failing assessment. This final point means that students are not encouraged to feel hope because the path to retrieving failed modules, years and even their courses, is not clear.
As a research group we are still to decide whether to relate our themes to a policy archaeology framework.
Explain how the ‘whole self’ approach intersects with issues of social justice and memberships of underrepresented groups.
Write up the theoretical and practice context.
Test out our themes on a group of academic staff and students to confirm and also to gain additional insights.
Develop quality indicators, which have been slowly forming during the project.
Develop a resource of reflective questions for policymakers
Strand 3: Feedback
This section of the project focuses on feedback as a critical area of practice where the emotional impact of assessment is at play (Falchikov and Boud 2007; Rowe, Fitness and Wood 2014; Winstone and Carless 2020) and where compassion may be enacted to support belonging. While the central role of feedback in students’ learning is well documented (Hattie and Timperley 2007), and guidance and support for staff for effective feedback are available across the sector (Boud and Molloy 2013), using compassion as the main driver of feedback has been less explored.
We have two key aims:
In collaboration with academic staff, develop definitions of what compassionate feedback might look like and devise guidance and support for approaches to compassionate feedback within the art and design disciplines. What advice might we give to ourselves, our colleagues, and students for a compassionate approach to feedback?
Explore with staff how existing structures and processes may be adapted and modified to enable compassionate feedback, bearing in mind the potential cost for staff of what E. Spaeth calls the ’emotional labour of feedback’ (Spaeth 2018), including issues of workload. The approach responds to Jan McArthur’s call to restore ‘joy’ in assessment (McArthur 2018) and make the assessment process both manageable and compassionate.
What have we been doing?
Through the initial scoping of the field, the project team identified a gap in support and provision for feedback guidance for staff that incorporates the principles of compassionate pedagogy. The team developed and delivered a collaborative workshop that asked invited participants from GSA’s five Schools to reflect on how we can support ourselves and our students in approaching feedback and feedforward, particularly at the formative stage. The workshop included framing presentations and two core discussion sections with the following guiding questions:
1: What is your understanding of compassionate feedback?
a) How do we define compassion?
b) In what ways can compassion be used in assessment?
c) What are the principles of compassionate feedback?
2: How can we enact compassion when giving feedback?
a) Can you describe what compassionate feedback would look like?
b) Can you describe what compassionate feedback would feel like?
The workshop focused on staff experiences of the positives and challenges that the assessment process can bring. Participants reflected on the advice we would give to ourselves, our colleagues, and students regarding the formative assessment process and its learning, including in our current landscape of hybrid modes of feedback and assessment delivery. With compassionate approaches critical for staff experience, we explored what guidance we might need to make this as manageable and compassionate as possible and enhance the sense of belonging in the student learner journey. What basic principles to define a compassionate approach would be most helpful to identify?
This first-phase research afforded us an understanding of the appetite, enthusiasm, and deep engagement with which staff participants recognised this reflective investigation. Our initial analysis summarises the insights participants raised according to three interrelated themes:
Defining and understanding compassion
With education as an act of love (Freire), belonging is understanding: students need to understand that their educators understand their learning journey. Interpretations of compassion include: mirroring so that the other feels ‘seen’; active listening; honesty and transparency; understanding individual requirements for learning, and awareness of diverse experiences. Assessment creates artificial structures dealing with non-human elements, including grades. More compassion built into the structures would be key, and acknowledging our own power position. Our pedagogy should suggest and guide rather than dictate.
Journey: how compassion can be used in feedback for assessment
Feedback for assessment is a durational learner journey, where formative feedback offers powerful constructive affordances. Given the value of feedforward, would compassionate approaches towards students and staff workloads place most value on the formative moment and frame the summative (albeit with a feedforward element) as more “contained”? Formative feedback might acknowledge past learning, and there might be value in ‘unlearning’ past educational experiences. Engaging students in assessment (e.g., self- and peer-assessment, co-creation) and ‘de-coding’ assessment structures (e.g., transparent constructive alignment) would inform mutual understanding. Feedback for assessment should be a two-way conversation and process recognising environment, tone, and language, a relational and dialogical ‘done with’ rather than ‘done to’.
Whole Self: student and staff experience
In this framework of reciprocal conversation, students should feel empowered to recognise their work and their journey. Here, empathy, sensitivity, and the ability to understand others and recognise risk-taking and obstacles lead to a conception of the ‘Whole Self’ in the experience of feedback for assessment. Some of the challenges include talking about the work, not the person, but recognising the person in the work; and interrogating how the role of the pastoral (e.g., personal tutor system) helps students feel “seen”. Reflecting on the impact of the pandemic and triangulating assessment with institutional systems, compassionate approaches also have a deep linkage to staff workload, given the value of time and the durational.
Our reflective review of this material and extrapolation of key themes form the basis for upcoming presentations at Learning and Teaching (L&T) [CL1] conferences at GSA and UAL. These presentations are opportunities to extend the exploration with the broader communities of the two institutions and to use feedback to refine the approach and incorporate insights.
We will use this analysis, reflection, and dialogue to develop and frame a subsequent reflective focus group event (UAL and GSA staff) to provide a comparative set of information and test adjustments to the co-creation process. We will review and evaluate this second co-creation exercise towards developing a draft resource for utilising the principles of compassionate pedagogy in feedback for assessment. To further enhance the proposed guidance, we plan to test this resource with GSA and UAL colleagues and discuss it with students. We see the development of this resource as a critical tool for learning and teaching communities to inform the evaluation of how we might enhance belonging, compassion, and understanding in the process, language and policy of feedback for assessment, and our work sits in parallel to current GSA L&T [CL2] enhancement work and UAL academic enhancement work on assessment and feedback.
Reflection on our collaborative approach
As a project team, we have identified common ground between the three arts institutions that have different languages and cultures around assessment. Our project work on belonging and compassionate assessment is becoming part of the culture at our institutions and is discussed frequently in different fora and embedded within staff development. We recognise both the personal and professional benefits of making connections with like-minded academics across institutions to strengthen assessment culture in the subject and across the higher education sector. The collaboration has allowed us to capture and build the ‘zeitgeist’ in relation to the thinking and practices around assessment. Our collaboration is leading to relationships that can be sustained beyond the project. We are building a rigorous foundation of scholarship that can drive future research in assessment literacy. Within our project team, we have experienced mentorship, as we learn from each others’ diverse experiences and find richness from commonalities and differences.
The project has been presented both within our institutions and externally, extending discussions with colleagues across the sector through committees, conferences, podcasts and more. Below is a list of outputs to date.
Assessment in Higher Education conference as part of paper on ‘Challenging Assessment Habits’ Dr Emily Salines, Cath Caldwell, Siobhan Clay (Manchester, June 22)
Do No Harm: Bringing compassion, joy and social justice into assessment, SRHE, Vikki Hill with Dr Jan McArthur (14 June 2022)
Enabling Compassionate Assessment – Do No Harm, Quality Insights conference, QAA, Vikki Hill (24 Feb 2022)
Educational Res seminar, Lancaster University (9 Feb 2022)
QAA Evolving Student Engagement Conference 2021, Liz Bunting & Dr Emily Salines (30 June 2021)
Belonging through Assessment, Authentic Assessment in a post-Covid world, QAA Annual Conference, Vikki Hill (13 May 2021)
UAL Academic Board (project update) Liz Bunting, Vikki Hill (10 March 2022)
UAL Education Conference, Dr Emily Salines (July 2022)
Leeds Arts University LTE Conference 2022 – The Inclusive University (Project update) Prof Sam Broadhead, Peter Hughes, Dr Laura Da Costa (24 June 2022)
GSA Learning & Teaching Conference, Dr Marianne Greated, Dr Thea Stevens, Robert Mantho (June 2022)
UAL Quality Standards Committee (pass/fail presentation) Vikki Hill, Dr Neil Currant (30 Nov 2021)
Belonging Through Assessment: Pipelines of Compassion, Leeds Arts University Learning, Teaching and Enhancement Conference 2021 – Decolonising the Curriculum (Project Update) Dr Laura Da Costa (25 June 2021)
Damásio, A. R. (2006). Descartes’ Error: Emotion, Reason and the Human Brain. 2nd Edn. London: Vintage Books.
Harland, T., Mclean, A., Wass, R., Miller, E. & Nui Sim, K. (2014) ‘An assessment arms race and its fallout: high stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education, 46 (1), pp. 105-117.
Hummer, V. L., Crosland, K., and Dollard, N. (2009). Applied behavioral analysis within a trauma-informed framework. Presented at the Florida Center for Inclusive Communities “Lunch n Learn” Series. Tampa, FL.
Huxtable, J. 2022 ‘Performance’ measures as neoliberal industrialisation of higher education: A policy archaeology of the Teaching Excellence Framework and implications for the marginalisation of music education. Broadhead, S. (Ed) The Industrialisation of Arts Education. Palgrave Macmillan. (in press)