In this good practice, we present Universidade Aberta´s pedagogical approach to assessment design. Created in 2007 and based on its pedagogical model, this approach has been validated and used throughout the years with success. The approach is introduced both in terms of its pedagogical reasoning and in terms of how it informs regulations and practices. The chapter then moves to reflections about the challenges and opportunities that Covid19 brought to the institution culminating with our thoughts about the future and how we can adopt an institutional approach for moving the assessment fully online.
The Present
The Universidade Aberta is the only state-funded distance learning higher education institution in Portugal. Although it has now 33 years old it only became fully online in 2008 with the development and implementation of its virtual pedagogical model (VPM) (Pereira et al., 2007). Building on existing best practices in online and distance learning, the VPM has become, throughout the years, the main foundation of how the University operates both in terms of the course units design and development choices as well as the delivery and instructional practices. The VPM is oriented in 4 main principles: student-centred, the flexibility of choice, interaction, and digital inclusion.
Included in the VPM, there are a set of guidelines and rules supporting assessment design and delivery. The first main rule is that all course units must allow a degree of choice in terms of the assessment. Students can choose from being continuously assessed – in which they will have to submit at least three elements per semester per course unit, or through a final exam - when the student chooses to be assessed in a single moment and typically through written tests. Exams were typically done face-to-face in local exam centres across the country and abroad. The UAb has historically supported this flexible option of assessment because of the need to respond to the principle of flexibility, one of the guiding principles of the VPM (Pereira et al., 2007). Continuous assessment elements (also known in UAb as e-folios) have always included a formative element in the form of feedback. As part of assessment design, course unit lecturers aim to integrate the three elements of assessment to enable a sense of relevance in the feedback provided for future assessments. This is the element of single-loop feedback proposed by Carless (2019) that represents how students learn when they work on assessments as part of their course units or programmes of study. For the author, a loop implies an endpoint of the feedback; it can be at an assessment/course unit level (single-loop) or at a course level (double-loop), which would represent a multitude of opportunities in different assessments; conversely, a spiral would imply feedback that is more ongoing and developmental (Carless, 2019).
For the UAb model, the single-loop feedback is the approach taken due to both the student background and the time they take to do their course. UAb students have an average of around 40 years old, they are typically working students and they have restarted their education after a long period of pause. Students will usually engage in their learning as part-time students taking an average of 5 years to finish their undergraduate degree. All of this suggests that UAb students prefer to receive feedback that has an immediate impact on the next assessment. Underlining the commitment of the relevance given by UAb to continuous assessment, the student can use individual e-folios marks as credits so that they can be used in the following academic year (). Annually, about 150,000 assessment elements are submitted to the UAb e-Learning platform with various types and formats that serve to fulfil the basic requirement of the pedagogical model. To complete a degree at the UAb, a student who chooses continuous assessment will have to take at least 90 elements of assessment - there are 30 course units (10 per year) and within each course unit at least 3 elements of assessment (Mendes, et al. (2019).
As with the exam option, and before the pandemic started, the students that chose the continuous assessment path would also need to take a face-to-face assessment which would come after the two continuous e-folios and normally at the end of the semester – the global assessment. Importantly, students could only take the global assessment if they surpass the two continuous assessments with higher a mark higher than 3.5 out of 8 (figure 1).
Although it may be argued that UAb is over assessing students, this approach was taken because of three main reasons.
• Assessment points are seen as important to signpost to students what they need to learn and how they are doing in each course unit.
• Assessment points allow us to monitor students´ learning and ensure that dropout risks are anticipated.
• It creates a robust assessment ecosystem which was seen as important due to the lack of credibility given to distance education in the country in 2008 when the model was adopted.
This approach links to how the university operates both in terms of regulations and in terms of its systems and workflow. For example, assessment dates and criteria are validated in the course unit guide and in the learning card. The learning card includes the structure of the assessments (the two e-folios and the global assessment) with the corresponding potential marks (figure 1). Those are validated annually by the central department to ensure that the pedagogical model adopted is integrated into each course unit master instance in Moodle. After validation, the different cohorts of students are then generated in each Moodle instance, and assessments in all different instances cannot be changed anymore during the semester. This approach enables the University to transport all marks from Moodle to the Student Academic System in one go using an automatic workflow as all cohorts will be using the same structure. This consistency in assessment is beneficial for the student and the lecturer (it allows transparency and consistency in the assessment processes) and for the institution (it facilitates digitization and efficient and error-free processing of grades in the Academic Portal).
Looking at the future
Over the years this logistical organisation has not been immune to the existence of some obstacles and challenges, particularly with students who live abroad or in regions more distant from urban centres and who must go to distant examination centres, which are not always easily accessible, and which therefore entail associated costs of travelling to take exams. This is an area that the UAb has been discussing with the Portuguese Secretary of State for Communities, but it has not yet been possible to create alternative mechanisms for holding exams to the frustration of students that live abroad.
Covid19 brought us challenges with the digitisation of the assessment process but opened-up new opportunities. As with other institutions, we needed to move swiftly all face-to-face encounters with our students. For a traditional university, this would imply teaching, support, social activities, and of course assessment. For the UAb the main challenge was the global assessment and the final exams. Those were mainly done in exam centres that needed to close during lockdown which coincided with assessment periods. The University needed to move to online submissions of exams and to adapt to new challenges. That of course brought challenges in assessment designs particularly as some were written exams, but also provided a range of new opportunities to fully respond to our principles of a student-centred institution and flexibility of choice. Students supported this option and enjoyed this new approach. Lecturers were able to adapt to new assessment strategies building new forms of assessments and the average grades across different courses didn’t change significantly.
We are now looking at making a final decision of moving completely online including for students that chose the final examination path. That however will require some form of online proctoring, identity certification, staff development, new regulations and data protection policy and, of course, working side-by-side with students to develop new approaches to online assessment.
As an institution, we are committed to our principles, and flexibility is a key element of distance education and one that our students are particularly fond of due to their background. Having a fully online assessment experience will ensure they can do all their learning in their home or in their place of choosing. Providing that we can ensure that the assessment is robust and trustworthy, we believe we can align ourselves to our principles and to our students’ expectations.
