Can Mentimeter Enhance University Student Participation in Taught Sessions?

There has been much research looking into the link between active engagement and learning in higher education students. This article presents an action research study that was conducted using a qualitative method to evaluate the impact of a specific technology on the learning for a cohort of students in a mandatory placement briefing session. Mentimiter, a classroom response system, was used to facilitate and evaluate the engagement of a 3 rd year cohort of occupational therapy students in a teaching session. The findings are that, in this instance, the technology was effective in enhancing the students’ engagement and subsequent learning.


Introduction
A successful experience for students in higher education involves more than just mere attendance and requires the students to be actively engaged in their learning (Rands & Gansemer-Topf, 2017).There has been much written about the relationship between engagement and attainment of students in higher education; for example Lee (2014) and Bovill et al. (2011).There are a wide range of factors which influence students' engagement and achievement in higher education, such as boredom (Pekrun et al., 2014) and academic self-efficacy (Kahu & Nelson, 2018).
Increasingly, technology is being used in higher education teaching and learning (An & Reigeluth, 2011) and these developments are influencing and shaping pedagogical approaches (Jackson-Barrett et al., 2019).Research has considered the role of technology in a range of higher education contexts, such as supporting blended learning (Ibrahim & Nat, 2019); the study described here looks at the role of technology in enhancing engagement.
Practice placements are an integral part of occupational therapy degree courses and placement is where theory and practice are synthesized (Finch, 2017).A mandatory part of the preparation for practice placements for occupational therapy students is attendance at a placement briefing.This briefing provides the students with a lot of information about the placement, such as the typical working hours, methods of assessment and learning opportunities, expected level of performance and guidance on how to complete the related documentation.Other areas discussed include the responsibility and role of the student on the placement and what support and input the student can expect from their practice educator to facilitate their success.The briefing also provides the students with opportunities to discuss how they are feeling prior to their placement and consider how to prepare for the placement experience.
The author's experience has shown that when students are on placement following the briefing, many of them make contact via telephone or email and seek information that has been provided or discussed in the placement briefing.Common examples are questions about the use of study time and how to write placement learning objectives.On reflection, it takes a lot of time to respond to all these messages and provide the information or support that the students are seeking.Other implications of this contact are that it can cause anxiety and lack of confidence in the students and may prevent them from fully engaging in their placement experience.
Historically, the briefing has been delivered without any formal checking of whether important learning has taken place.This study was developed to inform the delivery of the placement briefing and included a specifically designed process to check the important learning of a cohort of students and review the amount of contact made by this cohort once they had commenced the subsequent placement.The checking of learning was facilitated using Mentimeter; a classroom response system that uses an app to generate real time feedback.Classroom response systems are designed "to increase active engagement of students by their response to a question," (p.5; Bachman and Bachman, 2011).Engagement in higher education has been well researched and there are many factors which have been identified as impacting on students' engagement in lectures and seminars.Among these are large cohort sizes (Mulryan-Kyne, 2010) and cultural background (Leese, 2010).Some studies have found that students do not feel confident enough to speak up in some lectures and prefer to remain anonymous within the group (Sawang et al., 2017).It is important for pedagogies in higher education to evolve and adapt and provide the most effective education for the students (Weller, 2016).Part of this involves using appropriate technology to support students' engagement and learning (Rashid & Ashgar, 2016).Many studies have looked at the role of technology in enhancing students' engagement in learning, such as Heflin et al. (2017).In addition, Beetham and Sharpe (2010) explain that digital technologies can create a new context for learning and teaching.
This study trialled the use of a technological enhancement to the pedagogy to evaluate if there is a role for this type of technology in enhancing student engagement in a lecture setting, as it has been recognized that digital technologies increase opportunities for active engagement (Weller, 2016).The objective of the study was to evaluate the role of Mentimeter in measuring the outcome of the engagement of occupational therapy students in a placement briefing session within a lecture environment.

Methods
The study took place with a cohort of 3 rd year occupational therapy students and was based on their placement briefing, which is a mandatory part the preparation for the students' practice placement.The students were on a full-time degree course leading to professional registration, and practice placement makes up a significant part of the curriculum in such courses (Polglase & Treseder, 2012).The mandatory placement briefing provides students with a range of information about their placements and gives an opportunity for students to ask questions as part of the process of preparing for their practice placement.The placement briefing was delivered at the halfway stage of the third year of the course, and six weeks before the commencement of their final placement.
This study was conducted using an action research approach, which is "… about practitioners creating new ideas about improving their work," (McNiff, 2017, p.7) and lends itself to this study because it is a method used for improving educational practice (Koshy, 2011).Another feature of action research is that it views the methodology as open ended (McNiff, 2017) and therefore supports the approach used in this study; Etherington (2004) notes that our own research can influence our practice and that of our students.This study has used technology to facilitate the data collection stage and the use of this technology will be evaluated to address the objectives of the study.
A qualitative method of data collection and analysis was used because it can generate findings that can improve practice (Lester et al., 2020), which is one of the aims of this work.The study design was to use a single question at the beginning and end of a placement briefing session and check for the difference in the responses to the question.The question used was 'How do you feel about your forthcoming placement?'.The research was aimed at identifying any change in response after the participants had been given key information about their placement and had the opportunity to ask questions and clarify key details that they needed to know.This would be an indication of the quality of learning from the session.The link between technology and pedagogy is an important one (An & Reigeluth, 2011) and informed the method of checking the learning of key pieces of information by using a questionnaire which was also conducted using Mentimeter.
The use of questions being posed and the responses being displayed on a screen in real time were the data collection methods.The students did not know the questions that they would be asked prior to the session.
The ethical considerations most associated with action research have been discussed by Nolen and Vander Putten (2007) and are considered to include the consent of the participants and the autonomy of the participants.Researchers have a responsibility to the participants, the organization under whose name the research is being conducted and their profession and colleagues (Etherington, 2004).To this end, ethical approval for this project was provided by Canterbury Christ Church University Research Ethics Committee.
The population for this study were final year occupational therapy students, though data gathered from first year occupational therapy students on the same programme was used as part of the data analysis process.This was to provide a comparison figure from the first years, who had not had the questions and Mentimeter use in their briefings, regarding the amount and type of follow up contact from the cohorts when on placement.In line with the work of Cleary et al., (2014), the participants were selected because of their personal experience or knowledge of the topic under study.The aim of this action research project was to evaluate the potential impact of technology on students' learning in the placement briefing and, therefore a convenience sampling (Etikan et al., 2016) approach was taken to identify the participants.The students had been made aware before the session that their answers to the questions would contribute to the data for this project.They were reminded of this at the beginning of the session and the option for them to not take part in the data collection was restated to them, in line with the recognized ethical practice within action research (Nolen & Vander Putten, 2007).The project was planned to take place when the placement briefing was due as this would allow for the data to be gathered and then the contact from the students during the subsequent placement would be evaluated to identify any difference in type of questions asked.This cohort of third year students were preparing for their third and final placement and had already attended the placement briefing in the first and second years of their course.Most members of the population can be described as Millennial students (Toothaker & Taliaferro, 2017), which means they are aged between eighteen and thirty-four years old and may learn more easily with non-traditional pedagogies (Lawter & Garnjost, 2021).For the purpose of this study, the placement briefing was enhanced with the Mentimeter based questions to evaluate the impact of the technology on engagement and checking of key learning within the session.
One of the main aims of the study was to evaluate the impact of the use of the Mentimeter facilitated questions on engagement within the session.The use of digital technology as the medium for the students to answer the questions presents issues of inclusivity.These issues can be addressed by considering the pedagogy and the content of what is being taught (Danowitz & Tuitt, 2011).One of the challenges of this type of study method is that some students may be under financial pressure (Holley & Oliver, 2010).This may mean that they may not be able to afford to own a smart phone or computer tablet, which would be required to take part in this study.To address this aspect of inclusivity, the parameters of the data collection questions were set to allow an individual's device (such as mobile phone or computer tablet) to record up to three responses.This allowed students to share each other's equipment.
Data for the study was collected during the placement briefing and a consent form was provided to all the students to complete prior to the briefing to give them the opportunity to give their consent to engage in the study.The students were made aware a few weeks in advance that the data would be gathered in their placement briefing.The aim of the study was explained and contextualized as part of the PGCAP programme.It was made clear that attendance at the placement briefing was mandatory, but engagement in the study was optional for the students.
At the beginning of the placement briefing session the first question was posed, which was 'How do you feel about your forthcoming placement?'.Time was then given for the students to consider this and provide their answers.The answers were briefly discussed and then the placement briefing was delivered, including essential information such as the procedure for reporting absences and how to complete the placement documentation.
The participants were a cohort of forty-nine third year occupational therapy students.The students were all in the final year of their degree programme and were preparing for their final practice placement, which has a duration of twelve weeks.There was a mix of male and female members within the cohort and a wide age range.This demographic presentation is typical of the cohorts on this academic programme within this university.
The data collection tool was the questions and a short three question quiz.The questions for the data collection tool were selected by the author.The initial question which was posed to the students at the beginning and the end of the session was drawn from a question that has been used to begin the placement briefing discussion with student cohorts across all three levels of the degree course over recent years.Mentimeter is an interactive classroom response tool which allows responses to set questions to be displayed on the scene at the front of the room anonymously in real time.The students' responses to the questions and the quiz questions were displayed on the large whiteboard at the front of the lecture theatre in real time.Further responses were added to the word cloud for the questions and the bar charts for the quiz, as more students provided their answers.The students were asked to provide a brief, preferably single word, answer to the question 'How do you feel about your forthcoming placement?'.The data was collected through the use of a series of questions using Mentimeter.The students were shown a Mentimeter code to enter into their mobile phones or other devices to enable them to provide their answers, which appeared in real time on the screen in the lecture hall.
At the end of the briefing, the students were asked to complete a short quiz consisting of four questions.This was aimed at checking if they had learned some key facts and information from the mandatory session.The questions were about who should sign their placement documentation, characteristics of good learning objectives, how many hours they would be expected to complete each week on placement and how to report any absences.These questions were asked in a multiple-choice format and the students were given a choice of three possible answers to choose from for each of these four questions.One of the options was the correct answer and the other two options were deliberately wrong.This was to support the students to focus on the correct information that they needed to learn from this session in order to support their performance on their placement.After the students had completed their answers to the short quiz, they were asked to answer the original question again-'How do you feel about your forthcoming placement?'.
The data was analysed in two stages.Firstly, the data that was gathered in the placement briefing was analysed by comparing the types of responses made by the students at the start of the session to the comments provided at the end of the session.This gave an indication of the success of the session in helping the students to be prepared and informed about their forthcoming placement.The second stage of the analysis was a review of the amount and type of questions that came from this cohort during their placement.This would then be contextualized by reviewing the type and number of questions that came in from the firstyear cohort who had attended their briefing but had not had the learning checked using the Mentimeter questions.This would provide a comparison and allow the efficacy of the Mentimeter intervention to be reviewed.

Results
Of the students present, thirty-five contributed answers to the question at the beginning and thirty-three contributed to the question at the end.This represents a response rate of 71% and 67%, respectively.This is a high response rate, and a positive response rate because the majority of participants are represented in the presentation of data, (Cleary et al., 2104).See Figure 1. for details of the responses to the question.The responses to the first question 'How do you feel about your forthcoming placement?' were varied and included some positive statements, such as 'excited' and 'ready', as well as some more negative replies, such as 'apprehensive' and 'daunting'.Some of the replies were more specific, such as 'setting is good' and one respondent recorded 'transport is the issue', which related to the daily commute to and from placement.Thirteen of the responses were seen as positive ('ready', 'happy') and fifteen were viewed as negative responses ('daunting', 'terrified').Seven of the responses were harder to classify in these terms (positive or negative) and included responses such as 'Christmas', 'in the future' and 'odd'.
The thirty-three responses to the second question also contained a range of answers, though there seemed to be a greater number of positive responses than to the same question posed at the beginning of the session.See Figure 2. for details of the responses to the question.Seventeen of the responses were considered to be positive and included replies such as 'good' and 'bring it on', as well as more concrete replies, such as 'prepared' and 'enthused'.Five of the responses were negative and included 'scared', and 'confused'.There were also a number of responses stating 'same' and 'still interested' and 'similar'.These responses covered both positive and negative attitudes, for example, 'still happy' and 'still anxious'.At the end of the briefing and before the students were asked to respond to 'How do you feel about your forthcoming placement?' for the second time, they were asked to answer a series of questions which were related to key pieces of information from the briefing.The questions were 'documentation-who should sign your documents to verify their accuracy?','learning objectives-your learning objectives should be…', 'hours-how many hours should you complete for each week of placement?' and 'reporting absence-in order to report absence from placement, such as sickness, you should…'.These questions also achieved a very good response rate of thirty-one, thirty-three, thirty-two and thirty-two, respectively.This equates to 63%, 67% and 65%.These four questions each had a series of three options as the answer.One of the options was the correct answer, as discussed in the briefing and was a key piece of information that would support the students in their placement.The other two possible answers were both deliberately wrong and designed to direct the students to the correct answer to enhance their learning.See Appendix 1 for the full details of the students' responses.The question about hours produced twenty-eight correct responses, which was 87.5%.There were twenty-three correct responses to the question about reporting absence, which equates to 71%.The question about documentation was answered correctly by twentyfour of the students, which was 77% and twenty-nine students gave the correct answer to the question about learning objectives-87%.

Discussion
By comparing the responses to the question about placement posed at the beginning and end of the placement briefing, it can be seen that there was an increased number of positive responses at the end of the session.There were also some responses that were more negative in tone, though this number had reduced from the responses given at the beginning of the session.The brief quiz at the end of the placement briefing, which was designed to test the learning of key information, yielded very positive responses, ranging from 71% correct answers to 87.5% correct answers.This would suggest that the technology used, and the manner in which it was trialled, had enhanced the students' engagement and learning in this session.The reason for only 71% of the students stating that they knew the correct answer could be due to the fact that two of the options were deliberately incorrect and intentionally humorous, and some of the students may have chosen to select an incorrect answer intently, picking up on the informal delivery style used in the session.The traditional, didactic method of teaching and learning in higher education has been well researched and it is considered that active learning is essential to reach Millennial learners (Roehl, Reddy & Shannon, 2013).This has been supported by the outcomes of this study.This method allowed anonymity, which helped to address some of the issues around lack of engagement, such as students not feeling confident to speak up in lectures due the size of the group.A study by Nash et al., (2016) looked at similar issues when they sought to measure the anxiety students experienced with regard to a public speaking assessment.
In order to evaluate the effectiveness of the learning through enhanced engagement which was the aim of the study, student communication regarding their placements was monitored to provide further anecdotal evidence.Though anecdotal evidence has been identified as one of the least reliable sources of evidence (Ingham-Broomfield, 2016), it is consistent with this study design as this type of evidence was one of the motivators for this research.It was found that during the subsequent practice placement (for which this briefing was part of the preparation), there were fewer questions and concerns raised by this cohort in relation to placement documentation and hours, for example, than had been the case during this cohort's previous placement experience and also from the previous third year cohort.This could be interpreted as suggesting that the use of Mentimeter to enhance the students' engagement, and the use of the quiz to check essential learning, had been successful.In contrast, there were a great deal more questions regarding completion of documentation and setting placement objectives, for example, from a first-year cohort who were on placement at the same time.The greater number of questions and contact from the first-year students may be because this was their first placement experience and because the questionnaire to check their learning was not used in their placement briefing as it had been for the third-year students.When considering the frequency and nature of the students' making contact for support, it is important to bear in mind that placements are stressful (Arielli, 2013) and that a certain amount of contact from the students may be inevitable.
Previous research has looked at the use of technology to enhance engagement (Heflin et al., 2017) and the link between engagement and learning has been well established (Bainbridge & Houser, 2015).The findings from this study would appear to support the notions that there is a role for technology in enhancing students' engagement in higher education and demonstrates the link between engagement and learning.
Among the implications for future practice is that the inclusion of the checking of learning in the placement briefing, as a routine item, may lead to further cohorts of students engaging in placements feeling more prepared.One of the recommendations from Davies et al., (2017) from the Higher Education Policy Institute, is that digital technology should be recognized as a key tool for higher education institutions responding to the Teaching Excellence Framework (TEF).This situates the incorporation of digital technology into current higher education policy; this has informed this study and contributed to by the outcomes discussed.
The use of questions in this context presumes that people have ready answers (Gillham, 2010).This is perhaps an indication of the researcher taking the students' participation in the study for granted, though it also validates the sampling method used.Additionally, much has been written about the researcher-researched relationship, such as the work of Råheim et al., (2016).Though the cohort in the study were identified as an appropriate sample (Cleary et al., 2014), their familiarity and the nature of their relationship with the researcher may have influenced the very high response rates and the incorrect answers to the four-question checking quiz used at the end.However, these high rates, which are viewed as a strength of this study, could also be attributed to the briefing being a mandatory part of the placement preparation process and being an essential forum for gathering key information relating to their forthcoming placement.

Conclusion and Reflection
Overall, it has been demonstrated that the use of digital interactive technology enhanced the students' engagement in the session.This was reflected in their learning and the quality of their subsequent placement experience.
Reflection is an integral part of action research (Ghaye, 2011) and reflecting on the process of completing the study and the influence of this on the author's future teaching practice has produced some valuable insights.Engagement is complex and multifaceted (Kahu, 2013), but this study has attempted to use technology in a clearly defined context to attempt to address some of the negative aspects associated with a lack of engagement.Among the challenges of this research were working with a population who were familiar to the researcher and learning how to use a piece of technology in order to incorporate it into practice.Through the process of undertaking this research, it has been demonstrated that engaging in research can improve practice (Ghaye, 2011) and further studies could be conducted to evaluate the benefit of introducing these methods more routinely into the practice briefing.
Among the limitations of this research is that the results lack generalisability.However, this study was conducted using an action research approach and though a lack of generalisability has long been a criticism of action research (McNiff, 2007), Koshy (2011) argues that the outcomes of action research may be of interest to those who wish to apply them to similar contexts.

Figure 1 .
Figure 1.Responses to the question 'How do you feel about your forthcoming placement?'

Figure 2 .
Figure 2. Responses to the question, 'How doe you feel about your placement after placement briefing?'