City of Wolverhampton College
Piloting the digital experience insights service
Use of the insights service is a must. It helps you to validate what you are doing, identify areas of strength and where you can improve. The insights data enables me to be responsive and to work towards providing the best possible experience for learners and staff.Project lead: Conrad Taylor, e-learning manager, quality, City of Wolverhampton College
Project aims: evidence-based decision making
City of Wolverhampton College wanted to take students views into account when making decisions about the development of the digital strategy and the environment and resources they provide. The college wanted to know the impact of initiatives already in place and whether it was achieving a positive return on investment.
The evidence gained from taking part in the 2016-17 pilot of the student digital experience insights service allowed project lead, Conrad Taylor, to address wifi blackspots across their five campuses, to invest in further resources for the library study hub and in loan equipment for students to use on and off campus.
In 2017-18 the college decided to participate in both the student and the staff digital experience insights surveys pilots to better understand the views of students and staff and to improve how well technology is embedded in learning and teaching.
905 students responded to the survey (response rate of 43%) and 70 academic staff responded to the staff survey (response rate of 35%).
Strategies for engaging students and staff
The high response rates from both students and staff were achieved by adopting a clear communication strategy that involved tutors and managers.
In terms of student engagement, the pastoral care team were an important audience to get on board as all full-time learners have regular, scheduled pastoral care sessions. Pastoral care tutors were informed that the survey was taking place, its purpose and why it was important for students to take part. They were tasked to share the survey link and make time to discuss it with students throughout the survey period; to highlight the benefits and to emphasise how valuable the students’ contribution is in informing the college’s strategy and purchasing decisions. Pastoral care sessions take place in spaces which have computers and internet access which facilitated students completing the survey during the sessions, but independently from tutors.Tutors of part-time learners were also informed of the survey and were asked to encourage their learners to participate and share the survey link via email.Running the staff insights survey required a different approach to that used for the student version as staff are always busy supporting learners and preparing for their next lesson. The initial approach was to send an email to academic staff highlighting the reasons for the survey and the benefits of completing it to them, their learners and the college. Responses were slow during the first few days, so the college sent email reminders to all academic staff at the start of each week during the survey period and also asked managers to remind staff of the benefits.In addition, the surveys were promoted via notices on the student and staff virtual learning environment (VLE), intranet, newsletters and posters around the college.
The guidance and responsive support from the insights service team and the templates provided, helped Conrad to summarise the main points quickly and to share the findings with staff and learners. The customisable posters provided by the insights team will allow the college to feedback the findings to learners and inform them of actions being taken. It is hoped this will ensure that students see the value of participating in the survey in future years.The national benchmarking data is helpful in identifying where the college is doing well and where it can improve.The college added four customised questions to the survey to help them to identify:
- The curriculum area
- The level of study
- Which of the five campuses the learners were based at
- Whether learners felt that their teachers were using technology effectively to support their learning, both in and out of the classroom
It’s great that institutions participating in the digital experience insights survey have the opportunity to customise a section of the survey to fit their needs. So whatever VLE you’re using, you can customise the survey to fit your needs.Project lead: Conrad Taylor, e-learning manager, quality, City of Wolverhampton College
As of August 2018, the main data analysis has been that of the student feedback which shows that:
- 78% of students at City of Wolverhampton College rated the quality of digital teaching and learning as being good or above. This is a 3% improvement from the 2016-17 figure.
- 33% of learners said that they would like more resources to be made available outside of the classroom via the VLE; and that they want a more flexible approach to learning with 24/7 access.
- There is perhaps a greater demand for assistive technologies than the college had anticipated.
- In common with national benchmarking statistics, there is a gap between the number of learners who recognise digital capabilities as important to their future careers and how well-prepared they feel for this. It is also interesting to note that 8% of learners have asked the college to ensure all courses have a dedicated time slot to improve their digital skills.
- 95% of students say that they feel safe online – validating the previous practice of including e-safety support in pastoral sessions and via online learning modules.
- A small number of learners on specific courses felt that the equipment and resources they were using was not of an appropriate industrial standard. At 4%, this is a low figure, traceable to specific curriculum areas by including customised questions in the survey.
- 17% of students felt there were still issues in accessing wifi in some areas.
- 64% of students said they would like technology to be used to the same extent as is currently the case, while 30% of students said they would like technology to be used more. This is interesting as the college has purposefully taken a considered approach to using technology as part of a varied teaching and learning strategy; working to ensure balanced and purposeful use.
I think it is really important that other students can get involved with the conversation around digital technologies because technology is so widely used everywhere in every job role.Josh, student at City of Wolverhampton College
Although the staff survey data is still being analysed there are some interesting findings emerging:
- Very few staff discuss teaching with their peers via an online network or forum
- 46% of staff felt that they do not have time to innovate
- Overall, staff rate the quality of the organisation’s digital provision as good or above, but on some campuses the level of satisfaction is lower and staff feel that the equipment is not at the right standard or is too slow. Many of these issues had already been identified by the IT team and were already scheduled to be addressed as part of the summer network upgrade.
Acting on the findings
Feedback like this is allowing us to target resources where they are most needed.Project lead: Conrad Taylor, e-learning manager, quality, City of Wolverhampton College
The structure of the survey and the ease of analysing the data is enabling the college to identify quick wins and respond promptly to the findings. This is important as it means current students can see that they are being listened to and that action is being taken on the issues that matter to them.
Actions already in progress in response to feedback from students include:
- Working with curriculum managers and curriculum teams to:
- Identify more opportunities for digital activities within the curriculum and within learning activities through schemes of work and lesson plans
- Increase opportunities to develop the digital skills of students within course activities
- Sharing the student survey feedback with staff and showcasing the benefits of using some of the VLE tools during the July continuous professional development (CPD) week.
- Highlighting the digital resources already available on the VLE to students. These include resources from the Blended Learning Consortium and resources to support e-safety, literacy, numeracy, employability and PREVENT.
- Discussing accessibility and inclusion issues with the e-learning manager, the special education needs and disabilities (SEND) manager and the IT manager. The college is also liaising with their Jisc account manager to identify what further support Jisc can offer.
- Ensuring that, from September 2018, all learners on full-time courses receive a 45 minute workshop on digital aspects of learning within the first six weeks of their learning journey with the college. This will be provided by the Technology Enhanced Learning (TEL) team who will also offer further 1:1 and micro sessions throughout the academic year.
- Reviewing software and equipment against industrial standards – this is taking place over the summer recess.
The data from the insights service has helped to drive change in a variety of ways. We listen to the learner voice on this. They tell us what sort of resources they want to see and we try to respond to that. We have been working quite closely with the IT department to upgrade resources and to increase the numbers.Rose Edwards, learning resource manager, City of Wolverhampton College
In responding to early findings from the staff survey the college is:
- Exploring how well opportunities and support for staff to develop their digital capabilities are promoted. The TEL team presented their survey findings at CPD events in July, relaunching the support offer and making sure staff are aware of the service and the options open to them.
- Working with the IT manager to ensure the cycle of improvement for software, hardware and the learning environment addresses any issues identified over and above that already scheduled.
The data helps me to prioritise staff training. We look at what the learners have asked for and at what staff require to get to that point. For the learners to be happy then we need staff to have the right skills to deliver that within the classroom.Janine Magee, quality assurance, teaching and learning manager, City of Wolverhampton College
Tips for others and lessons learned
- The communication and collaboration between other institutions participating in the digital insights pilot was helpful and highly valued. In particular, as project lead, Conrad found it interesting to see ideas put forward by others as to how and why they included customised questions. Some of the ideas gained from the community will be adopted in the college’s next launch of both the student and staff insights surveys
- Establish a clear communication strategy for each survey which is tailored to reflect the audience needs and that articulates the potential benefits to learners, staff and the organisation
- Encourage learners to be honest in their responses – remind them that you cannot improve without knowing the truth
- Take care to run the student survey at a time that doesn’t clash or compete with other major surveys as this could negatively impact on your response rate
- Think carefully about the best window for running the staff survey and avoid peak workload times
- Use the templates provided by the insights service team – these will help you to analyse your data quickly and to communicate key findings
- Be responsive and show how you are acting on the findings
The digital experience insights surveys provide tools and mechanisms that are enabling the college to identify and validate the work that they are doing to improve the digital environment. It has helped them to identify successes that they can celebrate as well as opportunities for growth.
The digital experience insight helps us to get clarity about our students’ voice about their digital skills. This enables us to inform our vision and how we can move forward as an organisation to support their needs.Louise Fall, assistant principal in student engagement
In addition to actions already taken, the college is looking to the future in several areas:
- Work is ongoing in trying to identify wifi blackspots. The e-learning and IT teams are asking students and staff where these areas might be and are providing additional access points to improve this. They are monitoring where students use loan equipment to ensure wifi access is available and have noted that students are choosing a wider variety of places from which to work independently.
- Feedback also indicates that students would like the college to provide more equipment for student loan. There is some evidence to indicate that increased use of digital technologies in classroom activities has had a corresponding impact on demand. The college plans to purchase additional Chromebooks over the summer to meet this demand.
- Learners said they would like there to be more resources available on the VLE. In response, the TEL team will be providing continuous professional development (CPD) and upskilling opportunities through ‘CPD Wednesdays’ and other key CPD events.
- As further data from the staff digital experience insights survey becomes available the college will look for any correlations or differences between the two data sets.
Conrad hopes to conduct both the student and staff surveys on an annual basis to inform planning for future academic years. This is important because the college community changes each year with different learners who have differing needs and skills sets. Working with Jisc through the insights service has helped the college to give its’ community a voice in the digital strategy and the digital environment.
Learners and staff now have a voice in key decisions regarding purchasing and improving the digital experience for everyone in the college community. We can show that we have taken forward suggestions in terms of practice as well as software, hardware and equipment.Project lead: Conrad Taylor, e-learning manager, quality, City of Wolverhampton College