University teachers' view of useful learning analytics

Several attempts at use of analytics to enhance teaching practices that were met with scepticism from academics. Retrospectives of those works were often challenged by “so what?” questions as academics couldn’t do much with insights from dashboards that lacked timeliness and context. This work explored what teachers really want from learning analytics to support their day-to-day teaching practice.

Learning Analytics is defined as the “measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs”. Source of definition is SoLAR.

Like many data science professionals, my first impressions of Learning Analytics were those flashy dashboards, laden with charts, crunching massive data and predicting outcomes from machine-learnt algorithms. I thought if I could build those, teachers will dwell right in. They’ll have the time to do all the sense-making. They’ll simply admire them. After making, or observing being made, a few such attempts, I can tell you nothing is farther from reality.

The reality is that teachers are really busy people. Teaching hundreds, if not thousands, of students requires adhering to tight schedules with lots to do. We can’t simply expect them to get on the analytics band wagon, especially if doing so adds to their workload. So I set out on a mission to find out how, really, can we make Learning Analytics useful. This exercise revealed something really interesting that I hope to share in this post.

For a start, teachers really want to see their students succeed. They work hard on all aspects of their unit so students achieve the best possible outcomes above relevant standards. If Learning Analytics was to gain traction with teachers, we need to rethink our approach. Whatever we provide to teachers should be very simple, timely and meaningful. We should trade difficult-to-understand charts for language that teachers readily understand. And most importantly, Learning Analytics should help reduce their workload, not increase it.

In several ideation and brainstorming workshops with teachers and academics from various desciplines, I explored the following prompt:

Most responses I received fall in to the following seven groups:

1. About the cohort
2. Content engagement
3. Synchronous engagement
4. Assignment timeliness
5. Reaching out to students at-risk
6. Discussion boards engagement
7. External factors

Read on to see the type of questions asked in each group.

1. About the cohort

Knowing about the cohort early allows teachers to adjust their teaching activities to suit students’ needs.

For example, some students might be repeating the unit while some others might have special needs related to any disability. Learning Analytics can help by answering:

Which students have special needs and what adjustments do I need to make?

Which students are repeating the unit?

I thought this one was very interesting: Teachers frequency experience students from different descriplines studying their unit as one class. Usually students from different courses need different level of teaching support. It would help teachers if Learning Analytics could answer:

What is the descipline (course) profile and demographic mix of students in my unit?

2. Content engagement

Early on in the trimester, teachers expect students to read the unit outline. They should also access learning materials on the unit site. Teachers would like to know early if students have to not engaged with unit site content.

Who has not accessed learning materials?

3. Synchronous engagement

Synchronous engagement is about attending and engaging in classes (or seminars). Classes are essential part of learning where students get the opportunity to interact with their teachers and peers. Classes usually occur every week. If students don’t attend, they are likely to miss out on all that interaction. It might become hard for them to keep up. Teachers would like to know timely:

Who is not attending classes or watching recordings?

4. Assignment timeliness

Assignments make the important link betweeen student learning and achieving expected standards for a unit. Assignments are graded and students usually work through high-rigor and deadline pressures to achieve good marks. If students are missing submissions deadlines, this might be a sign that they’re struggling. Teachers would like to know from learning analytics

Who did not submit assignment or submitted late?

If students miss or delay multiple assignments, they might be at-risk of failing the unit. Teachers being able to know this early could mean they could reach out to students offering help.

5. Reaching out to students at-risk

While teachers expect learning analytics to indicate which students might be at-risk (as early as reliably possible), they’ve also often stressed that it’s very hard to get struggling students to speak up or reach out for help. Often by the time we know they are struggling, through Learning Analytics, they are already disengaged and little can be done to bring them back on track.

Who might need my help most and how might I help?

This is where Learning Analytics could do more to recommend the best known check-in strategies that work is different situations. For example some communication templates could be provided that make it easier for both teachers to reach out, saving their time, and for students to respond to those messages. This could be done with some proven (tried and tested) check-in templates available between the academic community.

6. Discussion boards engagement

Students use discussion boards in their unit site to stay engaged, with the learning community, through dialog about course material. Discussion boards are also a good way helping students build their support networks. Moreover, not all engagement in discussions is graded, but some is. Teachers would like to know, through learning analytics:

Who is not engaging in discussion forums? They might not necessarily reply or post, but are they atleast reading the forums?

7. External factors

University life could come with many challenges and problems. Students juggling many competing priorities and usually navigate through a lot. If students are struggling in a unit, it might be because they’re under a lot of stress from other units or some other aspects of their university life. Teachers would like to know from learning analytics how their students are being supported in other areas of their university life so this context can help them build more personal relationships with the students base on empathy.

Are students being supported in their other units?

If someone is disengaged, are they struggling with other aspects of uni experience, such as outstanding tickets with university IT Help services?

Conclusion

I found these learnings very interesting, mainly because they weren’t about charts or tech., they were about what teacher care for ‘which of my students might need my help so they can be successful’. They was a pure reflection of what teachers stand for. Teachers spend so much time and energy towards making their students successful. Those in Learning Analytics domain should provide products that helpful, and underpinned by simplicy, making the job of teacher easier not harder.

Further work

Careful consideration needs to be given to student privacy and consent (how much data could we collect and for what purpose). There are also concerns related to Bias, such as Confirmation Bias, that should be addressed. When teachers know more about their students through Learning Analytics data, how do we ensure this additional knowledge does not negatively influence their objective judgement of student work. For example, a high achieving student in other units could be perceived as more competent than their peer student regardless of their quality of work in this unit.