Failing our students: not noticing the traces they leave behind


Image credit:

Last week on 1 November, Jesse Stommel hosted a panel discussion on Ethical online learning  – which stayed with me and haunted me since I’ve watched it. Somehow this morning as I was writing this blog, some of the things that were said during the panel discussion came back to demand an audience. So while this post is not about the panel discussion (a reflection on the panel discussion is – hopefully – forthcoming), I want to acknowledge the impact that the panel discussion had and still have on my thinking – but more about this later.

This is not the blog that I wanted to write this week.

The blog I wanted to publish this week is half-way and as I was finalising the blog – I suddenly remembered that I fell behind with sending a mid-term assessment of my students’ participation grades. The purpose of this mid-term assessment in the context of the online course I am co-teaching, is to assess students’ general participation but more importantly, to warn those students who are at risk of failing the course.

As I was working through my students’ participation logs, the overview of what they submitted, their grades, the number of logins, their number of posts in the course’s discussion forum, etc.,– I realised that, somehow, I did not notice that one student (the gender and name of the student, in this account, is not important) was not as active as s/he should have been. This is an understatement. S/he was at serious risk of failing and only this morning did I pick it up. Damn it. Why did I not see it earlier?!

During the panel discussion Kate Bowles said that ethics in an online learning environment means noticing the footprints and/or the artefacts that someone has left online for me to notice. And we have to treat these footprints, these details, with reverence; almost in awe that someone left me something to discover, to engage with, to make sense of, and to respond to. And somehow I missed the evidence that s/he left me. I did not notice. And because I did not notice, I did not respond.

I do not share this reflection looking for sympathy.

I share this reflection/confession/despair in hoping that it will prompt deeper, more critical reflections on the ethics of teaching online, of having access to students’ data and the things they leave for us to find and the responsibility that comes with knowing when last they logged on, how many times they accessed the course site, when they last logged on and not really knowing what all of this means.

Let me provide context: The course I am teaching on is a 14-week, fully online graduate course with over 30 students and three instructors. While the instructors share the responsibility of responding to students queries and posts in this highly interactive and well-structured course, I am specifically responsible for the pastoral care of 10 students and to mark their various assignments and activities. Yes, only ten.

Over the course of the 14 weeks, students submit 3 essays, compile an e-portfolio and submit their progress in building this portfolio on four occasions. There are also eight ‘skill builder’ exercises such as using Diigo, creating a Twitter profile, annotating an article, etc.  Over the period of 14 weeks, students submit 16 various forms of activity/assessments that provides us opportunities to engage with students’ learning, to acknowledge their thoughts, to provide feedback and of course, allocate a mark. Over and above these activities, every week has assigned readings and students are required to post a comment (fully referenced) with regard to the readings per week, and respond to other students’ posts. As instructor I log on at least once a day to read through these posts, respond to a post or query. Between the three instructors we share this responsibility and this really helps. The three instructors are based in three different time zones which almost allow us a 24/7 opportunity of responding to students.

I also have access to an overview of each of my students’ progress. I am provided with a dashboard that provides me details of how many times they clicked on the separate pieces of content or topics, their number of logins, and an overview of their submission of the different assessments and skill builder exercises. For example, this morning I can see that one of the students has visited 81 of 213 content links (38% with 4 weeks to go),  has logged in 43 times since the start of the course 8 weeks ago, and  submitted (so far) 11 pieces of assessment and attained average scores.  Another student, in contrast, has visited only 23 of 213 content links (12% with 4 weeks to go),  has logged in 38 times since the start of the course 8 weeks ago (not much less than the first student in this narrative), and has submitted (so far) only 4 pieces of assessment which s/he passed. S/he has not submitted a second compulsory assignment and various other pieces of the learning journey.

There is a serious risk, for three weeks to go, that s/he will fail.

Why have I not noticed this earlier?

My guilt is even more when I drill down on his/her profile and look at the following information:

The student visited the topics/content in the course 64 times (but of the total of content links, only accessed 23 links – so s/he visited some topics repeatedly). Since the start of the course, s/he spent 2h 7 minutes and 33 seconds on the course site. S/he read 53 posts in the discussion forum, s/he responded to 7 and s/he posted 8 first-level posts (starting a thread). S/he logged in 112 times since the beginning of the course 8 weeks ago. The last time s/he logged in was yesterday. So it is not a question that s/he was not engaged.

I don’t have access to his/her prior education experiences. From the student’s profile picture s/he looks as if s/he may be in her/his late twenties or early thirties. S/he wears dark sun glasses on his/her profile picture. S/he is allocated to me as instructor. I failed her/him.

For the last 8 weeks I logged in almost every day. I am not behind with marking the assignments and skill builders. I provided and provide detailed feedback to my allocated students. It takes me an average of 90 minutes to read through each student’s essay assignment and provide detailed feedback. I think I am fair in my assessments and provide detailed evidence of what I appreciate in each essay and how I think they can improve their writing. In my responses to their posts in the discussion forums I try to stimulate them to think differently, more critically.

I thought I cared.

But I did not notice her/him falling behind. I had access to data about her/his engagement. And somehow did not notice.

Don’t get me wrong. I don’t propose that the data I have of her/him give me a holistic picture of her/his learning, her/his aspirations, and her/his life-worlds. Not at all. The data I have access to provide me partial data of a student in her/his late twenties or early thirties. A student who wears dark sun glasses on her/her profile picture. I also acknowledge (and I am on record) that we don’t yet understand what the number of logins means. What does it mean that s/he logged onto the course website 112 times over an eight week period? What does it mean that s/he spent (so far) 2h 7 minutes and 33 seconds on the course platform? Should I have noticed earlier that s/he does not post often? Is there a way that an algorithm could have picked up that s/he did not submit his assignment on time and could have warned me so that I could have written an email? Or was I so busy with grading the assignments of those who did submit on time in order to be ready for the next submission or responding to the next post or quickly logging on while I am responding to an article that just came back from the reviewers or whatever – that I did not notice that a student who is in her/his late twenties or early thirties wearing dark sun glasses has fallen behind?

This morning when I revisited her/his posts I noticed that in the first week when students were required to post a short bibliography of themselves, s/he sounded eager, enthusiastic, a go-getter (like most of the class). Did I miss something? Would it have helped if I had more data on her/him? Would it have helped if I knew her/his race or the gender with which s/he identifies? Would I have been more careful to notice her/his absence if I knew her/his socio-economic income or her/his familial responsibilities?

In this case, I think, it would not have helped if I knew more. I already had access to a lot of data s/he had left behind for me to find and make sense of. And somehow I did not notice.

It is easy to look for factors that would somehow, if not absolve, but would mitigate my guilt. But this blog is not about absolution or a lesser sentence due to mitigating factors. I should have noticed and I did not. Full stop.

So where to now? I sent her/him an email to voice my concerns and to offer my assistance if s/he would need any. I just hope I am not too late. And while I wait for her/him to respond (or not) let me conclude with some remarks/pointers:

If education (including online learning) is in its essence, relational, with different roles and responsibilities, we cannot negate the fact that in the asymmetries of power between us as teachers and students, that we have a fiduciary duty of care. If we decide to teach online, this is what we commit ourselves for – to care, to enable, to find whatever our students leave behind for us to find and treat those finds with respect, with reverence, and care. It is so easy (and tempting) to think about students’ login details, their time spent on task, and their patterns of engagement as interesting data points that we can interpret, that we can use to determine their risk and allocate a number on a spreadsheet uploaded to a grading system.

It is so easy to forget that the data points, the patterns, the number of logins are things our students leave behind, for us to find, engage with, make sense of, and treat with respect.

Does this have implications for student: teacher ratio? Yes. Do we need to consider the number and detail of responsibilities we expect of our online teachers and facilitators? Yes. Do we need to reconsider the way we design these online experiences and the number of people who take co-responsibility for different aspects of students’ learning journeys? Absolutely. And can we, carefully and considering all the challenges in algorithmic decision-making consider how to use algorithms as first warnings for me to notice, evaluate and consider and then act? I think so.

I am responsible for ten students in a small cohort of students in a highly structured and activity-intensive graduate online course. Over the course of eight weeks they leave me traces to make sense of, to engage with and to respect.

One student,  in her/his late twenties or early thirties  wearing dark sun glasses left me traces that s/he was in trouble.

 I did not notice.

About opendistanceteachingandlearning

Research professor in Open Distance and E-Learning (ODeL) at the University of South Africa (Unisa). Interested in teaching and learning in networked and open distance and e-learning environments. I blog in my personal capacity and the views expressed in the blog does not reflect or represent the views of my employer, the University of South Africa (Unisa).
This entry was posted in Uncategorized and tagged , , , , , . Bookmark the permalink.

11 Responses to Failing our students: not noticing the traces they leave behind

  1. Maha Bali says:

    hi Paul, my friend. I’m feeling the pain.

    I remember the first (and only) time I gave a student an “F”. My co-teacher and I discussed it and there was no other way. He wasn’t coming to class, he wasn’t doing any work, he wasn’t responding when we reached out to him, and I remember thinking that it wasn’t the student who failed, but I who had failed him, I had failed to explore what was happening in his life that made him so disconnected that in an easy-ish to pass course like ours, he wasn’t even making the minimum effort to pass.

    I could look at your situation and think, but they’re only 10 students… how hard can it be to miss that one of them isn’t participating? But I’m a mom and sometimes something is happening to my ONLY daughter and I’ll not notice. Because we’re human. And you’re not looking for excuses, so this is neither judgment or excuse.

    To answer your question – yes, I suspect the data/algorithms/analytics can help us detect warning signs when a student falls so far behind the class average in terms of number of submissions, dates of submissions, number of logins or whatever criteria we deem important (do we have a choice over which criteria we would like, as teachers, to be warned about? do we have real data on which criteria matter in our context?).

    Also: what kind of open doors do we give students to help them with “social absence”? Social absence, a term I made up, is the online situation where you know you will be away for a while but you want to inform important people to expect you to be absent. Kind of like when you and I are working on something and you let me know you’ll be traveling for 3 days, or you have a family thing this weekend, or you had connectivity problems yesterday or you broke up with your partner or such. When we aren’t in social spaces with our students, they may not know they can talk to us

    And you just gave me an idea – I should open those doors for my f2f students as well. Even though their absence is visible, I still need to let them know they can speak to me if there is something wrong, something they’re struggling with in their own lives or in the course itself.

    The other aspect I thought could help is when you have students working together – they themselves will notice each others’ absences, right? My 5 year old comes back every day from school and says which of her friends at school was absent!

    Anyway… my point is… yeah – data can help with these things… as long as we’re careful it doesn’t give us the wrong impression about someone and that there are ways for people (learners, teachers) to have conversations about why they behave in certain ways (what if someone logs on for short periods of time because their connectivity is bad, but they download stuff and read it offline? What if someone logs on OFTEN because their connection keeps getting disconnected and they need to re-login every 5 minutes? What if someone has been absent for a week because someone died, because their computer died, because…whatever).

    And you know what else, Paul? I think you need warning signs for your own self. How do *we* as teachers know to warn ourselves that something is wrong with our own lives, that our own work is falling behind and we might be approaching failure.

    • Thanks so much Maha. Despite all my protestations about the ethical implications of our collection of student data and the fact that we should critically and constantly reflect on ‘what do the data mean?’ – this weekend was sort of a wake-up call. While I agree with you that it is human to not have noticed, I think we can do better 🙂

      What I appreciate the most in your comment is the last paragraph where you open the conversation to include us as teachers. Who will notice when I fall behind? And more importantly, do I want someone to notice when I fall behind with my marking, with my administration, with maintaining my social presence? And how do we prevent this from becoming a claustrophobic Panopticon of performativity?

      More importantly I think is what I have learned from you and so many others in my digital live – to be a caring and critical audience, to be engaged, to be there for me. And for that my friend, I am indebted. Foreva and eva 🙂

      • Maha Bali says:

        Let’s take the panopticon one step further. Should teachers surveil students, or should students get their own data and surveil themselves (still panoptic of a different kind, according to Foucault).

        So let’s consider this:
        a. Give students full access to data being collected about themselves.
        b. Give students control over how to manipulate that data to know about their own digital patterns of behavior
        c. Give students agency to decide what their learning goals are, and to track only the things they’re interested in tracking, to flag the things they consider worth flagging, and…
        d. Give students opportunities to manually input other data (e.g. “time spent working on the course offline” or “connectivity problems” or such – they decide what the variables are and how they want them to be tracked).

        And make the system work for the students.

        If we make it like that for us as professionals and teachers, it makes complete sense, right? We set our own goals, put our own to-do lists, manage our own deadlines, and usually, unless it is something big like a date for submitting grades or reporting on a grant, no one keeps track. Systems for such tracking need not be fully online… or do they? I’m not sure…

        On a side note, if you missed this for 10 students, I imagine it’s much worse for people who have 100 students and so I assume there already exist these warning things in LMSs. I’m pretty sure that’s what people are talking about in conferences. I always used to think teachers should notice earlier than those warning bells go off, or that warning bells can be false alarms… so it needs a lot of human judgment, and consideration of how we react with… “is everything ok?” instead of “what the heck is wrong with you that you’re not doing the work?”

  2. francesbell says:

    Hi Paul,
    I note that you said “I do not share this reflection looking for sympathy.” so I won’t do that here- but you know my Skype/DM 🙂
    I found this post to be an amazing stimulation for reflection on the years I spent in teaching – so thank you!
    I learned long before my relations with students were increasingly mediated through technology that for some undergraduate students, it’s the first couple of weeks that are the most significant in their ability to ‘stick’ to the course. That’s probably why I devoted quite a lot of time and energy to induction activities that helped to set up real and latent connections between students and staff. I also learned that for some students, it isn’t the right time (for different reasons) to start the course and the best help I could give them was to listen to them and help them leave the course with minimal financial and social penalties, and getting any additional support they needed. Early in my HE teaching, when students had grants, there was a grace period ( before week 7 of Semester 1 in my distant memory) when leaving the course didn’t ‘count’ as a grant year.
    As the years passed, grants morphed into loans (with increasing future burdens) and the array of email, VLE, social media increased. I was interested when I realised that I could target emails to students by their activity (or lack of it). But then, of course, I remembered about the students who didn’t ‘stick’, and they turned out to be less likely to check their emails, and the VLE functionality was irrelevant for them.
    I know you are talking about online learning but the relationality is important for all mediated learning. My reflection led me to think about what Mejias called the paranode – what was happening in students’ lives that makes sense of what we see at the node. For me, ‘network goggles’ are dangerous not just in terms of how teachers support students but also in terms of decisions that are made about how students are enrolled and discarded if they fail to ‘stick’ to the systems that mediate education with its possibility of learning. By the time I retired, I realised that I knew much less about the financial consequences for students who failed to ‘stick’ in the era of loans. My gut feeling was that the consequences could be dire.

    • Thanks so much for the response Frances. And thanks for not offering sympathy 🙂

      This experience made me think seriously about the way we design and structure courses and how we, in our attempt to ensure that our students remain engaged, may become caught up in frenzied smaller activities losing sense of the bigger picture and looking for bigger trends. The comment by Kate Bowles really made me think differently about the traces students leave, and the respect and care required in engaging with these traces.
      Thanks also for the reference to Mejias’ notion of the ‘paranode’ – will follow up and read.

      Once again, thanks. Paul

  3. Wow, Paul, brutally honest and thoroughly motivating. I will be more conscious of my students’ breadcrumbs now. As indicated, I won’t discuss sympathy, but you can guess what I’m feeling/thinking.

    We have access to a lot of data about our (f2f) students where I teach, but receive very little support on interpreting it.

    Thanks for the motivational post,.

  4. Gabi Witthaus says:

    Hi Paul. Thanks for sharing this experience with us. Sympathy withheld as per your request – but I would like to say I really love the description of the course you’re teaching on, and the team-teaching approach – I’d love to be a student on that one 🙂

    About five years ago I was at a multi-institution project meeting at the OU, where a colleague had been invited to give us a presentation about an OU pilot project experimenting with the uses of data analytics. One anecdote from his talk has stuck in mind ever since. He told us that the VLE had been programmed to automatically send an email to students who had not posted on a discussion forum a certain number of times. The email stated clearly in the introductory paragraph that it was automatically generated, and went on to provide a gentle warning that the student might be at risk of failing, along with some suggested behaviour changes and a reminder of where the student could get support from if they wanted to talk to someone. At the end of the pilot, students who had received these emails were asked how they felt about them. The feedback was apparently highly positive, with one student even commenting, “I finally feel like I’m being treated as a person, not just a number in the OU system!” (Those might not have been his/her exact words, but it was the gist.)

    So I’m wondering how you would feel about allocating this “noticing” task, and appropriate follow-up to the VLE in the case you’ve described? I certainly agree with the previous commenters that the teacher needs support too, and I don’t think you should bear the burden of guilt for letting the data slip through your human fingers…

    • Thanks so much for the feedback Gabi. Exactly my point, I know there are various and often serious concerns regarding the use of algorithms and automated decision-making in education – but your example and my gut feeling proves that ‘machines’ can help us to humanise feedback and care. Considering the difficulty to scale care and support, I really do think we need to seriously consider making use of the affordances of technology. If I could not even keep up with 10 students, what about courses with enrollments with thousands of students?

      Thanks again, As always.

  5. Jo-Anne Botha says:

    What I find interesting is that the student either did not realize that he was falling behind, or, if he realized it, did not contact you of his own accord. My interest and passion is adult learner self-directedness and I would expect a student who is moderately self-directed to also take the initiative to contact the instructor for assistance if he/she struggled with work, comprehension, time management and so on yet this student did not take the initiative. And I would like to know why. For me learning is a partnership, shared between the instructor (we call ourselves lecturers) and the learner. The analogy I always use is joining the gym. If one joins the gym but only occasionally exercises at the gym one will not get fit. One has to work together with the personal trainer or gym instructor to get fit. So why did the student not take any initiative? And I am in no way blaming the student, it seems to me that sometimes students lack competencies that we assume they have, in order to be successful in online or distance learning.

    • Paul says:

      Jo-Anne – thank you so much for your reply and engagement. I cannot speak for the student – s/he has not contact me yet and I don’t have access to his/her contact numbers. I agree with you that learning is a partnership – but taking into account the asymmetrical power relationship between me and the student, I have more responsibility. It is one thing to say that s/he could have contacted me and it is on their head if they did not – but somehow teaching is a moral act where teachers do carry a bigger responsibility, especially if we knew… And that is my point. The purpose of my post was not to say that teachers are the only ones carrying the responsibility to ensure student success, but rather that if we have access to data and the information embedded in the data, then our responsibilities increase.

      And I had ten students. Despite this small number I still missed his dis-engagement. I could have done better.

      Getting back to your analogy of joining the gym – would it not be great if you get a call from the gym to say “You joined and somehow we recently noticed that you have not been coming as regularly as you used to. We sincerely hope that the reason you have not been coming regularly is not serious! Please know that we care and if you need advice on how to stay motivated, or want to make use of our offer of two free sessions with a personal instructor, please let us know?”

      Such a response from a gym will be exceptional in the context that as long as I pay they may not really care. Higher education is so much more than a gym. Students are so much more than customers. Teaching is so much more than dropping content into a folder in a learning management system. Teaching is so much more.

      Just thinking.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s