pedagogy

Ethical principles of learning analytics – mini critique

This is just a short blog post to capture some thoughts on the ethical principles of learning analytics as set out in official documentation provided by The Open University.  I have attended various briefings at the OU around this subject, mainly because there is a lot of complexity here with regard to the ethical significance of these technologies.  I was also a member of the advisory panel for the JISC Code of Practice for Learning Analytics.

Here are the ‘ethical principles’ with my own brief annotations (click to enlarge).  (This is just an internal critique of these principles as they are set out here, not of the wider project of learning analytics.)

learninganalytics

The principles have been categorised in the following way:
Screen Shot 2015-12-03 at 13.03.15You can see the original list at http://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/using-information-to-support-student-learning.pdf.

In essence, these are the points I would make about these principles are as follows:

  • Point 1.  It is asserted that learning analytics is an ethical practice, but this has yet to be established.  Arguably we should state that it should be thought of an ethical practice, but this is quite different in terms of ethical principle.  ‘Ought’ statements are much harder to justify.
  • Point 2. There is a confusing mix of deontological and consequentialist-utilitarian consideration here.  Unpicking it, I interpret it to mean that the university considers itself to have a responsibility to maximise the utility of the data about students that it owns.  The important points here are that a.) stakeholders are not clearly defined and could include (for instance, privately owned data brokers; b.) there is no acknowledgment of the possible tension between different forms of self-interest; c.) no criteria are given for ‘feasibility’.
  • Point 2. It’s difficult to see how feasibility should be a criterion for whether something is ethical.  After all, ethics is something that regulates the realm of the feasible, the possible, the actual.  This would be a much stronger principle if this word was replaced with ‘ethical’, or ‘justified’.
  • Point 3 infers that students should be at least partly defined by their data and the university’s interpretation of it.  This may not be that contentious to most people, though without clear parameters for the other criteria that are considered it could be taken to mean ‘mostly’ defined by the data held by the university.  It’s not clear what this means in practice except putting in some wording to ward off concerns about treating students as nothing more than a set of data points.
  • Point 4 seems right in setting out a principle of transparency in the process, purpose and use of student data.  But it doesn’t make a commitment to full transparency for all.  Why not?
  • This is brought into sharper relief in Point 5, which sets out a commitment to full transparency for data collection. Taken in conjunction with Point 4, it seems that transparency is endorsed for collection, but not use.
  • Point 6 is on the theme of student autonomy, and co-operation in these processes.  These are good things, though claims to have given informed consent are potentially undermined by the possible lack of transparency in use in Point 4.
  • A further possible undermining of student autonomy here is the lack of clarity about whether students can entirely opt out of these processes.  If not, how can they be considered ‘active agents’?
  • I’m not an expert in big data but I know a little bit about predictive modelling.  In Point 7. the idea is that modelling ‘should be’ free from bias.  Well, all modelling should be free from bias, but these effects cannot be truly eradicated.  It would make more sense as a principle to speak of ‘minimising’ bias.
  • Point 8. endorses adoption of learning analytics into the institutional culture, and vice versa.  It asserts that there values and benefits to the approach, though these are largely hypothetical.  It basically states that the institutional culture of the university must change, and that this should be ‘broadly accepted’ (whatever that might mean).

The final point I’d make about this is that, for me, these are not generally worded as principles: rather as vision statements or something intended to guide internal decision making.  But when it comes to ethics, we really need clear principles if we are to understand whether they are being applied consistently, sensitively, and systematically.

 

noun_153130_cc

Open education and critical pedagogy

I have a new publication entitled ‘Open Education and Critical Pedagogy’ upcoming with Learning, Media and Technology.  Here’s the abstract.

This paper argues for a revaluation of the potential of open education to support more critical forms of pedagogy. Section 1 examines contemporary discourses around open education, offering a commentary on the perception of openness as both a disruptive force in education, and a potential solution to contemporary challenges. Section 2 examines the implications of the lack of consensus around what it means to be open, focusing on the example of commercial and proprietary claims to openness commonly known as ‘openwashing’. Section 3 uses Raymond’s influential essay on open source software ‘The Cathedral and the Bazaar’ as a framework for thinking through these issues, and about alternative power structures in open education. In Section 4, an explicit link is drawn between more equal and democratic power structures and the possibility for developing pedagogies which are critical and reflexive, providing examples which show how certain interpretations of openness can raise opportunities to support critical approaches to pedagogy.

Keywords: open education; OER; MOOC; critique; evidence; critical
theory; critical pedagogy; discourse analysis; openwashing

The paper is available online but it unfortunately behind a paywall. I have an author’s link that allows free downloads for the first 50 requests, so if you’d like a copy then redirect your browser to http://www.tandfonline.com/eprint/YtPXqFkVSmteYfbgURJc/full.

[If you try the link and all the free copies are gone, let me know and I’ll send you a pre-print version.]

Workshop Notes: #Ethics and #LearningAnalytics

This morning I’m attending a talk given by Sharon Slade about the ethical dimensions of learning analytics (LA), part of a larger workshop devoted to LA at The Open University’s library on the Walton Hall campus.

I was a bit late from a previous meeting but Sharon’s slides are pretty clear so I’m just going to crack on with trying to capture the essence of the talk.  Here are the guidelines currently influencing thinking in this area (with my comment in parentheses).

  1. LA as a moral practice (I guess people need to be reminded of this!)
  2. OU has a responsibility to use data for student benefit
  3. Students are not wholly defined by their data (Ergo partially defined by data?)
  4. Purpose and boundaries should be well defined and visible (transparency)
  5. Students should have the facility to update their own data
  6. Students as active agents
  7. Modelling approaches and interventions should be free from bias (Is this possible? What kind of bias should be avoided?)
  8. Adoptions of LA requires broad acceptance of the values and benefits the development of appropriate skills (Not sure I fully grasped this one)

Sharon was mainly outlining the results of some qualitative research done with OU staff and students. The most emotive discussion was around whether or not this use of student data was appropriate at all – many students expressed dismay that their data was being looked at, much less used to potentially determine their service provision and educational future (progress, funding, etc.). Many felt that LA itself is a rather intrusive approach which may not be justified by the benevolent intention to improve student support.

While there are clear policies in place around data protection (like most universities) there were concerns about the use of raw data and information derived from data patterns. There was lots of concern about the ability of the analysts to adequately understand the data they were looking at and treat it responsibly.

Students want to have a 1:1 relationship with tutors, and feel that LA can undermine this; although at the OU there are particular challenges around distance education at scale.

The most dominant issue surrounded the idea of being able to opt-out of having their data collected without this having an impact on their future studies or how they are treated by the university. The default position is one of ‘informed consent’, where students are currently expected to opt out if they wish. The policy will be explained to students at the point of registration and well as providing case studies and guidance for staff and students.

Another round of consultation is expected around the issue of whether students should have an opt-out or opt-in model.

There is an underlying paternalistic attitude here – the university believes that it knows best with regard to the interests of the students – though it seems to me that this potentially runs against the idea of a student centred approach.

Some further thoughts/comments:

  • Someone like Simon Buckingham-Shum will argue that the LA *is* the pedagogy – this is not the view being taken by the OU but we can perhaps identify a potential ‘mission creep’
  • Can we be sure that the analyses we create through LA are reliable?  How?
  • The more data we collect and the more open it is then the more effective LA can be – and the greater the ethical complexity
  • New legislation requires that everyone will have the right to opt-out but it’s not clear that this will necessarily apply to education
  • Commercialisation of data has already taken place in some initiatives

Doug Clow then took the floor and spoke about other LA initiatives.  He noted that the drivers behind interest in LA are very diverse (research, retention, support, business intelligence, etc).  Some projects of note include:

Many projects are attempting to produce the correct kind of ‘dashboard’ for LA.  Another theme is around the extent to which LA initiatives can be scaled up to form a larger infrastructure.  There is a risk that with LA we focus only on the data we have access to and everything follows from there – Doug used the metaphor of darkness/illumination/blinding light. Doug also noted that machine learning stands to benefit greatly from LA data, and LA generally should be understood within the context of trends towards informal and blended learning as well as MOOC provision.

Overall, though, it seems that evidence for the effectiveness of LA is still pretty thin with very few rigorous evaluations. This could reflect the age of the field (a lot of work has yet to be published) or alternatively the idea that LA isn’t really as effective as some hope.  For instance, it could be that any intervention is effective regardless of whether it has some foundation in data that has been collected (nb. ‘Hawthorne effect‘).

Socratic Method, Mazur and ‘Peer Learning’

I realise I haven’t been keeping up with blogging here (the perennial blogging complaint), mainly because I’ve been contributing blogs to the OER Research Hub project.  But I think there should be a bit more activity here as well as a bit of cross-posting (oh for the ability to re-blog between .com and .org installs on WordPress…).

What’s prompted this is a presentation I’ve just attended by Kari Arfstorm of the Flipped Learning Network.  I hadn’t previously been aware of an explicit connection between the ‘flipped’ learning methodology and a particular kind of pedagogical approach.  For those who are unaware, here’s the skinny on ‘flipped’ learning from the FLN site:

Flipped learning is intended to make more efficient use of classroom time by making sure that passive learning (e.g. reading, watching lecture) takes place away from the classroom and the time spent in the classroom makes the most of discussion, debate and inquiry.  Kari made an explicit connection between this approach and Socratic method – one I hadn’t made myself until now.

I confess to still being a little unclear on exactly how Socratic we should take flipped learning to be; or the ways in which flipping encourages a certain kind of pedagogy.  Kari suggested in the session that some subjects (e.g. mathematics) we might abide by a more authoritative teaching model in order to reflect the ‘hard’ nature of the subject while in the humanities we might employ a more authentically Socratic method.  I’m behind the curve on the whole thinking behind flipping, but a few thoughts come to mind:

  • Part of my confusion seems to arise from the fact that I understand Socratic method as anti-didactic.  Of course, it’s still a form of instruction (hence the irony) but ultimately a kind of facilitated self-instruction.
  • Socrates does in fact use this method to ‘teach’ mathematics.  In Plato’s Meno he takes a young slave boy and demonstrates his innate knowledge of Euclidean geometry by asking him questions rather than ‘teaching’ him.
  • Do you observe, Meno, that I am not teaching the boy anything, but only asking him questions; and now he fancies that he knows how long a line is necessary in order to produce a figure of eight square feet; does he not?

  • ‘Flipping’ seems (at least at first glance) to be an irony-free approach to pedagogy, which makes its Socratic status unclear to me.
  • In some dialogues, Socrates is often regarded as a mouth-piece for Platonic ideas. We don’t mean this kind of instruction when we talk about Socratic pedagogy but obviously there’s a lot of debate about when & where this is happening and why.

In any case, it seems that simply ‘flipping’ the classroom need not in itself result in more critical forms of reasoning and/or learning, though it’s easy to see how having a bit more time devoted to discussion or ‘higher-order’ pedagogical exercises might facilitate this.

When I asked Kari about this aspect of ‘flipping’ she said that the connection was not simply to Socratic methods but to some of the pedagogical approaches inspired by critical approaches.  Foremost among these seems to be physicist Eric Mazur‘s concept of peer instruction which dates from the early 1990s.

Peer Instruction encourages more interactive engagement by replacing lectures with small group discussions of concepts, supplemented by larger discussions punctuated by mini-lectures between questions. Students contemplate answers individually then discuss the explanations for their answers with their peers and come to agreement on the underlying physics (or models).

Here’s an outline of the process based on C. Turpen and N. Finkelstein (2010).

  • Instructor poses question based on students’ responses to their pre-class reading
  • Students reflect on the question
  • Students commit to an individual answer
  • Instructor reviews student responses
  • Students discuss their thinking and answers with their peers
  • Students then commit again to an individual answer
  • The instructor again reviews responses and decides whether more explanation is needed before moving on to the next concept.


  • Here’s a lecture by Eric Mazur which explains his approach to peer-based learning.


    I feel like I’ve learned something today as I have a better understanding of some of the pedagogical principles underlying the idea of ‘flipping’ the classroom.  But I also feel that it may be better to think of these approaches as encouraging a kind of intellectual or critical autonomy (undoubtedly a Socratic aspiration) rather than thinking of them as Socratic per se.

    Interestingly, the idea of Socratic reasoning also came through in a separate presentation today. Thanh Le (of the Vital Signs project) explained how ther schoolchildren use negative method of hypothesis elimination in order to improve their knowledge of native and invasive flora and fauna.

    Freire & Fromm on ‘Necrophily’

    Just reading Pedagogy of the Oppressed and had to make a note of this quote.  Freire encourages us to adopt the general thesis that “only through communication can hold meaning” and suggests that in pedagogical situations the teacher may only authenticate their own thoughts through the thoughts of their students.  Hence “authentic thinking, thinking that is concerned about reality, does not take place in ivory tower isolation”.

    When pedagogy is based on a false, objectified or instrumental understanding of human beings, Freire contends, it cannot promote what Fromm terms ‘biophily’ (deep connection to life) but instead promotes its opposite.  Fromm’s description of this phenomena is strikingly Hegelian.

    “When life is characterized by growth in a structured, functional manner, the necrophilous person loves all that does not grow, that is mechanical.  The necrophilous person is driven by the desire to transform the organic into the inorganic, to approach life mechanically, as if all living persons were things…  The necrophilous person can relate to an object – a flower or a person – only if he possesses it […] if he loses possession he loses contact with the world… He loves control, and in the act of controlling he kills life.”

    This seems distinct from, say, Freud’s depiction of the death-drive termed Thanatos.  If anything, it reflects a particular form of reification or category mistake.  But what I find more interesting is Freire’s way of describing this in an educational context.  Freire (perhaps not unproblematically) co-identifies impaired communication, dominance of thought in education and the necrophilic attitude.

    “Oppression – overwhelming control – is necrophilic; it is nourished by love of death, not life.  The banking concept of education, which serves the interest of oppression, is also necrophilic.  Based on a mechanistic, static, naturalistic, spatialized view of consciousness, it transforms students into receiving objects.  It attempts to control thinking and action, leads men and women to adjust to the world, and inhibits their creative power.”

    Freire, P. (1996) Pedagogy of the Oppressed. Penguin, London. p.58

    EDIT: Someone else has also picked up on this and written something a bit more developed

    Philosophical Pedagogy: Wittgenstein

    Philosophical pedagogy is unusual in that the learner is being encouraged by the teacher to think for themselves and develop critical skills rather than absorb a certain concept or datum.  Here’s a report on Wittgenstein’s teaching style that I read today.  According to two of his students, this is how Wittgenstein described his own teaching style (D. A. T. Gasking and A. C. Jackson, ‘Wittgenstein as a Teacher’, in K. T. Fann (ed.), Ludwig Wittgenstein: The Man and His Philosophy, New Jersey: Humanities Press; Sussex: Harvester Press, p.52, 1962.).

    In teaching you philosophy I’m like a guide showing you how to find your way round London. I have to take you through the city from north to south, from east to west, from Euston to the embankment and from Piccadily to the Marble Arch. After I have taken you many journeys through the city, in all sorts of directions, we shall have passed through any given street a number of times – each time traversing the street as part of a different journey. At the end of this you will know London; you will be able to find your way about like a Londoner. Of course, a good guide will take you through the important streets more often than he takes you down side streets; a bad guide will do the opposite. In philosophy I’m rather a bad guide.