This is just a short blog post to capture some thoughts on the ethical principles of learning analytics as set out in official documentation provided by The Open University. I have attended various briefings at the OU around this subject, mainly because there is a lot of complexity here with regard to the ethical significance of these technologies. I was also a member of the advisory panel for the JISC Code of Practice for Learning Analytics.
Here are the ‘ethical principles’ with my own brief annotations (click to enlarge). (This is just an internal critique of these principles as they are set out here, not of the wider project of learning analytics.)
The principles have been categorised in the following way:
You can see the original list at http://www.open.ac.uk/students/charter/sites/www.open.ac.uk.students.charter/files/files/ecms/web-content/using-information-to-support-student-learning.pdf.
In essence, these are the points I would make about these principles are as follows:
- Point 1. It is asserted that learning analytics is an ethical practice, but this has yet to be established. Arguably we should state that it should be thought of an ethical practice, but this is quite different in terms of ethical principle. ‘Ought’ statements are much harder to justify.
- Point 2. There is a confusing mix of deontological and consequentialist-utilitarian consideration here. Unpicking it, I interpret it to mean that the university considers itself to have a responsibility to maximise the utility of the data about students that it owns. The important points here are that a.) stakeholders are not clearly defined and could include (for instance, privately owned data brokers; b.) there is no acknowledgment of the possible tension between different forms of self-interest; c.) no criteria are given for ‘feasibility’.
- Point 2. It’s difficult to see how feasibility should be a criterion for whether something is ethical. After all, ethics is something that regulates the realm of the feasible, the possible, the actual. This would be a much stronger principle if this word was replaced with ‘ethical’, or ‘justified’.
- Point 3 infers that students should be at least partly defined by their data and the university’s interpretation of it. This may not be that contentious to most people, though without clear parameters for the other criteria that are considered it could be taken to mean ‘mostly’ defined by the data held by the university. It’s not clear what this means in practice except putting in some wording to ward off concerns about treating students as nothing more than a set of data points.
- Point 4 seems right in setting out a principle of transparency in the process, purpose and use of student data. But it doesn’t make a commitment to full transparency for all. Why not?
- This is brought into sharper relief in Point 5, which sets out a commitment to full transparency for data collection. Taken in conjunction with Point 4, it seems that transparency is endorsed for collection, but not use.
- Point 6 is on the theme of student autonomy, and co-operation in these processes. These are good things, though claims to have given informed consent are potentially undermined by the possible lack of transparency in use in Point 4.
- A further possible undermining of student autonomy here is the lack of clarity about whether students can entirely opt out of these processes. If not, how can they be considered ‘active agents’?
- I’m not an expert in big data but I know a little bit about predictive modelling. In Point 7. the idea is that modelling ‘should be’ free from bias. Well, all modelling should be free from bias, but these effects cannot be truly eradicated. It would make more sense as a principle to speak of ‘minimising’ bias.
- Point 8. endorses adoption of learning analytics into the institutional culture, and vice versa. It asserts that there values and benefits to the approach, though these are largely hypothetical. It basically states that the institutional culture of the university must change, and that this should be ‘broadly accepted’ (whatever that might mean).
The final point I’d make about this is that, for me, these are not generally worded as principles: rather as vision statements or something intended to guide internal decision making. But when it comes to ethics, we really need clear principles if we are to understand whether they are being applied consistently, sensitively, and systematically.
Here is the latest list of books available for review from JiME. If you’re interested in reviewing any of the following then get in touch with me through Twitter or via rob.farrow [at] open.ac.uk to let me know which volume you are interested in and some of your reviewer credentials.
Reviews will be due at the end of February 2016, and should be in the region of 1500-2000 words. You can see examples of previous reviews at http://jime.open.ac.uk/.
If you’re an academic publisher and you’re reading this you my have noted we have a lot of books from Routledge in the backlog. If you’d like to have your books considered fro review in JiME then please mail them for my attention at the address in the sidebar.
- Curtis J. Bonk, Mimi M. Lee, Thomas C. Reeves & Thomas H. Reynolds (eds.) (2015) MOOCs and Open Education around the world. Routledge: Abingdon and New York. link
- Charles D. Dziuban, Anthony G. Picciano, Charles R. Graham & Patsy D. Moskal (2016). Conducting Research in Online and Blended Learning Environments. Routledge: Abingdon and New York. link
- Susan Garvis & Narelle Lemon (eds.) (2016). Understanding Digital Technologies and Young Children: An International Perspective. Routledge: Abingdon and New York. link
- Seth Giddings (2014). Gameworlds: Virtual Media and Children’s Everyday Play. Bloomsbury Academic. link
- Lori Diane Hill & Felice J. Levine (eds.) (2015). World Education Research Yearbook 2015. Routledge: Abingdon. link
- Wanda Hurren & Erika Hasebe-Ludt (eds.) (2014). Contemplating Curriculum – Genealogies, Times, Places. Routledge: London and New York. link
- Phyllis Jones (ed.) (2014). Bringing Insider Perspectives into Inclusive Learner Teaching – Potentials and challenges for educational professionals. Routledge: London and New York. link
- David Killick (2015). Developing the Global Student: Higher education in an era of globalization. Routledge: London and New York. link
- Piet A. M. Kommers, Pedro Isaias & Tomayess Issa (2015). Perspectives on Social Media – a yearbook. Routledge: London and New York. link
- Angela McFarlane (2015). Authentic Learning for the Digital Generation – realising the potential of technology in the classroom. Routledge: Abingdon. link
- Jill Porter (ed.) (2015). Understanding and Responding to the Experience of Disability. Routledge: London and New York. link
- Steven Warburton & Stylianos Hatzipanagos (eds.) (2013). Digital Identity and Social Media. IGI Global: Hershey, PA. link
I’ve started writing this post at the Open Education 2015 conference at the Fairmont Hotel in Vancouver because I want to try and capture some thoughts about the evolution of this movement and community. But I’m finishing it from home after a little bit of time to digest and also after attending OpenUpTRU in Kamloops earlier in the week.
This has been my fifth consecutive Open Education conference and I’ve been privileged enough to hear from a lot of different people from around the world about their use of OER and the impact it has for them. Over these years there has been a steady move towards raising the game with research into impact and strategising ways to mainstream the adoption of OER; perhaps the clearest example of this is the may presentations that have been devoted to open textbook adoption and efficacy studies at this conference. This is entirely understandable given the co-ordinated focus in the USA on open textbook adoption as a tangible and measurable goal for advocacy and research.
Great things have been achieved by researchers working with the Open Education Group in this regard. In terms of controlled studies which attempt to isolate the effects of moving to an open textbook while controlling for other variables (like instructors, etc.) there really isn’t any other game in town that comes close. And there is a real need for this kind of work, since it is creating the body of evidence that can be used to reject the claim that open resources are of inferior quality. The endgame here is to support widespread adoption of open textbooks in colleges. This is something that can be measured and the savings calculated, so it’s a great strategic choice for advocates in the USA.
Now we have established that this research is great, I feel there are a couple of points to raise. Firstly, a methodological issue related to the tension between two virtues of open textbooks that we like to put forward: that they are ‘efficacious’ (they ’cause’ learning)  as established by controlled studies; and that they can be freely adapted. How much adaptation can a text withstand before the efficacy studies – which are based on carefully controlling variables – must be repeated? Of course, in many cases the textbooks are just adopted wholesale. They are mapped onto common curricula and so can be used to teach a whole programme. But if someone decides not to tamper with the textbook, isn’t the net result of all this just that the commercial textbook has been replaced by an open textbook? But if they do ‘tamper’ with the textbook, might they be in danger of making their textbooks less ‘efficacious’?
Maybe that depends on how good they are at teaching. What I mean by this is that, aside from all the fantastic savings made by students, the course may be taught in exactly the same way as before. In effect, the open textbook strategy might (when fully realised) leave us with more or less the same educational systems as before (although a lot more affordable for many, and this would undoubtedly be a fine thing).
In effect, this is an attempt to ‘colonise’ an existing system by taking it over from within. Maybe something more radical follows from this – open textbooks are a great way to introduce students and faculty to OER, and who knows what might happen a few years down the line in a situation where everyone knows about open?
For now, though, nothing much need change except using an open textbook. Except it’s not just an open textbook, because to scale up and keep making the case for efficacy the data gathered must grow, which means more metrics, open learning analytics, and possible homogenization of the learning process.
This was how I captured the thought at the time:
Are there signs of a creep towards technocratism at #opened15? Metrics, efficacy, scalability, replicability, homogeneity, predictability
— Dr. Robert Farrow (@philosopher1978) November 18, 2015
What was less obvious at the conference this year were the voices coming from a different part of the OER movement: the people who emphasize the radical potential of OER.
This end of the spectrum may be hard to clearly define. They might be edupunks or critical pedagogues. They might identify with the open source, copyleft, open data or open government movements outside of education. They might just be libertarians who like the idea of greater personal freedom. But the thing that unites them is that OER is, for them, more about challenging existing practices and forms of knowledge transmission than replicating commercial provisions on open licences.
Because they’re a disparate bunch it’s hard to put a label on this group, even though by the title of this piece I’m referring to them as ‘edupunks (&c.)’. The important thing is that they are more radical in ambition, and in that sense they occupy the opposite end of the spectrum from the ‘colonisers’.
Here are some illustrative comments shared on Twitter at the time.
— David Kernohan (@dkernohan) November 19, 2015
The unfortunate equation of open education w/ free text books has made the movement seem more and more myopic and less and less compelling.
— Jim Groom (@jimgroom) November 19, 2015
Just going to say it, in a spirit of love and optimism: “open textbook” is an oxymoron. #OpenEd15
— Robin DeRosa (@actualham) November 20, 2015
There were plenty of others to choose from, as well as plenty of support for what is being achieved with open textbooks. Robin actually went a step further and wrote a blog post which expressed her frustration with the dominance of open textbooks and outlined the kinds of things that she wants from a conference like Open Education.
- Engage learners in contributing to their learning materials so that knowledge becomes a community endeavor rather than a commodity that needs to be made accessible. To that end, let’s stop fetishizing the textbook, which is at best a low-bar pedagogical tool for transmitting information. OER is better than that.
- Make open licenses the focus of our advocacy for learners, teachers, scholars, which means explaining how the open license enables us to do more with the ideas that we ourselves as learners, teachers, scholars are generating. It’s not the open textbook, it’s the open license that matters here.
- Consider public funding models for open education (OER, open pedagogy, open access). “Philanthropy” is the wrong word for a model in which the public pays itself for what it needs and can generate on its own. And I am not buying that private, for-profit companies– while capable of being good community partners– are the only way we can build a public infrastructure for publishing and organizing and economically supporting open work.
- Build a better mission statement for why we work in the open. I took a stab here, but it was just one tiny specific start. I need help explaining this why. We need the why before we can develop the what (who cares about our open tools and apps and platforms? that’s the easy stuff, so let’s do it second). We need the why before we can assess whether or not we achieved success. Will working in the open serve a social justice vision? improve retention and enrollment? increase interdisciplinary collaboration and improve the quality of our scholarship? Yes? Why? How? And what will it look like if our vision succeeds?
So, should the open education movement seek to colonise education, or transform it? In can be tempting to think that the difference here is really between evolution and revolution. The colonisers want to evolve formal education in a helpful way while the ‘edupunks (&c.)’ are more interested in empowerment and the freedoms provided by open licensing.
We might also surmise that this is a false dichotomy. Most people are somewhere in the middle, and relatively few people go around calling themselves ‘edupunks’. In some ways this can be seen as the return of the familiar gratis (‘colonisers’) vs libre (‘edupunk (&c.)’) distinction that has been with the OER movement since the very early days: is the OER movement about freedom, or about things being ‘free’?
C. P. Snow famously wrote about the divergence of science and the humanities in the influential The Two Cultures and the Scientific Revolution. Snow foresaw that the aspirations, language and standards of validity of academic cultures were moving apart in ways that prevented cross-pollination of ideas and findings. Thus, we have science professors who have never read Shakespeare, literature professors who cannot explain the laws of thermodynamics, and so on. Now arguably there are more interdisciplinary thinkers than there used to be but education does still tend to siphon learners off into one or the other camp.
Without getting too far into that debate, I think we can use the basic idea of ‘Two Cultures’ as a way of thinking about changes in the OER movement, and being aware of people pulling in different directions. Everyone is still part of the same conversation at the moment, but it doesn’t feel like it would take much to see new, more niche conferences and journals springing up. In my view, both of these cultures need each other, because each ameliorates the vulnerabilities of the other and encourages attentiveness to the bigger picture. So keep talking!
 I’m a little uncomfortable personally with the language of efficacy, which risks being scientistic – I’m not sure that isolating a lot of variables and then attributing any difference to the intervention is reliable in education research per se – though it is certainly commonplace and there is of course a need for evidence.
Today I’m on Coventry at the ‘Beyond the Neoliberal University‘ conference in Coventry, just half an hour on the train from Milton Keynes.
Proceedings began with quote from Brecht on the value of criticism and how criticism is itself an art. The conference brings together (i) those interested in critical pedagogy (as a theory of transformative education); (ii) people from the UCU union facing practical issues on a daily basis. The idea is to bring the two together through a focus on practical issues: job insecurity, tuition fees/loans/debt; instrumentalisation of higher education.
Keynote: Andrew Gettigan, author of The Great University Gamble
- Financial pressures on HE insitutions will increase, be tighter, and be varied
- It can be shown that the cuts to the HEFCE teaching grant have been replaced by full time undergraduate fees
- The future will be dominated by questions about the extent to which HE improves human capital and future earnings
- Neoclassical ideas about market competition underly much of the reform
- Most profitable courses for HE providers are often the ones with the worst return on investment for the government
- In the last 5 years replacing the teaching grant with fee income made fiscal sense – in future, increasing fees will just increase government costs
- Since the 2015 election, £150m a year has been cut from teaching grant
- New export target of £30bn by 2020 – focus on international students, distance learning
- Quality assurance review and Teaching Excellence Framework aim to ‘double the proportion of disadvantaged young people in HE relative to 2009’
- Graduate tax acts as a tax on social mobility: poorer students can expect to be paying back their loan for an additional four years, paying back an additional £9k
- There is no more scope for budgetary tricks to increase university income
Keynote: Sarah Amsler, author of The Education of Radical Democracy
What is ‘the beyond’? A space between utopianism and the status quo – something that allows us to think of radical alternatives in ambiguous ways. Higher education institutions are increasingly empowered to act undemocratically and autocratically in favour of market forces and neoliberalism. Managerial power is calibrated and refined, and since the 1970s it has become harder to teach critically, connect with social movements, and provide a space for progressive potential. What makes it possible for people to believe that this is somehow not the case (i.e. that there is hope)?
We experience deep ambiguity towards the university: e,g, both wanting to save it and destroy it. We see an exodus of academics leaving the academy because of low pay, unstable contracts, etc. (The same phenomenon can be seen in other educational sectors as well as the NHS.) Many feel that the neoliberal university holds little appeal for them.
It is only through practical experimentation that we can explore possibilities for resistance. Counter-hegemonic forms of resistance must be explored through praxis, and becoming comfortable in an uncomfortable, dialectical space. We tend not to use the language of grief and loss when discussing the neoliberal university but we should mourn the loss of space in higher education for thinking and acting progressively.
(I struggled to identify the practical things that can be done from this talk – mainly it seems to be about imagining otherwise, identifying concrete possibilities for social change and spaces for radical democracy but to me these things seem quite abstract.)
- First question criticised Amsler for concentrating on rather bourgeois concerns – doctors, lecturers, etc. They may be in a position to opt out, but what about those for whom opting out means not paying the rent?
- McGettigan makes the point that many academics are not really engaged with policy – this shold be something that is brought more fully into the limelight of academic culture
- Amsler criticised for paying lip service to praxis but not identifying practical forms of resistance: self indulgence? One person accused the questioner of being dogmatic (I disagree completely).
- Amsler responds by emphasising autonomous self-organization in response to power, no real specifics though
- McGettigan notes that UK HE culture is quite robust against direct threats, but is not good at dealing with the indirect threat of financialization and instrumentalization – this does not inspire solidarity between the different elements of the university.
- Gurnham Singh noted that the neoliberal project has been extremely successful and it’s quite difficult to imagine real resistance. But McGettigan’s work suggests that it will fall at some stage – hence the need for a reimagining of what is possible.
- Two sociology teachers emphasized the need for engaging with students and understanding that we have a role in explaining the situation to them. Students (understandably) are focused on getting a good degree and a good job, not the long term future of the institution. But how can we do this? The point is made that for students from disadvantaged backgrounds going to university at all might have been a radical engagement.
- McGettigan notes that the success of the last 20 years of HE policy has been to establish metrics for success (e.g. participation rates, retention rates, graduate employability, increasing access). Much of these metrics use the garb of equality and social mobility and this makes it harder to critique. But questioning this reframing should be at the core of rethinking all this – what would success look like? What is truly progressive?
Workshop: Introduction to Critical Pedagogy
Introduction by Gurnham Singh: Pedagogy is the means by which learning is enabled. It has its roots in the Greek ‘pedagogeu’, literally meaning ‘to lead the child’. More generally, the word refers to the science and art of human education and cognition. Influential theorists include Piaget, Vygotsky, Bruner, Kolb, Shor, Freire, and Knowles.
Critical pedagogy radically challenges traditional, didactic models of education. Shor characterises this as ‘questioning the answers’ rather than ‘answering the questions’. It is rooted in the invitation to examine power and the role of power relations in knowledge and cultural formation. Education is never politically neutral: it results either in control and subjugation or liberation and greater autonomy.
The methodology of critical pedagogy: teaching methods may be largely the same an for non-critical pedagogies in practice. The key thing is the disruption to traditional pedagogical relationships and expectations.
Freire emphasized the importance of literacy: not just as the ability to read and write but also as an attitude towards personal transformation and social revolution. He was interested in connecting learning to social action through a pedagogical cycle (Reflection / Analysis / Action ). Learners must discover new avenues of thought and consciousness for themselves to become better moral subjects who assume greater responsibility for the other. Only a self-managed life can be call education: preparation for work is ‘training’ that focuses on conformity and predictability. Similarly, critical pedagogues often reject classical measures of intelligence and intellectual ability.
Friere referred to his method of teaching as ‘conscientisation’: it represents a self-awakening and coming to critical consciousness. This is closely related to Marxist ideas about the emancipation of the proletariat through greater awareness of self and world. Conscientisation seeks to reverse a prior process of objectification or reification. In order to move away from false consciousness subjects must come to understand themselves and the world differently. This usually involves the rejection of fatalism, karmic understandings, naive consciousness and learned helplessness.
For critical pedagogy the vast majority of educational institutions act to maintain the status quo and are thus instruments of power and dominance. In Marxist terms, a political understanding of power can be developed through understanding the underlying political economy (the example of Jeremy Corbyn was raised).
The distinction between education and training raised in the morning sessions came up again in the discussion. The mindset of students and graduates is changing and becoming more focused on a return on investment and a way to pay back massive debts accrued while studying.
One delegate noted that universities can easily absorb critical pedagogies and in some sense neuter them by incorporating them into curricula. How can the critical impulse be retained? Do we just concentrate on the time spent in the classroom and see this as a possible space for ‘subversive’ activity?
One distinction that was brought out in the discussion was the distinction between self-directed learning (heutagogy); critical thinking skills; and critical pedagogy.
Workshop: Today’s Learners – Consumers or Producers?
This session began with an overview of the housing crisis for students in Coventry. Because university-provided accommodation is oversubscribed many students are forced to use privately rented accommodation which is often substandard. They also often have to work long hours in employment, and this can also have an impact upon their studies.
Students at the university feel aggrieved at this state of affairs. Their campaign uncovered that the university had declared a profit of some £25 million while still cutting student provision. The argument was made that students should have a say about how the university resources are used and prioritised. The suggestion was also made that student representatives should sit on the executive boards of the university and ensure that the student voice is heard.
Library of Birmingham Occupation Movement: 2.5 million people visit this library per year, with a diverse user base. As part of the austerity faced by the city they cut opening hours and budgets for new books. Spaces are closed at key times so they can be rented out to private enterprise (receptions, seminars, etc) and more than a hundred staff have been made redundant. The campaigning group initially coalesced around reduced opening times, then used the library space as a focus for meeting and planning further activity. A big rally and occupation is planned for October 2015.
The question is posed: why don’t students who are unhappy simply just go off to a different university, or do something else entirely? Classical economics makes several assumptions about how agents operate within markets, but can we assume that students have enough knowledge of the situation to act as ‘rational’ consumers? Neoliberalism assigns value through (‘free’) markets, but are there values which we need to impose upon these markets?
- Teach ourselves university accounting so that narratives can be challenged with authority
- Document and share the outcome of interventions
- Break down barriers between different interest groups, promote solidarity between staff and students
A series of podcasts on critical pedagogy can be found at https://critped.wordpress.com/resources-3/talks-and-interviews/.
Mark is a community activist and technology executive who currently serves as a director of the Mozilla Foundation, makers of the Firefox web browser. He began by noting the need for digital literacies, suggesting that literacy is characterised by the ability to:
Technologies allow us to express ourselves help us to read, write, and participate in new ways. And in important ways, since ideas and communication shape the world. “The Roman empire and city states were essentially products of writing“. Yet at the same time, how do we direct this process?
“We drive into the future using only our rearview mirror” is a modern take on “The owl of Minerva flies only at dusk” (Hegel) #oeglobal
— Robert Farrow (@philosopher1978) April 24, 2015
Mark referred to the media theorist Marshall McLuhan’s assertion that “We drive into the future using only our rearview mirror“. Agreements like the Cape Town Declaration help to orient us around a conception of openness that can inform strategies and ambitions. Mozilla’s Firefox browser is an example of success in reframing the status quo through collective action. The Internet Explorer browser went from 98% market dominance, and Microsoft lost a hold on their monopoly. Similarly, we are now in a position to rethink educational systems and break the patterns of the past. We can see this happening in a shift around the expectation around use of public funding, with programmes like the TAACCCT grants which mandate for OER production in community colleges. We have won battles, but we are losing the war: vast portions of the internet are walled gardens, and monopolies/oligopolies are emerging in educational markets. Companies like Google potentially control almost every aspect of a range of services with a business imperative based on gaining complete vertical control of our digital lives. The intent of companies like Uber is to become the monopolist of the ‘internet of things‘. Many people don’t really understand how the internet works, or what is happening when they use it. New literacies are needed if we are to influence the future development of digital life. The modern from of empire is based in Silicon Valley and Palo Alto.
— Pilar Saenz (@mapisaro) April 24, 2015
As William Gibson said, ‘the future is already here — it’s just not very evenly distributed“. By building up web literacties and knowing how to use the web provides a way to build resistance and alternative pathways into content. In a sense, the argument here is that knowledge (as a kind of savoir-fair) is power – or at least empowering. A culture of making – whether webpages, OER, creative endeavours – is also a culture of learning and empowerment.
We need to be more ambitious in terms of taking back control of the web through digital and web literacies. Mozilla is running short training courses and conferences to encourage this culture. We are at a kind of ‘Guttenbergian” moment – the extent to which we get the right kind of solution now will influence how information is produced and shared in future years.
This morning I’m attending a talk given by Sharon Slade about the ethical dimensions of learning analytics (LA), part of a larger workshop devoted to LA at The Open University’s library on the Walton Hall campus.
I was a bit late from a previous meeting but Sharon’s slides are pretty clear so I’m just going to crack on with trying to capture the essence of the talk. Here are the guidelines currently influencing thinking in this area (with my comment in parentheses).
- LA as a moral practice (I guess people need to be reminded of this!)
- OU has a responsibility to use data for student benefit
- Students are not wholly defined by their data (Ergo partially defined by data?)
- Purpose and boundaries should be well defined and visible (transparency)
- Students should have the facility to update their own data
- Students as active agents
- Modelling approaches and interventions should be free from bias (Is this possible? What kind of bias should be avoided?)
- Adoptions of LA requires broad acceptance of the values and benefits the development of appropriate skills (Not sure I fully grasped this one)
Sharon was mainly outlining the results of some qualitative research done with OU staff and students. The most emotive discussion was around whether or not this use of student data was appropriate at all – many students expressed dismay that their data was being looked at, much less used to potentially determine their service provision and educational future (progress, funding, etc.). Many felt that LA itself is a rather intrusive approach which may not be justified by the benevolent intention to improve student support.
While there are clear policies in place around data protection (like most universities) there were concerns about the use of raw data and information derived from data patterns. There was lots of concern about the ability of the analysts to adequately understand the data they were looking at and treat it responsibly.
Students want to have a 1:1 relationship with tutors, and feel that LA can undermine this; although at the OU there are particular challenges around distance education at scale.
The most dominant issue surrounded the idea of being able to opt-out of having their data collected without this having an impact on their future studies or how they are treated by the university. The default position is one of ‘informed consent’, where students are currently expected to opt out if they wish. The policy will be explained to students at the point of registration and well as providing case studies and guidance for staff and students.
Another round of consultation is expected around the issue of whether students should have an opt-out or opt-in model.
There is an underlying paternalistic attitude here – the university believes that it knows best with regard to the interests of the students – though it seems to me that this potentially runs against the idea of a student centred approach.
Some further thoughts/comments:
- Someone like Simon Buckingham-Shum will argue that the LA *is* the pedagogy – this is not the view being taken by the OU but we can perhaps identify a potential ‘mission creep’
- Can we be sure that the analyses we create through LA are reliable? How?
- The more data we collect and the more open it is then the more effective LA can be – and the greater the ethical complexity
- New legislation requires that everyone will have the right to opt-out but it’s not clear that this will necessarily apply to education
- Commercialisation of data has already taken place in some initiatives
Doug Clow then took the floor and spoke about other LA initiatives. He noted that the drivers behind interest in LA are very diverse (research, retention, support, business intelligence, etc). Some projects of note include:
- SOLAR – Society for Learning Analytics Research
- LAK Conference – Learning Analytics and Knowledge Conference
- LASI Workshops – Learning Analytics Summer Institute
- Journal of Learning Analytics
- LACE – Learning Analytics Community Exchange
Many projects are attempting to produce the correct kind of ‘dashboard’ for LA. Another theme is around the extent to which LA initiatives can be scaled up to form a larger infrastructure. There is a risk that with LA we focus only on the data we have access to and everything follows from there – Doug used the metaphor of darkness/illumination/blinding light. Doug also noted that machine learning stands to benefit greatly from LA data, and LA generally should be understood within the context of trends towards informal and blended learning as well as MOOC provision.
Overall, though, it seems that evidence for the effectiveness of LA is still pretty thin with very few rigorous evaluations. This could reflect the age of the field (a lot of work has yet to be published) or alternatively the idea that LA isn’t really as effective as some hope. For instance, it could be that any intervention is effective regardless of whether it has some foundation in data that has been collected (nb. ‘Hawthorne effect‘).
Slides for my presentation tomorrow at the 8th Grace EDEN workshop.
Here are my notes from this afternoon’s session at the ALT-C 2014 conference. There were three presentations in this session.
Richard Walker (University of York) – Ground swells and breaking waves: findings from the 2014 UCISA TEL survey on learning technology trends, developments and fads
This national survey started in 2001 and has since expanded out from a VLE focus to all systems which support learning and teaching. The results are typically augmented by case studies which investigate particular themes. In 2014 there were 96 responses from 158 HE institutions that were solicited (61% response). Some of the findings:
- Top drivers for TEL are to enhance quality, meet student expectations and improve access to learning for off-campus students
- TEL development can be encouraged by soliciting stuent feedback
- Lack of academic staff understanding of TEL has re-emerged as a barrier to TEL development, but time is still the main factor
- Institutions perceive a lack of specialist support staff as a leading challenge to TEL activity
- In future, mobile technologies and BYOD will still be seen as significant challenges, but not top as in last year
- E-assessment is also a leading concern
- Moodle (62%)is the most used VLE, with Blackboard (49%) the leading enterprise solution
- Very small use of other open source or commercial solutions
- Institutions are increasingly attempting to outsource their VLE solutions
- Plagiarism and e-assessment tools are the most commonly supported tools
- Podcasting is down in popularity, being supplanted by streaming services and recorded lectures, etc.
- Personal response systems / clickers are up in popularity
- Social networking tools are the leading non-centrally supported technology used by students
- There is more interest in mobile devices (iOS, Android) but only a handful of institutions are engaging in staff development and pedagogic activity around these
- Increasing numbers of institutions are making mobile devices available but few support this through policies which would integrate devices into regular practice
- The longitudinal elements of the study suggest that content is the most important driver of TEL for distance learning
- Less than a third of institutions have evaluated pedagogical activity around TEL.
Simon Kear (Tavistock & Portman NHS Foundation Trust; formerly Goldsmiths College, University of London) – Grasping the nettle: promoting institution-wide take-up of online assessment at Goldsmiths College
When we talk about online assessment we need to encourage clarity around processes and expected results but learners don’t need to know much about the tools involved. Learners tend to want to avoid hybrid systems and prefer to have alternative ways of having their work submitted and assessed.
There are many different stakeholders involved in assessment, including senior management, heads of department, administrators, and student representatives.
Implementation can be helped through regular learning and teaching committees. It’s important to work with platforms that are stable and that can provide comprehensive support and resources.
Simon concluded by advancing the claim that within 5 years electronic marking of student work will be the norm. This should lead to accepting a wider variety of multimedia formats for student work as well as more responsive systems of feedback.
Rachel Karenza Challen (Loughborough College) – Catching the wave and taking off: Embracing FELTAG at Loughborough College – moving from recommendations to reality
This presentation focused on cultural change in FE and the results of the Feltag survey.
- Students want VLE materials to be of high quality because it makes them feel valued
- The report recommends that all publicly funded programmes should have a 10% component which should be available online
- SFA and ILR funding will require colleges to declare the amount of learning available online and this will not include just any interaction which takes place online (like meetings)
- There is a concern that increasing the amount of learning that takes place online might make it harder to assess what is working
- Changing curricula year by year makes it harder to prepare adequate e-learning – a stable situation allows for better planning and implementation
- Ultimately, assessment requires expert input – machine marking and peer assessment can only get you so far
- In future they intend to release a VLE plugin that others might be able to use
- Within 5 years the 10% component will be raised to 50% – this means that 50% of provision at college level will be without human guidance and facilitation – is this reflective of the growing influence of the big academic publishers? Content provided by commercial providers is often not open to being embedded or customised…
- Ministerial aspirations around online learning may ultimately be politically driven rather than evidence-based.