This presentation outlines some key considerations for researchers working in the fields of open education, OER and MOOC. Key lines of debate in the open education movement is described and critically assessed. A reflective overview of the award-winning OER Research Hub project will be used to frame several key considerations around the methodology and purpose of OER research (including ‘impact’ and ‘open practices’). These will be compared with results from a 2016 OER Hub consultation with key stakeholders in the open education movement on research priorities for the sector. The presentation concludes with thoughts on the potential for openness to act as a disruptive force in higher education.
Today I’m in London for the Digital University Network Seminar at the Society for Research into Higher Education. Lesley Gourlay began proceedings by noting that openness is an area which needs to be looked at in the context of the ‘digital university’ series. Here are my notes on the presentations by the other two speakers.
Openness and praxis: Exploring the use of open educational practices for teaching in higher education
Catherine Cronin (National University of Ireland, Galway)
Catherine’s talk was focused on how actors in higher education make sense of OEP. She emphasized that “education is inherently an ethical and political act” (Michael Apple). As educators we face fundamental questions about the role of higher education in the future, and the kind of skills and literacies we are trying to develop. She believes we need more criticality, more theoretical work and more focus on privilege.
In her PhD work the focus is on the ethos of transparency and sharing. Some of the learning spaces in higher education are experiencing changing boundaries, becoming more networked and less bound by physical space. In ‘open’ spaces different voices and interactions are emphasized. Much has been published on openness. There are many different interpretations, but there are few empirical studies, or studies that adopt a critical approach. How do people make choices around the benefits and risks? It was noted that openness cannot it itself be considered an educational virtue.
OEP are perhaps even harder to define than OER. Some approaches include open pedagogy, critical (digital) pedagogy, digital scholarship and networked participatory scholarship. Further complexity is added by the different levels of application (from individual to institutional, for instance). Catherine’s research looks at shared values, the use of OEP in teaching and way that decision-making about OEP adoption takes place. A constructivist grounded theory approach is taken (Strauss & Corbin, 1990) with analysis that acknowledges the subjective and interpretivist understandings of individuals (Charmaz, 2014).
So far it has been found that it is hard to determine who qualifies as an “open practitioner” because there is a wide spectrum of practices and pedagogical choices. A minority of participants use OEP for teaching (e.g. social networking, open VLE, use & reuse of OER, . Most perceive potential risks with OEP. Findings include:
- 2 levels of OEP use identified:
- “Being open”
- “Teaching openly”
- 4 dimensions shared by educators
- Balancing privacy and openness
- Developing digital literacies
- Valuing social learning
- Challenging traditional teaching
Catherine suggests that these are intimately connected. For instance, it is impossible to effectively manage online privacy without developing digital literacies. Valuing social learning involves implicitly challenging traditional learning approaches.
Some educators talk about openness as a kind of ethos or way of being. Others see it as a distraction, or as a pragmatic approach. These differences can be observed as the nano, micro, meso and macro levels. Most guidance is offered at the macro level, but the day-to-day decisions are smaller and less well supported. Other issues that were highlighted were the anxiety and stress experiences by individuals who feel that by being open they are inviting observation and possibly controversy; and the sense that institutions are not providing adequate support.
Some general conclusions:
- OEP use is complex, personal, contextual and continuously negotiated
- More evidence is needed on the actual experiences and concerns of staff and students
- Open education strategies need to reflect the real benefits and risks
- HEIs should provide support for developing digital identities, navigating tensions between privacy and openness, and spaces to reflect on changing roles in a more participatory culture.
Exploring higher education professionals’ use of Twitter for learning: issues of participation
Muireann O’Keeffe (Dublin City University)
Muireann’s research focused on use of Twitter by 7 HE professionals. Martin Hawksey’s TAGS explorer was used to collect data. Semi-structured interviews followed – these underwent thematic analysis. Some important theoretical influences:
- Eraut (2004) identifies three factors for informal learning: feedback; challenge; confidence/commitment.
- White & Le Cornu (2010) on ‘spaces’ rather than communities of practice and the distinction between ‘visitors’ and ‘residents’ (http://firstmonday.org/article/view/3171/3049)
‘Visitors’ tended to be information gatherers, with little social presence. They tend not to ask questions of others.
- Barriers for this group include time, cautiousness, vulnerability, capacity to participate, confidence
- A tendency to lack confidence in their own knowledge
- Tendency to think of themselves as an observer rather than participant
- A belief that the platform was designed for someone else – not them
- Feel marginalized and excluded
‘Residents’ positively experience questioning, challenge, and other forms of academic debate on Twitter. They engage in non-educational commentary.
- Unlike the ‘visitors’, this group tended to speak in terms of enablers
- They are confident with Twitter etiquette: playfulness, tone, etc.
- They were more likely to have a professional confidence, and a capacity to participate
Today at Open Education 2016 I presented the provisional results of a research consultation exercise we have been doing at OER Hub over the last year. Several people asked for copies of the slides, which are available here and on the OER Hub SlideShare account.
All feedback welcome. You can still take part in the project by completing the form at tinyurl.com/2016ORA.
This session began with an introduction from John Hilton III, who leads the OER Research Fellows programme. The project is intended to build future research capacity in the OER field. Most of the work done by this group uses the COUP framework, which focuses on cost savings and learning outcomes. At present there are:
- 43 Fellows
- 18 articles submitted
- 1 article accepted
- 1 article published
Marcela Chiorescu of spoke about her work at Georgia College. On an algebra course, $86 was saved per student, and students expressed gratitude for monies saved. Between Spring 2014 (78.2%) and Spring 2015 (84.3%) there was a significant uplift in students receiving a C grade or above. There was also a statistically significant increase in the proportion of students receiving the top grade.
Christina Hendricks and Ozgur Ozdemir spoke about their work with the COOL4ED project in California. They focused on faculty motivations, cost savings for students, perceptions and impact on other factors. The OER included an OpenStax textbook on Sociology and a Libretext on Chemistry. They found that students has some negative attitudes towards the content of open textbooks as being rather basic. The impact on learning and retention outcomes were less clear because fewer faculty reported back on these. However, no-one reported a decline while some reported an improvement. Cost savings was the most prominent aspect for both faculty and students. Only 4% of faculty and 12% of students had anything negative to say about the open textbooks.
— COOL for Education (@cool4ed) November 2, 2016
Tsung-han Weng (University of Kansas) reported on a qualitative case study involving students from economics and statistics. He found that students tend to have ambivalent attitudes to open textbooks. They appreciated the cost savings but had some reservations about content and quality. This ambivalence was also found in teachers, whose main complaint was that using the open textbook required them to spend more time preparing assessments and supplementary materials.
Royce Kimmons (Brigham Young University) told conference about allowing students to select which textbook a project management studies class would use. Students decided the evaluation criteria (not including cost). What were the effects of this approach? The two most popular choices were subjected to a more detailed evaluation. They arrived at the conclusion that an open textbook was the best offering. Kimmons recommends involving students in the selection prices, arguing that textbook quality metrics are not objective, but relative to the needs of a particular class.
— Christina Hendricks (@clhendricksbc) November 2, 2016
Christopher Lawrence (Middle Georgia State University) spoke about the Affordable Learning Georgia initiative, which aimed to replace proprietary textbooks on American government with open versions. It was found that most students obtained used or new copies of the traditional text. On the whole, they felt that the proprietary version should continue to be used. In comparison with the traditional book, the quality of the open textbook was perceived to be lower. The online version of the open textbook was found to be a useful supplement. However, there was no significant difference in results between those using commercial and open textbooks. Particular challenges in this context included a poorly funded production process which led to a lack of polish in the open textbook; fixed textbook content; and a lack of ancillary materials. An emphasis on the need for sustainability was mentioned.
There was a question from the floor about open access publication of results. The Open Research Fellows are not committed to open dissemination – indeed, there is funding set aside for publication fees – but anonymised research data could be shared.
The blog has definitely felt a bit neglected of late. This is partly because I’ve been quite busy over the summer and blog updates are usually the first thing to go when you have a lot of writing on. But I’ve been posting at OER Hub and OER World Map, so it hasn’t been a complete hiatus.
I thought I would write a summary of the various piece of work that I have on at the moment. This is partly about facilitating later reflection on these projects and how they have developed, but also to offer some online record of my activities in case they are interesting to people who might want to connect.
I’ve also just come from a ‘Developing Researchers’ workshop in IET where we discussed online presence and how we present our activities, so I have some impetus to get this done. First of all, here are the various projects I’m working on right now:
Open Education Research Hub (OER Hub) has managed to establish itself as an ongoing identity for various pieces of work around OER and open education. The main recent deliverable work we have been doing as the OER Hub itself is designing and delivering a range of open courses (including several which can now be called ‘award winning!).
There isn’t as much of a dedicated focus on evaluating the impact of OER implementations as we used to have, but I have recently completed some preliminary analysis of the OER Wales Cymru project that could be developed further. Another strand of OER Hub work that I am leading is one we call ‘The Open Research Agenda’. This project takes an action research approach to trying to develop an understanding of the research needs of the open education / OER movement. We started off with a simple online survey that anyone can take. The results are then taken to face-to-face meetings with representatives of the OER community for discussion. Data is collected at these meetings, becoming part of the data set for future sessions. So far we have held sessions at:
- The William and Flora Hewlett Foundation’s Annual OER Meeting 2016 (Feb 16-19, Louisiana, USA)
- Open Education Global 2016 (Apr 14, AGH University of Science and Technology, Krakow, Poland)
- OER 16 (Apr 20, University of Edinburgh, UK)
- CALRG Annual Conference 2016 (Jun 6, The Open University, Milton Keynes, UK)
Still to come we have the following sessions:
- 5th International Meeting of OERu Partners (Oct 3, University of the Highlands and Islands, Inverness, UK)
- Open Education 2016 (Nov 2-4, Richmond, VA, USA)
Once the last of these is done I’ll be writing up the results. The experimental action research approach seems to work in terms of promoting engagement, so hopefully it will produce something interesting from a research point of view.
I haven’t written much about OER World Map here but the project has come on a really long way since I joined it in 2014. The best way to catch up is to check out the project blog. Although I have some input into the design and technical side, my main role is to act as a conduit to the OER community (which happens through running the Twitter account, collecting data from practitioners, and, recently, drafting papers). Having worked on the OER Impact Map which took a much more basic approach to the underlying data, I continue to be impressed with the developers, designers and library scientists I work with on the project and the meticulous approach to data. Now we have a pretty good infrastructure in place I’m hoping the next year will see a real uptake by the OER community. Earlier this month I hosted the first UK meeting of the OER World Map project here at The Open University, UK.
Another big element of the OER Hub portfolio is the Global OER Graduate Network (GO-GN) which we took over the administration of last year. GO-GN is a global network of support and research capacity building for doctoral students working in open education. In addition to running a webinar programme, we hold a face-to-face two-day seminar once a year and bring students together. This year we met in Krakow, Poland, and in 2017 the meeting will be held in Cape Town, South Africa before OE Global 2017. I also built the GO-GN website, where using Reclaim Hosting for the first time has allowed all kinds of experimentation that vanilla WordPress wouldn’t readily permit.
The most recent project I’ve taken on is working as part of an IET consultancy team for the European Union Agency for Law Enforcement Training (CEPOL) who are looking for input into the redevelopment of their online learning portal. This is a bit of a departure from the focus on open approaches (since there are operational issues around the security of the information) and there are other concerns perhaps unique to law enforcement training. I’ve primarily been involved in shaping some thoughts around the vision and design principles, but also carrying out some qualitative evaluation of the legacy site. It’s been quite stimulating in a number of ways.
There have been various bits of travel associated with these different projects, including several trips to Germany for OER World Map and to Budapest for CEPOL. I was was invited to give two keynote presentations this year: at the Open Education Symposium at Utah Valley University and at the 3rd e-Learning and Distance Education Conference at Virtual University Pakistan. I was also pleased to be invited to the annual OER meeting of The Hewlett Foundation which took place in Louisiana earlier in the year.
But having done a fair bit of travelling over the last five years I’ve attempted to slow it down a little in the latter part of 2016 in the interests of completing writing projects. I’ve published several papers so far this year, and have a couple of book chapters in the pipeline. Currently, I have these writing projects on the go:
- I’m contributing to a post-project analysis on forms of OER implementation in the Bridge to Success project with a couple of colleagues
- Writing a history of the OER World Map project that will serve as the basis for a couple of articles
- Re-working a manuscript on OER policy
- I started writing a piece of Popper’s concept of an open society and how this might provide insight into wider normative understandings of openness
- I’ve been considering putting together a monograph based on several papers written over the last few years; or an edited collection (or both).
One theme that I’ve been considering for the last of these is that of utopian approaches to educational technology. I’ll be examining a PhD thesis related to this theme in the new year, which gives somewhat of an impetus to brush up. But I find myself often thinking of Adorno’s work on utopia (which I mainly know through the work of a fellow doctoral student from the University of Essex). Critical theory frameworks are still very relevant to what is happening in educational technology and education more generally. But it can be hard to find the time to work meaningfully on a book-length proposal with lots of project work and shorted writing commitments taking up headspace. The last few weeks have been particularly intense from a grant application writing point of view as the deadlines seem to coincide with the start of the academic year.
I’ve also become more involved in course production at The Open University: specifically in relation to H819: The critical researcher: educational technology in practice where some of the insights we’ve gained through exploring open practices are being shared. The transition from IET student to course writer will hopefully soon be completed by the award of the MA in Online and Distance Education for which I recently submitted the final piece of coursework. I started the MA in 2010 when I was still new to educational technology, and it hasn’t always been easy to find enough time to devote to studying, but it has been really useful for improving my understanding of research practices in educational technology as well as providing insights into the lot of the distance learner. That said, I will be glad to have some more weekends available for other things in the future.
Alongside all this I have been learning to drive for the past year or so, and my exam isn’t too far away. I failed my first test as a teenager nearly twenty years ago and never retook it, so it does feel as if some long-delayed gratification is within reach…
So, basically I feel like I have many irons in the fire at the moment – and if you’ve made it all the way to the end of this post then I salute you. Reflecting on what I have written, it strikes me that it can be pretty demanding to work across such a wide range of activities, but having the connections between theoretical and empirical work, between evaluation and design, and between research and practice allows for a very productive synergy. Getting the balance right can be hard, though, especially when projects have competing timelines.
One thing that’s also of great benefit is being able to draw on the expertise of several strong project teams in moving your own thoughts forward. I’ve been thinking recently about the nature of collaborative work in academia and note that we rarely tend to frame research skills in terms of the way people collaborate. It strikes me that we should both consciously strive to be catalysts for others while being open to allowing others to act as catalysts for us. I don’t suggest that as a grand theoretical statement (although a connection could perhaps be made with open practices) but rather as an attitude towards effective collaboration. Not allowing the perfect to become the enemy of the good is essential here. With lots of different stuff happening concurrently it’s also really important to keep track of how much time and effort is going into each element to make sure one doesn’t suffer at the expense of another; and on that note I sign off.
Here are the slides I’ll be using today for my presentation at the CALRG Annual Conference. The Open Research Agenda is an international consultation exercise focused on identifying research priorities in open education.
You can read more about the project here:
Here are the slides used in the OER Hub workshop at OER16 today.
For the morning of Day Two of the workshop the group split into working groups. I floated between the groups and tried to capture a sense of what was being discussed in each.
Qualitative Data Analysis (led by Freda Wolfenden & David Porter)
Freda presented some key things to consider:
- What does data analysis mean in the context of a research inquiry?
- The relation of data analysis and research dissemination
- Alternative forms of data analysis
- Drawing conclusions from data analysis and evaluating evidence
- Findings should be relevant and credible
- Be aware of the relationship between the research rationale and data analysis
- Several approaches to data analysis might be taken within one research project in order to meet different needs or ask different questions
- Four types of analysis: thematic, frequency, discourse, causal
David then spoke about qualitative data analysis and OER studies. He said that qualitative data is really important for understanding how practitioners perceive the influence of OER on their own practice. He then connected this to the idea of communicating research findings through images and stories, using the ‘Artic Death Spiral’ an as example.
David joined BC Campus in 2003 he become involved in the Online Program Development Fund (OPDF), a programme fubnded by the Canadian government to develop online learning materials under open licence. It became apparent that liberal arts, health and science were the subject areas with most interest in this approach. These projects were seen as successful, but the worry of the funders was that there were pockets of activity rather than wholesale adoption, In 2012, the Canadian government tried to further stimulate adoption by funding the production of open textbooks.
To support this work, research was done into the impact, successes and failures of the OPDF project. They proceeded by interviewing and looking at the existing literature to structure the study, identifying gaps in knowledge and any potential methodological barriers. Seven themes (comprising quality, instructional design, technologies, business models, cultures, and policies and localisation) were identified. This also afforded an opportunity to reflect on the important of establishing how well OER was understood in the various institutions.
Ultimately (Third Generation) Activity Theory (Engestromm, Nardi) was selected as a model for understanding the impact of OER as a whole, and and as a framework for producing and aligning interview questions. Interviews took about an hour and were subsequently transcribed.
Qualitative data analysis can produce richer understandings of context (Weiss, 1995). Coding was done through Nvivo and ATLAS.ti (special software for thematic analysis). 203 codes were identified, and their frequency and proximity to each other were analysed. This was still too many for a reasonable analysis, so these were then clustered into nine overall themes (some including as many as 46 sub-themes). (It’s worth thinking carefully about the relationship between the questions asked and the themes that emerged, since themes are bound to be in the transcript if questions are asked about them – RF.)
The outcome was that a deep understanding of OER implementation was still lacking, and new tools and practices would have to be introduced in order to drive open textbook adoption. This influenced the design of the new framework for reviewing and distributing open resources.
Quantitative Data Analysis
When the Q&A session began I headed over to the discussion of quantitative data. I was coming into the discussion that was already getting into the nitty gritty, but here are some of the points that I took away:
- A consistent approach to categorising resources and how they are used (driven by subject understanding rather than data-driven)
- There are different ways of categorising OER – according to the reason they were produced; formatting; level of production (individual/institutional)
- In producing a matrix for analysing quantitative data there should be some flexibility so as to account for regional / cultural difference, etc.; different groupings might be appropriate for different regions / countries
- Piloting the method with 2-3 studies from within ROER4D is a good way to evaluate the approach taken
- Stratification of the sample can be achieved by categorising the institutions according to size, level, purpose, etc. This would allow for comparison across and within countries
- One approach could be to use respondent codes (or some other naming convention) consistently across both the qualitative and quantitative analysis processes
After coffee the focus moved to strategies for data curation and communication.
Data Curation and Communication
The commitment to Open Research is the foundational principle that guides ROER4D in its approach to creating and sharing research data, documents and other outputs. The five key aspects of Open Research so far identified are:
- Transparency in the research process
- Open licensing on all project outputs
- Maximising human readability and accessibility through multiple locations and open formats
- Maximising machine readability through online open formats (such as .xml)
- Long-term preservation curation and accessibility of outputs through a multi-platform data management plan
However, openness is a concept with which many researchers will be unfamiliar, and that an uncritical approach to openness may result in problems. Thus, the commitment to openness is qualified through the two considerations below:
- Open access to resources where openness adds value
- Protection of the dignity and privacy of individuals involved
Information needs to be organised and communicated if it is to have impact and good visibility. This is especially important for ROER4D in raising awareness of OER use in the Global South. Once this is in place then it can be communicated to different audience in an iterative feedback loop. For ROER4D, there are particular challenges around languages, diverse culture, and measuring the impact of the project. Effective use of metadata is crucial here – we might even call it a ‘love note to the future‘! URI / DOI should be employed to track the use of data by others. It’s also important to make sure that you comply with your own institutional data curation policies.
When thinking about whether to release data on a CC0 licence it’s important to realise that this does not require attribution and there’s nothing to stop anyone working with this data and failing to give any attribution to the original researchers. Effective registration of metadata about the project outputs on repositories will encourage better propagation of the research.
One thing we didn’t have time to discuss was how it was anticipated that people would arrive at the UCT repository in the first place. (Maybe the idea is that the OpenUCT repository has good integration with search engines.)
Both a book summarising the ROER4D project and an interactive research report are anticipated. The latter could include multimedia content summarising different strands of the work and link through to the more detailed reports and the open data itself. Through a modular approach to reporting it should be possible to generate reports with different emphases or geospatial dimensions.
After lunch, Atieno Adala gave a neat summary of things to think about when writing research articles and gave an overview of good practice.
I then presented some work from the OER Research Hub and OER World Map projects. This was an impromptu activity, but a good opportunity to bring the map project to the attention of a network who are potentially really important for uptake among the global community. Here are the slides I used, some of which are taken from the OER 15 presentation last week.
Next Patricia Arinto gave an overview of the different dimensions of impact that the ROER4D Impact Studies will look at. These covered a range of the spectrum of potential OER impact such as educator and student practice, institutional impact, effect on quality of resources,
From this point on the meeting broke into smaller working groups and I drifted off to the GO-GN Global Graduate Network meeting of PhD students, some of whom are likely to spend more time at The Open University (UK) which is taking over administration of the network.
Both the ROER4D project and the GO-GN network have tracks in the OER Global conference as we progress through the week in picturesque Banff.
Catherine Cronin (National University of Ireland, Galway) – Navigating the Marvellous: Openness in Education
Catherine began with a quote that illustrates her view of eduction:
“Education is inherently an ethical and political act.” (Michael Apple)
Catherine spoke about growing up in New York and the political milieu in the 1960s (including the assassinations of Martin Luther King and John F. Kennedy that helped her to grow to political awareness and the role of education for supporting healthy political life. Different people have different parts to play in the political process. Education thus conceived necessitates criticism of what exists, pointing to what has been lost, and identifying possible futures.
Openness: Catherine identifies this with sharing resources and thoughts in a freely available way. Lots of resources that claim to be ‘open’ aren’t necessarily licensed in appropriate ways, and open practices should be understood as a more radical level built on top of this.
“Openness is an ethos, not just a license. It’s an approach to teaching and learning that builds a community of learners” (Jim Groom)
Catherine was keen to identify openness with a kind of humility rather than the hubris of seeking greater attention for one’s work:
“I don’t think education is about centralized instruction anymore; rather, it is the process [of] establishing oneself as a node in a broad network of distributed creativity.” (Joichi Ito)
As networked individuals, we need to overcome the distinction usually recognised between formal and informal learning. Students come with different expectations and experiences that they bring to the spaces within which they learn. Couros (2006) refers to the ‘networked’ teacher who makes use of a range of digital technologies.
Learning spaces can be physical or online, and tend to be bounded in different ways. Different spaces can facilitate community building to different degrees, but in any space there will be some voices that are privileged and some which are excluded. When online we experience fewer markers of identity, with differing ideas about the effects of presence and telepresence on pedagogy. Open online spaces tend to disregard institutional, national or physical barriers to entry and so facilitate greater sharing and connectivity.
The network is the organising principle of open online spaces – but how should this work in practice? Openness here refers not to licensing but to the practice of facilitating this connectivity.
When students enter institutions, we can ask them about the tools they use and their views on transparency, privacy, and experimental pedagogies. These discussions can be open, and help to form a shared understanding and expectation. Open discussions can take place on social media which draw on the idea of networked learning. Students should be encouraged to connect across cohorts and levels to build community and learning skills.
We can minimise the power differential between student and teacher through open approaches, though it should be noted that some students worry about being judged for thoughts and contributions shared in the open. Identity is key to understanding these concerns because identities are constructed through dialogue and sharing. Students should be supported in building and trying out different identities because so doing will help build digital skills and confidence. Online identity doesn’t so much transform one’s own sense of self but it can help us become more aware of the contingent and contextual nature of our identities, and help us to see possibilities for being otherwise.
We can see open learning spaces as ‘third spaces’ which are neither formal nor informal but draw on both the skills of formal learning and the informal identities that have a kind of authenticity. One risk with developing e-learning is in believing in a kind of subjectless learner who does not bring their own identity to their learning. We need to recognise difference: gender, race, religion, disability and other potential sources of ‘Otherness’. Open practices are a brilliant first step towards this.
As September beckons, here’s a quick round-up of what’s been happening at the Hub over the past month:
Join us to explore Open Research
As you might be aware, we’ve been working on our School of Open Open Research course over the past few months … August saw the launch of our four-week exploration into of the concept and practices of open research, including how to conduct research ethically and in the open, dissemination and the relation between openness and reflection.
Sign-up is open until Friday 12 September with the course formally starting on Monday 15 September 2014. We’re super excited and honoured to have had so much interest in the course and hope that you’ll be able to join us!
View original post 401 more words