For the morning of Day Two of the workshop the group split into working groups. I floated between the groups and tried to capture a sense of what was being discussed in each.
Qualitative Data Analysis (led by Freda Wolfenden & David Porter)
Freda presented some key things to consider:
- What does data analysis mean in the context of a research inquiry?
- The relation of data analysis and research dissemination
- Alternative forms of data analysis
- Drawing conclusions from data analysis and evaluating evidence
- Findings should be relevant and credible
- Be aware of the relationship between the research rationale and data analysis
- Several approaches to data analysis might be taken within one research project in order to meet different needs or ask different questions
- Four types of analysis: thematic, frequency, discourse, causal
David then spoke about qualitative data analysis and OER studies. He said that qualitative data is really important for understanding how practitioners perceive the influence of OER on their own practice. He then connected this to the idea of communicating research findings through images and stories, using the ‘Artic Death Spiral’ an as example.
David joined BC Campus in 2003 he become involved in the Online Program Development Fund (OPDF), a programme fubnded by the Canadian government to develop online learning materials under open licence. It became apparent that liberal arts, health and science were the subject areas with most interest in this approach. These projects were seen as successful, but the worry of the funders was that there were pockets of activity rather than wholesale adoption, In 2012, the Canadian government tried to further stimulate adoption by funding the production of open textbooks.
To support this work, research was done into the impact, successes and failures of the OPDF project. They proceeded by interviewing and looking at the existing literature to structure the study, identifying gaps in knowledge and any potential methodological barriers. Seven themes (comprising quality, instructional design, technologies, business models, cultures, and policies and localisation) were identified. This also afforded an opportunity to reflect on the important of establishing how well OER was understood in the various institutions.
Ultimately (Third Generation) Activity Theory (Engestromm, Nardi) was selected as a model for understanding the impact of OER as a whole, and and as a framework for producing and aligning interview questions. Interviews took about an hour and were subsequently transcribed.
Qualitative data analysis can produce richer understandings of context (Weiss, 1995). Coding was done through Nvivo and ATLAS.ti (special software for thematic analysis). 203 codes were identified, and their frequency and proximity to each other were analysed. This was still too many for a reasonable analysis, so these were then clustered into nine overall themes (some including as many as 46 sub-themes). (It’s worth thinking carefully about the relationship between the questions asked and the themes that emerged, since themes are bound to be in the transcript if questions are asked about them – RF.)
The outcome was that a deep understanding of OER implementation was still lacking, and new tools and practices would have to be introduced in order to drive open textbook adoption. This influenced the design of the new framework for reviewing and distributing open resources.
Quantitative Data Analysis
When the Q&A session began I headed over to the discussion of quantitative data. I was coming into the discussion that was already getting into the nitty gritty, but here are some of the points that I took away:
- A consistent approach to categorising resources and how they are used (driven by subject understanding rather than data-driven)
- There are different ways of categorising OER – according to the reason they were produced; formatting; level of production (individual/institutional)
- In producing a matrix for analysing quantitative data there should be some flexibility so as to account for regional / cultural difference, etc.; different groupings might be appropriate for different regions / countries
- Piloting the method with 2-3 studies from within ROER4D is a good way to evaluate the approach taken
- Stratification of the sample can be achieved by categorising the institutions according to size, level, purpose, etc. This would allow for comparison across and within countries
- One approach could be to use respondent codes (or some other naming convention) consistently across both the qualitative and quantitative analysis processes
After coffee the focus moved to strategies for data curation and communication.
Data Curation and Communication
The commitment to Open Research is the foundational principle that guides ROER4D in its approach to creating and sharing research data, documents and other outputs. The five key aspects of Open Research so far identified are:
- Transparency in the research process
- Open licensing on all project outputs
- Maximising human readability and accessibility through multiple locations and open formats
- Maximising machine readability through online open formats (such as .xml)
- Long-term preservation curation and accessibility of outputs through a multi-platform data management plan
However, openness is a concept with which many researchers will be unfamiliar, and that an uncritical approach to openness may result in problems. Thus, the commitment to openness is qualified through the two considerations below:
- Open access to resources where openness adds value
- Protection of the dignity and privacy of individuals involved
Information needs to be organised and communicated if it is to have impact and good visibility. This is especially important for ROER4D in raising awareness of OER use in the Global South. Once this is in place then it can be communicated to different audience in an iterative feedback loop. For ROER4D, there are particular challenges around languages, diverse culture, and measuring the impact of the project. Effective use of metadata is crucial here – we might even call it a ‘love note to the future‘! URI / DOI should be employed to track the use of data by others. It’s also important to make sure that you comply with your own institutional data curation policies.
When thinking about whether to release data on a CC0 licence it’s important to realise that this does not require attribution and there’s nothing to stop anyone working with this data and failing to give any attribution to the original researchers. Effective registration of metadata about the project outputs on repositories will encourage better propagation of the research.
One thing we didn’t have time to discuss was how it was anticipated that people would arrive at the UCT repository in the first place. (Maybe the idea is that the OpenUCT repository has good integration with search engines.)
Both a book summarising the ROER4D project and an interactive research report are anticipated. The latter could include multimedia content summarising different strands of the work and link through to the more detailed reports and the open data itself. Through a modular approach to reporting it should be possible to generate reports with different emphases or geospatial dimensions.
After lunch, Atieno Adala gave a neat summary of things to think about when writing research articles and gave an overview of good practice.
I then presented some work from the OER Research Hub and OER World Map projects. This was an impromptu activity, but a good opportunity to bring the map project to the attention of a network who are potentially really important for uptake among the global community. Here are the slides I used, some of which are taken from the OER 15 presentation last week.
Next Patricia Arinto gave an overview of the different dimensions of impact that the ROER4D Impact Studies will look at. These covered a range of the spectrum of potential OER impact such as educator and student practice, institutional impact, effect on quality of resources,
From this point on the meeting broke into smaller working groups and I drifted off to the GO-GN Global Graduate Network meeting of PhD students, some of whom are likely to spend more time at The Open University (UK) which is taking over administration of the network.
Both the ROER4D project and the GO-GN network have tracks in the OER Global conference as we progress through the week in picturesque Banff.