workshops

Research, Impact and the UK Parliament

This event took place in the Darwin Building at University College London on the 7th June 2017 and was organised by the Universities programme at UK Parliament Outreach and Engagement Service.  These are my personal notes which may be of interest to any researchers who wish to improve the profile of their work among policymakers.

 

An Introduction to the UK Parliament

The UK Parliament is made up of The House of Commons; The House of Lords; and The Monarch.  The Monarch’s role is mainly ceremonial and is not a focus for impact activities.  There are typically 650 MPs in The House of Commons.

The Commons is the democratically elected chamber of Parliament.  The party (or parties) who can command the confidence of the House of Commons form the UK government – typically the party with a majority of members.  If no party commands a majority then a a minority government or coalition government may be formed.

The House of Lords used to be largely filled with hereditary peers.  The 1958 and 1999 reforms did away with most of these and most current Lords are life peers.  It is possible to become a Lord through hereditary title; public nomination to the House of Lords Appointments Commission; or (most typically) Prime Ministerial prerogative.  Traditionally the 26 highest ranking bishops and archbishops of the Church of England sit in the Lords.  Life peers can choose to retire but they typically serve for life.  There are 92 places for hereditary peers.  Many peers have an allegiance to a political party, but there are also cross-benchers who retain independence. The House of Lords does not conventionally block bills that were in a government manifesto, and can only delay and request amendments of legislation, not block it.

What does Parliament do?  The main activities are:

  • Making new laws
  • Raising and debating issues
  • Scrutinizing the work of the UK government

Parliament is not the same thing as Government (the party or parties that command the confidence of the commons).  For the development of policy and legislation the focus should be on the Government.  If the focus is on applying pressure or criticising a piece of legislation then Parliament is likely to be a more appropriate place to start.

How does Parliament use academic research?

  • House of Lords/Commons Select Committees (groups of MPs/Lords involved in an inquiry into an area of government activity or spending)
  • Parliamentary Office of Science and Technology (POST)
  • House of Commons Library
  • House of Lords Library
  • Public Bill Committees

How can you contribute to legislation?

  • Respond to consultations (before it goes to parliament, e.g. green & white papers)
  • Make sure the subject specialists at the House of Commons Library knows you and your area of expertise
  • Submit evidence to pre-legislative scrutiny committees and/or Public Bill Committees
  • Brief opposition/backbench MPs and Peers to assist them in legislative debates

You can send 250 word summaries of subject expertise to papers@parliament.uk for the attention of subject specialists.  This means you will be entered onto a register of experts.

 

Academics and the UK Parliament

The Parliamentary Office of Science and Technology (POST) supports and advances the use of research evidence in Parliament.  Core activities include:

  • Producing 4 page briefings for MPs and Lords on topics deemed to be of policy relevance. The process includes literature reviews, interviews with stakeholders, etc.
  • Supporting Select Committees and Libraries (providing contacts and bespoke briefings
  • Connecting Parliament with researchers through events, fellowship schemes, etc.
  • Capacity Building through providing training to Parliamentary staff about using research and research methods

 

There are four POST areas:

  • Biological sciences and health
  • Energy and environment
  • Physical sciences and ICT
  • Social sciences

 

As part of the social science strand some work has been done on Parliamentary engagement through analysis of REF2014 case studies.  20% of impact case studies (N=1282) referred to engagement with Parliament.  88% of UK universities are engaged with Parliament in this way (from all 36 subject areas).  The universities which engaged the most were UCL, Oxford, Cambridge, KCL, Manchester, Bristol and Edinburgh.  The areas of Parliament that were engaged with most commonly were Select Committees (35%); Individual MPs or peers (28%); legislation (11%); debate (11%); APPGs (10%); libraries (3%); parliamentary questions (3%) and POST (2%).

The most typical form of engagement is through citation or mention (37%).  The next common is to provide evidence (18%).  Other examples include giving evidence, consultation, speaking or presenting, or through direct correspondence.

The POST note process provides a way to engage:

  • Written by postgraduate fellows over a period of three months
  • Topics are approved by the POST board
  • You can propose a POST note, or contribute to notes that are presently being worked on
  • First drafts of POST notes are usually written by non-specialists
  • Finished drafts are sent to all MPs and peers and made available through the Parliamentary website
  • The website shows notes that have been approved for drafting as well as work currently in progress
  • There is a mailing list and a Twitter account (@post_uk)
  • It can also be useful to follow the social media accounts of Select Committees or Library Sections, etc.
  • Fellowships are available through research councils, learned societies and charities
  • Academic fellowships are available for academics at institutions which hold an ESRC or EPSRC Impact Acceleration Account (currently being piloted, deadline 30th June 2017)
  • Fellows are increasingly offered the chance to work directly with a Select Committee or Library Section

 

Libraries are keen to work with academics but often too busy to seek them out.    The Commons Library is made up of specialists who produce briefing papers and debate packs; the Lords Library comprises generalists and is a smaller team who focus on answering enquiries.

The contact email address for Libraries is: papers@parliament.uk

 

Engaging with UK Parliamentarians

The first step is to contact your local MP, who can be found on the Parliament website (www.parliament.uk) or by calling the House of Commons Information Office on 020 7219 4272.  Many MPs have a constituency office where they can be contacted.

It may be appropriate to contact other MPs – one approach could be to ask which other MPs might share an interest.

Members of the House of Lords have no constituency, but neither do they have the staff support that MPs have.  Many Lords are busy and have jobs outside of Parliament.  It is important to identify peers who will support your campaign. The email address  contactholmember@parliament.uk can be used to contact any peer.  Don’t bulk send information – if more than six copies are received all are deleted.

Individual parliamentarians are free to evaluate your communication and act in whatever way they feel is appropriate. There is no formal quality assurance process, so finding a sympathetic ear can be useful.  Targeting communication is important; having some sense of the action that you wish Parliamentarians to take helps to structure and strategically focus the communication.

Ways to find out more about Parliamentary interests:

  • All-Party Parliamentary Groups
    • These informal groups function like clubs and have been demonstrated to be a good way for researchers to gain influence
    • APPGs typically focus on a particular issue (‘subject groups’) or country
    • They can operate in wildly different ways because they are not uniform in their organisation or structure and they are self-run
    • There is a register of APPGs on the parliament.uk website
    • It can also be useful to identify Parliamentarians who might be obstructive to your legislative agenda through searching APPGs
    • Granularity can be an issue: ‘health’ is quite a broad category but there can be groups for specific areas of medicine or even specific medical conditions. Find the right group for your particular agenda
    • Ask for a list of contact details for members of the relevant APPG
  • Hansard (records of debate)
    • These can be searched for key words
    • Useful for identifying Parliamentarians
  • Early Day Motions (suggestions for future debates)
  • Select Committees

Another inquiry point could be to identify clerks or co-ordinators and contact them directly.  They are likely to be organised and quick to respond.

If you’re going to contact an MP or peer, how should you present yourself?

  • Be polite
  • Have a clear purpose for contacting them
  • Try to stand out from the hundreds of other emails they have received that day
  • Be clear about the new knowledge that has been produced by your research
  • Communicate broad lines first and drill down into the details
  • Parliamentarians are ‘intelligent non-specialists’ who are used to taking in complex information – there is no need to dumb down research for them but it is good practice to minimise jargon and communicate the main points clearly

 

What is good Select Committee evidence?

Select Committees:

  • Are intended to hold the government (or relevant governmental department) to account
  • Are independent in terms of their focus
  • Examine expenditure, administration and policy of each Government department
  • Do not investigate individual complaints
  • Are cross-bench and reflect the makeup of the House of Commons – serving ministers are not part of Select Committees

 

How Select Committees work:

  1. Choose inquiry
  2. Announce Terms of Reference (narrowing areas of focus)
  3. Open the call for evidence (typically open about 6 weeks)
  4. Collect written evidence
  5. Commission research (this is infrequently done, but still happens)
  6. Visits (to relevant stakeholders)
  7. Take oral evidence (these sessions are open to the public and sometimes televised)
  8. Discuss conclusions & recommendations
  9. Draft and agree report
  10. Publish report
  11. Receive a response from the Government, who are obliged to respond to each of the specific recommendations

Recommendations may be accepted in full or in part, or rejected.

Why engage with Select Committees?

  • Evidence based policymaking
  • Publicise research
  • Impact

How should one engage with a Select Committee?

  • Submit written evidence
  • Oral evidence
  • Act as a specialist advisor
  • Highlight relevant research

What is good evidence?

  • Relevant to the inquiry
  • Accessible, not academic – minimise jargon
  • Provides context and assesses the significance of a piece of research
  • Gives clear recommendations to the Committee
  • Avoid political point-scoring since Select Committees are cross-bench
  • Bear in mind the original terms of reference (and possibly use this to structure the report)

One area of focus for Select Committees is to improve the diversity of those who are asked to provide evidence.

The best place to start when thinking about approaching a Select Committee is Twitter – every SC has a Twitter account where requests are made.

Things to think about:

  • How is your research relevant to public policy?
  • Which inquiries could you submit evidence to?
  • How will the REF influence your research?

On the subject of REF:  it is not entirely clear how one might use Parliamentary activity as a way of demonstrating impact.  Keeping records of engagement (e.g. a letter of thanks) is a good idea because this could potentially be used as part of a case study.

Select committees have no role in legislation, though they may be asked to provide scrutiny on bills that are in early stages.

Bill committees also take evidence on a particular subject and related legislation.   Bill committees are run by the Public Bill Office and chosen by party whips.  They are more political and controlled by political parties.

 

 

Advertisements

Critical issues in contemporary open education research #srhe

This presentation outlines some key considerations for researchers working in the fields of open education, OER and MOOC. Key lines of debate in the open education movement is described and critically assessed. A reflective overview of the award-winning OER Research Hub project will be used to frame several key considerations around the methodology and purpose of OER research (including ‘impact’ and ‘open practices’). These will be compared with results from a 2016 OER Hub consultation with key stakeholders in the open education movement on research priorities for the sector. The presentation concludes with thoughts on the potential for openness to act as a disruptive force in higher education.

#liveblog: Critical Perspectives on ‘Openness’ in Higher Education #srhe

Today I’m in London for the Digital University Network Seminar at the Society for Research into Higher Education. Lesley Gourlay began proceedings by noting that openness is an area which needs to be looked at in the context of the ‘digital university’ series.  Here are my notes on the presentations by the other two speakers.

Openness and praxis: Exploring the use of open educational practices for teaching in higher education

Catherine Cronin (National University of Ireland, Galway)

Catherine’s talk was focused on how actors in higher education make sense of OEP. She emphasized that “education is inherently an ethical and political act” (Michael Apple). As educators we face fundamental questions about the role of higher education in the future, and the kind of skills and literacies we are trying to develop. She believes we need more criticality, more theoretical work and more focus on privilege.

In her PhD work the focus is on the ethos of transparency and sharing. Some of the learning spaces in higher education are experiencing changing boundaries, becoming more networked and less bound by physical space. In ‘open’ spaces different voices and interactions are emphasized. Much has been published on openness. There are many different interpretations, but there are few empirical studies, or studies that adopt a critical approach. How do people make choices around the benefits and risks? It was noted that openness cannot it itself be considered an educational virtue.

OEP are perhaps even harder to define than OER. Some approaches include open pedagogy, critical (digital) pedagogy, digital scholarship and networked participatory scholarship. Further complexity is added by the different levels of application (from individual to institutional, for instance). Catherine’s research looks at shared values, the use of OEP in teaching and way that decision-making about OEP adoption takes place. A constructivist grounded theory approach is taken (Strauss & Corbin, 1990) with analysis that acknowledges the subjective and interpretivist understandings of individuals (Charmaz, 2014).

So far it has been found that it is hard to determine who qualifies as an “open practitioner” because there is a wide spectrum of practices and pedagogical choices. A minority of participants use OEP for teaching (e.g. social networking, open VLE, use & reuse of OER, . Most perceive potential risks with OEP. Findings include:

  • 2 levels of OEP use identified:
    • “Being open”
    • “Teaching openly”
  • 4 dimensions shared by educators
    • Balancing privacy and openness
    • Developing digital literacies
    • Valuing social learning
    • Challenging traditional teaching

Catherine suggests that these are intimately connected. For instance, it is impossible to effectively manage online privacy without developing digital literacies. Valuing social learning involves implicitly challenging traditional learning approaches.

Some educators talk about openness as a kind of ethos or way of being. Others see it as a distraction, or as a pragmatic approach. These differences can be observed as the nano, micro, meso and macro levels. Most guidance is offered at the macro level, but the day-to-day decisions are smaller and less well supported. Other issues that were highlighted were the anxiety and stress experiences by individuals who feel that by being open they are inviting observation and possibly controversy; and the sense that institutions are not providing adequate support.

Some general conclusions:

  • OEP use is complex, personal, contextual and continuously negotiated
  • More evidence is needed on the actual experiences and concerns of staff and students
  • Open education strategies need to reflect the real benefits and risks
  • HEIs should provide support for developing digital identities, navigating tensions between privacy and openness, and spaces to reflect on changing roles in a more participatory culture.

 

Exploring higher education professionals’ use of Twitter for learning: issues of participation

Muireann O’Keeffe (Dublin City University)

Muireann’s research focused on use of Twitter by 7 HE professionals. Martin Hawksey’s TAGS explorer was used to collect data. Semi-structured interviews followed – these underwent thematic analysis. Some important theoretical influences:

  • Eraut (2004) identifies three factors for informal learning: feedback; challenge; confidence/commitment.
  • White & Le Cornu (2010) on ‘spaces’ rather than communities of practice and the distinction between ‘visitors’ and ‘residents’ (http://firstmonday.org/article/view/3171/3049)

‘Visitors’ tended to be information gatherers, with little social presence. They tend not to ask questions of others.

  • Barriers for this group include time, cautiousness, vulnerability, capacity to participate, confidence
  • A tendency to lack confidence in their own knowledge
  • Tendency to think of themselves as an observer rather than participant
  • A belief that the platform was designed for someone else – not them
  • Feel marginalized and excluded

‘Residents’ positively experience questioning, challenge, and other forms of academic debate on Twitter. They engage in non-educational commentary.

  • Unlike the ‘visitors’, this group tended to speak in terms of enablers
  • They are confident with Twitter etiquette: playfulness, tone, etc.
  • They were more likely to have a professional confidence, and a capacity to participate

ROER4D Workshop, Banff 2015 – Day Two

For the morning of Day Two of the workshop the group split into working groups.  I floated between the groups and tried to capture a sense of what was being discussed in each.

Qualitative Data Analysis (led by Freda Wolfenden & David Porter)

Freda presented some key things to consider:

  • What does data analysis mean in the context of a research inquiry?
  • The relation of data analysis and research dissemination
  • Alternative forms of data analysis
  • Drawing conclusions from data analysis and evaluating evidence
  • Findings should be relevant and credible
  • Be aware of the relationship between the research rationale and data analysis
  • Several approaches to data analysis might be taken within one research project in order to meet different needs or ask different questions
  • Four types of analysis:  thematic, frequency, discourse, causal

image

David then spoke about qualitative data analysis and OER studies.  He said that qualitative data is really important for understanding how practitioners perceive the influence of OER on their own practice. He then connected this to the idea of communicating research findings through images and stories, using the ‘Artic Death Spiral’ an as example.

David joined BC Campus in 2003 he become involved in the Online Program Development Fund (OPDF), a programme fubnded by the Canadian government to develop online learning materials under open licence.  It became apparent that liberal arts, health and science were the subject areas with most interest in this approach.  These projects were seen as successful, but the worry of the funders was that there were pockets of activity rather than wholesale adoption,  In 2012, the Canadian government tried to further stimulate adoption by funding the production of open textbooks.

To support this work, research was done into the impact, successes and failures of the OPDF project.  They proceeded by interviewing and looking at the existing literature to structure the study, identifying gaps in knowledge and any potential methodological barriers.   Seven themes (comprising quality, instructional design, technologies, business models, cultures, and policies and localisation) were identified.  This also afforded an opportunity to reflect on the important of establishing how well OER was understood in the various institutions.

Ultimately (Third Generation) Activity Theory (Engestromm, Nardi) was selected as a model for understanding the impact of OER as a whole, and and as a framework for producing and aligning interview questions.  Interviews took about an hour and were subsequently transcribed.

Qualitative data analysis can produce richer understandings of context (Weiss, 1995).  Coding was done through Nvivo and ATLAS.ti (special software for thematic analysis).  203 codes were identified, and their frequency and proximity to each other were analysed. This was still too many for a reasonable analysis, so these were then clustered into nine overall themes (some including as many as 46 sub-themes).  (It’s worth thinking carefully about the relationship between the questions asked and the themes that emerged, since themes are bound to be in the transcript if questions are asked about them – RF.)

The outcome was that a deep understanding of OER implementation was still lacking, and new tools and practices would have to be introduced in order to drive open textbook adoption.  This influenced the design of the new framework for reviewing and distributing open resources.

Quantitative Data Analysis

When the Q&A session began I headed over to the discussion of quantitative data. I was coming into the discussion that was already getting into the nitty gritty, but here are some of the points that I took away:

  • A consistent approach to categorising resources and how they are used (driven by subject understanding rather than data-driven)
  • There are different ways of categorising OER – according to the reason they were produced; formatting; level of production (individual/institutional)
  • In producing a matrix for analysing quantitative data there should be some flexibility so as to account for regional / cultural difference, etc.; different groupings might be appropriate for different regions / countries
  • Piloting the method with 2-3 studies from within ROER4D is a good way to evaluate the approach taken
  • Stratification of the sample can be achieved by categorising the institutions according to size, level, purpose, etc.  This would allow for comparison across and within countries
  • One approach could be to use respondent codes (or some other naming convention) consistently across both the qualitative and quantitative analysis processes

After coffee the focus moved to strategies for data curation and communication.

Data Curation and Communication 

The commitment to Open Research is the foundational principle that guides ROER4D in its approach to creating and sharing research data, documents and other outputs. The five key aspects of Open Research so far identified are:

  1. Transparency in the research process
  2. Open licensing on all project outputs
  3. Maximising human readability and accessibility through multiple locations and open formats
  4. Maximising machine readability through online open formats (such as .xml)
  5. Long-term preservation curation and accessibility of outputs through a multi-platform data management plan

However, openness is a concept with which many researchers will be unfamiliar, and that an uncritical approach to openness may result in problems. Thus, the commitment to openness is qualified through the two considerations below:

  1. Open access to resources where openness adds value
  2. Protection of the dignity and privacy of individuals involved

Information needs to be organised and communicated if it is to have impact and good visibility. This is especially important for ROER4D in raising awareness of OER use in the Global South.  Once this is in place then it can be communicated to different audience in an iterative feedback loop.  For ROER4D, there are particular challenges around languages, diverse culture, and measuring the impact of the project. Effective use of metadata is crucial here – we might even call it a ‘love note to the future‘!  URI / DOI should be employed to track the use of data by others.  It’s also important to make sure that you comply with your own institutional data curation policies.

When thinking about whether to release data on a CC0 licence it’s important to realise that  this does not require attribution and there’s nothing to stop anyone working with this data and failing to give any attribution to the original researchers.  Effective registration of metadata about the project outputs on repositories will encourage better propagation of the research.

One thing we didn’t have time to discuss was how it was anticipated that people would arrive at the UCT repository in the first place.  (Maybe the idea is that the OpenUCT repository has good integration with search engines.)

Both a book summarising the ROER4D project and an interactive research report are anticipated.  The latter could include multimedia content summarising different strands of the work and link through to the more detailed reports and the open data itself.  Through a modular approach to reporting it should be possible to generate reports with different emphases or geospatial dimensions.

Dissemination

After lunch, Atieno Adala gave a neat summary of things to think about when writing research articles and gave an overview of good practice.

I then presented some work from the OER Research Hub and OER World Map projects. This was an impromptu activity, but a good opportunity to bring the map project to the attention of a network who are potentially really important for uptake among the global community. Here are the slides I used, some of which are taken from the OER 15 presentation last week.

Next Patricia Arinto gave an overview of the different dimensions of impact that the ROER4D Impact Studies will look at.  These covered a range of the spectrum of potential OER impact such as educator and student practice, institutional impact, effect on quality of resources,

From this point on the meeting broke into smaller working groups and I drifted off to the GO-GN Global Graduate Network meeting of PhD students, some of whom are likely to spend more time at The Open University (UK) which is taking over administration of the network.

Both the ROER4D project and the GO-GN network have tracks in the OER Global conference as we progress through the week in picturesque Banff.

ROER4D Workshop, Banff 2015 – Day One

Today and tomorrow I’m in Banff ahead of the OE Global 2015 conference at the invitation of the ROER4D project to take part in their latest research workshop.  I’m interested to learn more about aspects of the project I’ve yet to encounter, and to meet more members of the wider ROER4D network.  (This blog present my impressions of events rather than a verbatim record of what was said.)

ROER4D is a big research project.  There are 18 sub projects and there are 86 researchers working across 26 countries and 16 time zones.  For a lot of the people who have travelled to Banff, Alberta in Canada this is the first time they have met face to face, and about half of them are new to me.

For such a diverse group to work together in the project, it has been necessary to develop a shared conceptual framework and collaborative working practices.  The ROER4D Bibliography of research into OER in the Global South is an important part of working towards such a framework. Here’s an infographic which shows the different strands of work across the project:

The range of data collected by the project is diverse, and to maximise the chances of combining data in useful ways will be increased by co-ordination of methodology and research practice where possible.  Some aspects will be highly contextual but where possible the project should strive to identify themes held in common.  Some research questions will be emphasised in some strands more than others but there is still value in pulling together all the relevant data for the key project clusters.  By mapping what is already known and sharing this throughout the project everyone should benefit from not having to tackle al the research in isolation. The different aspects of the project should complement each other.  This workshop provides an opportunity to create new connections and better coordinate across the project.  The workshop will focus on updating the collective understanding of progress made so far; sharing ideas for data analysis and data visualization; and discussing the  ROER4D final outputs and their anticipated formats.  There are several projects looking at specific areas of OER impact and opportunities for working together on similar issues and themes should be identified.

There was some interest in the OER Research Hub survey questions so I made these available to some of the group via http://tinyurl.com/OERRH-surveys.  Anyone is free to re-use our questions under a CC-BY licence – we only ask that attribution back to the OER Research Hub project is forthcoming in return.

After some brainstorming work the main objectives for the workshop were identified.  Most are keen to try and establish agreed methods for data curation and analysis which can be applied consistently across the project.  Another theme that emerged was the idea of making best use of any data collected through a consistent strategy for exploitation and evaluation.

Presentations

Sarah Goodier presented some work on the workpackage which looks at the role of public funding in supporting OER adoption and advocacy in South Africa.  This comprises desk research and interviews with policymakers and officials. A country report is expected early in 2016.  This presentation provoked some collective reflection around the difficulties of building up a holistic picture of change.

Amalia Toledo (Columbia) spoke about OER policy and advocacy in Chile, Columbia and Uruguay.  They are recording the current methods used by governments in the region to promote OER and Open Access.  Data was pulled from public databases as well as through desk research and interviews. Country reports are being produced (in Spanish) and there is a summary report that was written for UNESCO as well as a publication in Open Praxis.  The overall findings are:

  • A variety of funding sources for public education are identified
  • There is a lack of clear policy support in Columbia and Chile, but less so in Uruguay
  • Programmes in science, technology and innovation are being developed

There was also a series of ‘World Cafe‘ presentations from the impact study grantees to introduce the wider group to their ongoing studies.  I’ll summarise these very briefly here:

SP10.1 Freda Wolfenden (UK) – the OU team will work with teacher educators in West Africa to understand their engagement and response to OER, looking for changes in their understanding of their own practice; their understanding of their own subject; and social order changes within and beyond their institution.  The data will mostly be self-reported by teachers in training and will include attitudinal data as well as self-reported changes in practice.  Some baseline data will also come from more general surveys.

SP10.2 Atieno Adala (Kenya) – this study looks at the impact of OER adoption on expanding access to quality teacher education in sub-Saharan Africa, where there is a lack of well trained teachers.  12 universities across 10 countries will comprise the locations for the study.  Some existing research (e.g. Diallo, 2013, Niang, 2013) suggests that access to an improved curriculum can strengthen institutional capacity.  Evidence will be sought to defend this claim.  The secondary hypothesis will examine whether OER has a positive effect on the quality of the curriculum.  A quantitative analysis of student performance and institutional reporting will be used as evidence.

SP10.3 Michael Glover (South Africa) – study of 3-5 MOOC at University of Cape Town, which is developing a wider MOOC strategy working with FutureLearn.  How do adoption of OER (in a MOOC format) impact upon educator practices? The study will focus on post-MOOC teaching and research practices.  The definition of open educational practices by Beetham et al (2012) which identifies six indicators for OEP will be used to measure changes in practice.  Generally, this seemed like a good approach to measuring impact of this sort.  Interviews and classroom observations will also be conducted, and analytics from the MOOC portal collected.  Activity theory will be used as a conceptual framework.

SP10.4 Lauryn Oates (Canada) – Canadian Women for Women in Afghanistan is running the ‘Darakht-e Danesh‘ programme which makes educational materials available openly as an online library.  Once registered, users can search for OER by type of resource, subject, language, etc. The research will focus on whether access to OER improves teacher subject knowledge or pedagogical practice.  Analytics from the website will provide most of the data for the impact study, with surveys as a follow up.

SP10.5 Yasira Waqar (Pakistan) – investigating impact of OER on secondary and tertiary education in Pakistan.  OER is not popular in Pakistan and possibly associated with ‘devaluation’ of intellectual work (because it is ‘given away’).  Open access is less of an issue in Pakistan as copyright is not particularly well observed or respected.  (It might even be that ‘open’ in Pakistan just means ‘free and online’.)  The main research questions here try to evaluate impact by measuring adoption and identifying benefits to educators and learners, using Fullan’s theory of change as a framework.  Interviews, surveys and classroom observations will be used to collect data about changes in pedagogy, but it will be important to demonstrate the specific  efficacy of OER.

SP10.6 Shironica Karunanayaka (Sri Lanka) – the concept of OER is new to Sri Lanka, and this study will introduce teachers to the concept and ascertain whether or not this leads to some changes in teaching practice.  The hypotheses being investigated is whether integration of open materials into teaching leads to a change in perception of the practices of student teachers and improved quality of teaching and learning materials.  An action research approach will be taken following a professional development programme for student teachers.

The day concluded with discussion in groups according to the different workpackages across the ROER4D project. Because of the structure of the World Cafe session, it meant that the presenters did not see the presentations of others.  We had a discussion around ‘impact’ and the difficulty of establishing a causal relationship between adopting openness and the impacts that result.  It was felt that a general theory of impact as ‘change’ could be a practical way of proceeding and specificity can be brought out in the subsequent analysis.

image

The impact study leads wrote their main research hypotheses on post-it notes and then tried to categorise them into three or four main themes.  Out of this exercise can the following rubric of central themes:

  1. IMPACT ON TEACHING PRACTICES
  2. IMPACT ON PERCEPTIONS
  3. IMPACT ON STUDENT LEARNING/ACHIEVEMENT
  4. IMPACT ON QUALITY OF EDUCATIONAL MATERIALS
  5. INSTITUTIONAL IMPACT
  6. IMPACT ON SUBJECT KNOWLEDGE AND CONFIDENCE (OF EDUCATORS)

Of course, there are other possible ways to perform this categorisation (.g. 6 might be reducible to 1.), and it could be further broken down by subject and teaching level. Wordings of survey and interview questions should be as consistent as possible, and demographic questions should be absolutely consistent across both the impact studies and the ROER4D project as a whole.  The project leaders agreed to work together to harmonise their questions across the key hypothesis areas.

The OER Research Hub question bank might provide some inspiration for the wording of questions asked across the impact studies.