Find and work collaboratively on open educational resources

Three workshops were organised by the ROLE team of the Open University (OU), presenting the basic scenario of a PLE to the audience, followed by an opportunity to experience using pre-selected ROLE tools. The underlying theme of the workshops was: “Finding and evaluating OER”, but the flexible nature of the embedded activity was such that individuals could tailor this theme to meet their own specific needs i.e. by choosing to look for or discover OER pertinent to their own subject areas. The workshops were conducted using ROLE tools with different groups, i.e. with learners, educators, and researchers. Collecting feedback from each group was organised through a survey. This generated both quantitative as well as qualitative data. Representatives from the ROLE project were present at each workshop to deliver information and to circulate during the hands-on part of the session. This was an excellent opportunity to hear how individuals did or did not engage with the ROLE tools. It was a chance to collect some direct qualitative data through comments and feedback from participants.

The first workshop took place at the Joint European Summer School on Technology Enhanced Learning (JTEL) in Crete, May 2011. Participants were postgraduate students from universities across Europe. The JTEL Summer School is an annual event and offers an opportunity for PhD students, in different subject areas, in TEL to meet, exchange knowledge and develop their research skills whilst engaging with the active TEL community of practice. The second workshop took place at The Open University, Milton Keynes, UK in July 2011. It was organised in conjunction with the Support Centre for Open Resources in Education (SCORE). SCORE offers a variety of support mechanisms to the OER community in England. The attending SCORE Teaching Fellows are appointed from a cross-section of English Higher Education Institutions (HEIs). The third workshop took place at the Open University during January 2012. It was organised in conjunction with the eLearning Community (eLC), who comprise of an established and active group of Open University staff with a common interest in educational technology development and its many applications in Higher Education scenarios. The workshop was part of a monthly showcase event demonstrating a wide range of both research and implementations of current eLearning applications. 

A similar but not identical, workshop format was used at each event. Whilst the workshop basis was the same (e.g. setting the scene, describing PLEs etc.), the hands-on materials and pre-selected ROLE tools were tailored for the different audiences. After a short introductory presentation about ROLE and PLEs a short question and answer session followed. The main hands-on section of the workshop was then delivered in the form of an activity. Essentially participants were asked to visit the dedicated OpenLearn webpage shown in Figure 1. This enabled the participants to access a group of pre-selected ROLE tools in the form of widgets.

Participants in all three workshops were asked to use the Binocs and ObjectSpot search widgets. Engaging in this activity would enable them to find OER that would be suitable to support them in their respective research or teaching scenarios. The EtherPad widget was also available for this activity and it enabled participants to report their findings in a collective notepad format. The FlashMeeting widget was not used in this activity, as all the participants were physically attending, therefore no videoconferencing facility was required. At the end of each workshop, a group discussion was also held with the participants contributing about their experiences of using the ROLE tools. Finally, the participants were asked to answer a short online survey (see example at https://fit-bscw.fit.fraunhofer.de/pub/bscw.cgi/39223921).

The results of all the workshops were recorded in a number of formats. It is fair to say that primarily quantitative data was collected from the questionnaire whilst the majority of the qualitative data was collected in situ when facilitators circulated amongst the participants. Secondly, however, some supplementary qualitative data was also gathered electronically via the pre-selected ROLE tool that enabled participants access to the EtherPad and record their experiences as they happened. In general, some participants were comfortable with using the EtherPad whilst others were most definitely unable to grasp the concept or indeed use it effectively. It was for this reason that the facilitators at each workshop collated notes of what they observed and heard during each event. It is important to note that the questionnaire also contained a number of semi-structured questions permitting free text individual responses. 

Figure 1. The setup of ROLE widgets used in the “Finding and evaluating OER” workshops

Results from the JTEL workshop

The first event, as previously mentioned, took place during the JTEL summer school in Crete, May 2011. The audience comprised of approximately 25 PhD students, all of whom were aged between 21 and 40. There was an even split between the genders. Most participants declared that they had a good knowledge of TEL (73%) whilst the majority also indicated that they had “some” knowledge of OER (73%). The purpose of the workshop being that participants were encouraged to use the ROLE tools to seek out appropriate OER materials that would support them in their subject areas of research.

In general the JTEL participants overall opinion of using the ROLE tools as part of the learning activity in the workshop was a positive one. Participants recorded in the free text responses of the questionnaire that their experiences of using the tools were “...useful, especially the search widgets” along with “LOVED THEM!!! I found them really useful both for search and collaboration” and “a great idea”. Collating the responses to the fifth question (What did you think of the widgets of the learning activity?) which was also a free text response, one can see that the overall opinion recorded was positive (80%) alongside a much smaller negative response (10%) as well as a small neutral response (10%).

With relation to a PLE scenario the responses were much more mixed (see Figure 2). Interestingly the groups’ strongest opinion related to the statement “Using a PLE would improve my motivation for learning” where some 57% registered a neutral response to this premise. Other strong opinions were also voiced in respect of the statements “I would find a PLE useful for my work” where some 52% agreed with 21% strongly agreeing and “I would find interacting with a PLE requires a lot of mental effort” invited a 52% disagreement to be recorded. This would suggest that many of the participants recognised that using a PLE required some effort initially along with a discerning thought process but such effort would offer individuals greater benefits in the long run. The remaining statements in this question invited a more evenly spread set of responses.

Figure 2. Responses to questions regarding the usefulness and ease of use of PLEs

Question 7 related to the main premise of the workshop i.e. participants’ success in finding relevant OER results from their enquiry using one of the ROLE tools (either Binocs or ObjectSpot). It was an opportunity, once again, for them to record their actual opinion using a free text response. Overall the majority (70%) recorded a positive use of the ROLE tools to find relevant OER materials, whilst a small number rated the experience as neither positive nor negative (20%). Only 2 participants, in fact, replied negatively (10%). Recorded comments to this question ranged from simply stating “Yes” through “I found some very useful resources for my research”. Some participants chose to record exactly what they found e.g. “Mainly videos and images” or “YouTube, Slideshare” whilst others commented about the actual process, for example: “...finding relevant (materials) ones is hard” or “It was good to be able to see how different licences can be used and how to to use the resource for my work”.

Participants were also invited in the final question of the survey to record their opinions related to comments or questions for improving the ROLE tools. Significantly fewer responses were recorded in response to this invitation (50% of group total). It is not clear why this is so. Nonetheless, some useful ideas were recorded, such as “it would be useful that each resource had more indications about how rich it is. Not only number of comments, but also links, embedded content etc”. In other words, the participant recognised the value of the ROLE tool for his/her research work and wanted more relevant information to be displayed once a search query had completed i.e. that materials were situated in a wider context (in this case in relation to OER subject matters).

The EtherPad widget, as indicated earlier, was received by workshop participants in different ways. Most of the research students, in this JTEL event, actively used the tool although some were a little surprised by the real-time aspect of it “...somebody is writing on my screen!!!! I am scared...”. Others considered additional aspects to the experience in that it highlighted some potential gaps in their own skill set “...I probably have to work on my search skills...”. Overall, however, there was a positive, yet critical, response to this invitation indicating that the students who chose to record their thoughts in the EtherPad widget did give some considered attention to their discussion and/or notes.

Other interesting responses ranged from “... I like that I can narrow down the search results to just pictures, apps etc” to suggestions that “...This one is very good! You find the licence and you search for it” indicating, once again, that some students were discerning users of the ROLE tools and thinking through a number of previously unconsidered approaches or ways of using such search engines. Others focused on previous experience e.g. “I used this tool in a conference, we took notes dude!” as well as the not unusual student response of “So... in the morning we can actually make notes together instead of coming to group therapy :)” suggesting that virtual communication might be a replacement for those who were reluctant to be early risers.

The workshop facilitator also noted that students chose to work in teams of two and that no significant technical issues were experienced during the event. She noted that, in some cases, a number of students search results were irrelevant and that the majority of results appeared to return YouTube video links. The latter would appear to happen if all the options in Binocs, in particular, are left checked which is the default option of this ROLE tool. This may suggest that it would be better to leave the default delivery of the tool unchecked thus inviting users to select and check the search engines that are relevant to them/their research.

The facilitator also documented that the majority of students appeared to focus on the Binocs tool rather than the ObjectSpot tool depending upon the type of material that they were interested in finding. There may have been a number of reasons for this occurrence e.g. Binocs was offering OER related materials whilst ObjectSpot concentrated on bibliographic searches which may have been more easily identifiable to the students. She also recorded that at least half the group only used these two tools and did not appear to engage with the EtherPad tool at all. Again, as indicated earlier, there could be a number of reasons explaining this situation.

Results from the SCORE workshop

The second workshop took place with an audience of 10 educators, whose age profile was somewhat in contrast to the JTEL Summer school: the majority were SCORE Teaching Fellows who were aged 30-50 with a 60:40% female:male division. Their knowledge of TEL also invited a wider range of responses in that 30% recorded themselves as “experts” with 40% stating “good knowledge” alongside 30% saying that they had “some knowledge”. They also recorded an identical response in respect of their OER knowledge. Once again the purpose of the workshop was that participants were encouraged to use the ROLE tools to seek out appropriate OER materials that would support them in their subject areas of either their Teaching Fellow or “normal” research.

The associated evaluation questionnaire for this second workshop contained similar questions to the JTEL Summer school student version but some were different (as illustrated below). In respect to question five “What did you think of the widgets of the learning activity?” the educator participants responded with an even split between positive and neutral comments such as “Good in principle, liked the ability to search file type. Needs wider range of search engines. Didn’t work properly on the iPad” as well as many responses of “useful/nice idea/worked well in general” in addition to “... but would be even better if the search results were filtered for Creative Commons licenced items” indicating that the educator recognised the potential of the search tool to be further refined.

There was a rather mixed response to question 6 “Your opinion of a widget-based PLE” but overall many of the educators (ranging from 40-60%) registering that they were neutral in their opinion of the nine listed statements. Likewise, only 10-20% of participants registered either strongly agreed or disagreed views. For example, with respect to the statement “It would be easy for me to use a PLE”, some 40% of the educators agreed but, as indicated earlier, another 40% held a neutral view of this statement alongside the remaining 20% registering that they disagreed. 

In relation to the success of participants finding suitable OER materials via the ROLE tools again the response was 50:50 in respect of neutrality and strongly agreeing with the premise. It is difficult to ascertain why this is so other than suggesting that the participants appeared to be reasonably comfortable with the idea of using the ROLE tools even though some experienced technical issues recording that “did not work on iPad”. It was noted by the facilitator and ROLE colleague present that there was a definite positive “buzz” amongst the audience during the learning activity. This manifested in a number of implicit ways: enthusiastic language being used amongst participants; a sense of excitement that emanated in above average noise levels for the group (N.B. The ROLE colleague in attendance is also a Teaching Fellow and a regular attender at these monthly SCORE events).

Question 8 related to the ease or difficulty of adaptation of the OER for the participants own purpose. Some 50% of the educators registered a neutral response with 30% recording that it was difficult along with the remaining 20% stating that it was very difficult to adapt their OER. This corroborated the previous premise that it is easy to find OER materials but less so to disaggregate the contents and repurpose or remix them to meet local needs.

With respect to the educators finding the learning activity useful for research needs and goals, once again the responses were evenly split (50:50) between a neutral stance and strongly positive. The actual comments centred on simple “Yes” replies through “useful but frustrating” to “It was useful to find that the search widget could be customised to a particular project’s needs” confirming that either the facilitator or ROLE colleague had explained that ROLE tools could be adapted to meet local needs too.

The final survey question requested comments or suggestions for improving the ROLE widgets. In exactly the same way as the JTEL Summer school workshop, this invitation revealed a limited number of responses. They were, however, helpful in terms of feedback for the ROLE tool developers and ranged from “support or examples of good use would be helpful - the interface is not immediately intuitive” through “... the search needs to direct users towards OER repositories and/or Google results filtered by licence” to “the search results I got were not necessarily OER”. the latter suggesting that definitions of what is being searched for need to be clearer as well as pre-selecting the most appropriate search engines/repositories rather than a wider set of resources that seem to confuse some of the end users.

It is fair to say that there was little engagement from the educators in this learning activity with regards to the EtherPad widget. Only 9 lines of text were recorded in it within this workshop compared to some 50 lines of text recorded during the earlier JTEL Summer school. To encourage use and demonstrate it, the ROLE facilitator used the EtherPad to record the location of the associated survey as an example of how further resources or links could be shared amongst a wider peer group. Nonetheless some useful information was recorded by the educators, one of whom remarked: “... I was wondering how this search tool chooses content to display and how it displays the search results”. Once again indicating that those who engaged with all of the tools did so with discernment and thought sometimes anticipating further potential refinements to the tools.

Results from the eLC workshop

Approximately 20 people were in attendance at the third workshop. There was an even gender split amongst participants who completed the survey and an indication that the majority of attendees were over 40 years old. This was not an unusual composition for an eLC group as many people who attend these regular events are generally experienced teachers, skilled researchers or significantly qualified learning technologists. The common denominator amongst them being an active interest in eLearning advances as well as hearing first-hand about new and innovative eLearning applications and experiences gained from their fellow colleagues. 

With this in mind those who completed the ROLE survey (about 50% of all participants) declared themselves to have an even split of knowledge in relation to Technology Enhanced Learning (TEL). Conversely, however, a significant 88% of those participants felt that they had “some” rather than a “good” knowledge of Personal Learning Environments (PLEs). In other words it appeared that the group as a whole were relatively new to the idea of PLEs. 

Observation of interaction with the EtherPad and Chat facility revealed that some participants were hesitant to use this type of technology and required encouragement from their peers or more experienced colleagues (either those in the group or from either of the two ROLE representatives who were facilitating the workshop). The types of information recorded by the participants ranged from anxiety to amazement that such tools could enable individuals to learn collaboratively. It should be noted, though, that there was a definite misunderstanding between the focus and use of these separate “recording” tools as demonstrated by one participant who noted that “This is the chat – got slightly confused with the wiki”. 

There was also evidence, however, that a more experienced group member took advantage of the same chat facility to communicate with a colleague in another part of the room about some mutual work unrelated to the subject in hand (i.e. ROLE and PLEs). It should be noted, nonetheless, that the same person used the chat facility firstly to give advice to his colleague regarding more effective use of the tool before moving onto the separate subject. What this also revealed, of course, is that despite encouragement to explore ROLE widgets it was not a compulsory activity and at least one participant chose to continue his own non-ROLE/PLE work as well as participating in the workshop.

It appeared that the EtherPad was used more constructively to exchange as well as record pertinent PLE related information. For example, one participant detailed a blogspot address that they felt to be “an interesting take on PLEs. Just a tad off topic… (Note to self this is the wiki)”. Others noted comments relating to enquiries about how each ROLE widget functioned. These ranged from “how are keywords supposed to work” to “cannot get ObjectSpot to show on iPad2”. Indeed this idea of recording questions was taken somewhat further by one participant who remarked: “Why do we need two widgets for search?”

During the structured activities both the ROLE facilitators moved around the room offering help to participants as required and attempting to aid them with their engagement with the ROLE widgets. This included fielding a wide variety of questions (some of which are outlined above in the EtherPad and Chat recordings) as well as offering constructive advice about how PLEs functioned and where to find out more information relating to this developing area. 

The final question in the survey asked participants to record their feelings about their use of PLEs. There was a 77% agreement that PLEs would be slightly useful for participants work followed by a slight disagreement that the same PLEs would help participants accomplish their work more effectively than their current use of learning technology. Again this was not surprising as most group members were established and experienced users of learning technology and had revealed that they only had limited use of PLEs. 

Half the group proffered a neutral response to the statement relating to “It would be easy for me to use a PLE” whilst the remainder recorded that there was a slight chance that that would be the case. There was a more even spread of responses to the statement “It would be clear to me how to assemble a PLE using widgets” ranging from slight disagreement (11%) through to slight agreement (also 11%). Most participants remained neutral on the subject. 

Interestingly the statement “I would find using a PLE frustrating” invited the most disagreement to be recorded with the majority (55%) remaining neutral alongside 33% saying they slightly disagreed and 11% strongly disagreeing. Once again the following statement of “I would find interacting with a PLE requires a lot of mental effort” statement invited a strong neutrality (55%) yet 22% of participants strongly disagreed with this premise whilst 11% recorded that they slightly agreed that this would be the case fro them. The remaining part of the survey related to participants motivation to using a PLE in their learning process whereby 55% remained neutral in their responses and 44% slightly agreeing with this statement. The last statement of “I predict that I would frequently use a PLE if I had access to it” invited an even response (33%) between slightly disagreeing through neutral to slightly agreeing. 

Conclusion

Overall, the three events were deemed to be quite successful in raising awareness about the ROLE project. The introduction about the remit of the ROLE project set the scene and, additionally, participants appreciated the opportunity to use the selected ROLE tools thus the workshops were warmly received by the audiences. These workshops also provided different audiences (learners, educators, and researchers) an opportunity to try a selection of ROLE tools within a guided learning activity. The reactions of all groups were recorded using a quantitative survey instrument as well as collected in an electronic format via an EtherPad facility. In addition, the facilitator at these events took the opportunity to make notes about general reactions to the ROLE tools, as well as dealing with any immediate queries or technical issues.

Acknowledgment

The OU team would like to thank their colleague Elpida Makriyannis for kindly volunteering to facilitate the ROLE workshop held at the JTEL Summer School in May 2011.