The JLeRN Experiment

JISC's Learning Registry Node Experiment at Mimas

Archive for the month “March, 2012”

Update on the JLeRN Experiment Paradata Challenge at Dev8D

Just a wee update on the results of the JLeRN Experiment Paradata Challenge at Dev8D 2012. The whole team put a lot of effort into the challenge, especially Bharti Gupta, who set up the pre-conference surgery, ran an introductory session at Dev8D, and really got involved in discussing the Learning Registry with developers and following up after the conference finished. A big thanks also to Steven Cook from Jorum who accompanied Bharti to Dev8D, and helped a lot with supporting the challenge.

So: after a lot of interest we had two entries, both in the form of use case ideas around using paradata about books. We (the JLeRN team and the judging panel) decided not to award any prizes as the ideas just weren’t quite developed enough in relation to the Learning Registry, but we did feel these were use cases worth keeping an eye on.

We really want to emphasise that the lack of prizes is not a reflection on the work we know the two entrants put in, in terms of getting their heads around the Learning Registry. It still takes a fairly big push of effort and thought to dig into this stuff, and we were asking developers to do this a bit too quickly, and in their own time. And, it has to be said, without the incentive of a big cash prize at the end! Thank you very much for participating Vani and Neeta!

Ultimately, we came up with the challenge idea as a way of publicising the JLeRN Experiment and more broadly the Learning Registry. We’re not too disappointed at not being able to award any prizes, because the whole experience sparked a fair amount of interest in what we’re doing, and discussion on Twitter and elsewhere. It also highlighted some of the requirements developers identified around supporting their engagement in the Learning Registry. It was genuinely a valuable exercise for the project.

We’d like to thank the Dev8D team, especially Mahendra Mahey, for their help in thinking through and setting up the challenge. I’d emphasise however, that any decisions we made (such as the nature of the prizes on offer) that may have contributed to people not entering, were entirely our own decision!

We’ve had an interesting discussion within the judging panel for the challenge (including folk involved in organising hackdays and other developer events) about where the community is currently at with developer challenges, so hopefully we’ve contributed to that as well. We’re grateful to them for their support throughout.

We’ll be releasing an interim report and a plan for the next 5 months of JLeRN work very soon. Lessons learned from running the challenge, as well as from the hackday, the CETIS Conference, and our other activities, will feed into these, so watch the blog for a post!

And there will be some other very good news coming soon too about some funded projects who will be working with JLeRN over the next 7-8 months, and some small bits of cash we may be offering for mini-tasks. Watch this space …

The Learning Registry and JLeRN at the CETIS Conference: Report and Reflections

In late February, the JLeRN team, along with Steven Cook who is working with us from the Jorum team, attended the annual CETIS Conference, and participated in the Learning Registry: Capturing Conversations About Learning Resources session. See this previous post for our presentation slides.

Our intention was that the CETIS Conference would be a significant point of reflection for the project: a chance to evaluate what JLeRN has achieved so far in light of community discussion and feedback, and a chance to get some pointers on how we should spend the next five months, since we finish at the end of July.

This week I’ll be meeting with Amber Thomas, our JISC Programme Manager, to write up an interim report for JISC, so I’m hoping this post will help me start to crystalise what I think personally, and what I’m hearing from the community. Skip right to the bottom of this post to see some bullet-pointed suggestions for further work from participants!

Firstly, can I just give a big shout out to Phil and Lorna from CETIS, and also to John who has now left CETIS but who helped a lot with organising the session. It was great!  Phil was particularly heroic in filling in for Dan Rehak by interpreting his slides for us. Also thanks to Walt Grata from the US Learning Registry team who came all the way to Nottingham just to hang out with all of us for a couple of days: it was absolutely invaluable having him there in person to advise, inform and have a laugh with.

Stepping Back: An Overview

As previously mentioned, Phil started us off with an overview of the Learning Registry. He presented some slides developed by Dan Rehak, which included diagrams peering into the Learning Registry model and concepts from a number of different angles. I’m getting the hang of it now, but I don’t doubt there were some baffled faces in the room at this point (I was at the front, and I didn’t look around). I keep reassuring people that you need to go through this stuff a few times using a few different resources to get a grip on it. However, speaking for myself, it was really useful to step back out of JLeRN and be reminded of how the Learning Registry is conceptualised by its creators.

The best thing for me about Dan/Phil’s presentation was being reminded of the Big Use Case. In the US, massive amounts of federal money is spent on educational resources which are then held in numerous silos, e.g. Nasa, PBS, NSDL, the Department of Defence may all hold different versions of the same resource. And therefore paradata, or usage data, about those resources may also be split into these different silos. Each portal only knows about a subset of what’s happening around their resources. How to pull it all together? The Learning Registry hopes to be a solution. This is highly relevant to Jorum here in the UK, which already shares resources with other services like Ireland’s National Digital Learning Repository.

We were in imminent danger of getting dragged into discussing definitions of paradata versus other kinds of data and metadata but Amber pulled us back from the brink, reminding us that we are not in the business of classifying types of data around resources: the nice thing about the Learning Registry is that it’s all in scope. Walt agreed, noting that “As long as you can express it as a string, and in JSON” it’s in scope. Walt also pointed out that even though there is a paradata specification available in the Learning Registry documentation, it’s not something that is required or validated; it’s just one suggestion to get folk going. “You can make up whatever you want”, he said.

Some Participants Chime In

This discussion led to Fred Riley from the SONET repository at Nottingham University, and Terry McAndrew from TechDIS asking some questions about how they could get involved. Fred would like to publish his repository’s OERs into the JLeRN node. There will be more on this in a later blog post, but the basic upshot is that this bespoke repository coded by Fred himself offers an RSS feed but not OAI-PMH, and until this point we’d been talking about publishing repository OAI-PMH feeds into our node. This was a timely discussion as Pat Lockley had just been working on a way to publish RSS into the Learning Registry, so it was a bit of edtech speed dating there.

Terry reiterated some of his ideas from the Hackday in January, following up with questions about how teachers can pull together stuff they have in a range of places (e.g. WordPress, Delicious, Posterous) to publish to the Learning Registry. All of this prompted some reminders about what an early stage proof-of-concept technical infrastructure project the Learning Registry is: some tools for developers and others are starting to emerge, but at this point just listening to these kinds of requests is all we can do. Walt noted that in the US there is work going on developing a few plug-ins and apps to work with various things teachers use to publish their content. Pat Lockley here in the UK also has some stuff on the go.

Terry also asked at a later point whether we can harvest from JISCmail as a source of data: folk can then see contributions that they’ve already made being used. Amber (from JISC) agreed strongly but noted that there are currently limitations- Brian Kelly has advised JISCmail to make their data harvestable.

Lorna on JISC / CETIS Interest in the Learning Registry

Why are JISC and CETIS interested in the Learning Registry? Folk are asking all the time whether they should get involved: is now the right time? Lorna noted again that this is an ongoing project; it’s being built right now. We don’t know how successful it will be. From CETIS’s point of view, it appears to be an interesting development that’s addressing problem we’ve had for ages.

Just for context, Lorna showed us a new OER timeline resource being developed by Lou McGill, which gives a graphical representation of the history of learning resources within JISC and CETIS. She noted that they used to recommend open standards for educational content, but that they were tricky to implement. Metadata for learning resources has been particularly problematic: describing educational context is very difficult. With the advent of the JISC OER Programmes, they decided to ask for descirptions of a few key aspects of resources, but not in standardised way.

The key questions that are still hard to answer are: How do we know where resources have gone, how they’ve been used, have they been remixed etc. So JISC and CETIS are interested in Learning Registry, because it acknowledges that we can’t normalise the mess of data. We need to get it into a network, and let clever people build services and tools to query the data. CETIS like that it’s dealing with the reality of the way the world is.

Amber on the Wider JISC Context

JISC supports the sharing of learning materials and digital library materials. Amber noted two other JISC Programmes that have looked at this stuff: the Activity Data and Discovery programmes. They are interested in open metadata; encouraging the flow of data. If you have data more openly available, makes resources more visible, this is a virtuous circle making discovery easier.

Andy Powell (who wasn’t present) Tweeted a question asking if we are talking about #bigdata at the Conference. This is about identifying patterns in data; it resonates with the Learning Registry model of the benefits of opening up data. The Managing Research Data programme has similar issues.

There is a question around how much effort are people willing to put into describing, rather than having that data inferred/extracted/derived. Amber name-checked Zoe Rose from the BBC on whether we should even try to describe educational resources, or at least the educational aspects of educational resources: Zoe thinks not, it just gets sketchy.

So, Amber reiterated again: JLeRN is not a pilot, it’s an experiment. Mimas run a node to be point of reference; JISC are not planning to necessarily turn it into a service. The questions starting to arise for her with her JISC hat on are: What are skills issues involved? What sorts of developers can start to learn these skills? It’s a capacity issue, and an appetite issue. Who are the people who want to make apps and widgets? What are their skillsets? JLeRN won’t be sustainable unless people get involved.

JLeRN’s Presentations

I refer you back to our slides and handout here. I like to think we’ve made a bit of a contribution to the resources helping folk understand what the Learning Registry is doing with these, so please do have a look.

Some Projects and Ideas

The next session was presentations on some work folk are doing or hoping to do around the Learning Registry and/or JLeRN. Hope to get some slides and links in here later.

Scott Wilson (Bolton University / JISC CETIS) on SPAWS: Sharing Paradata Across Widget Stores

- There are 3 educational widget stores sharing paradata (reviews, ratings, downloads) using the Learning Registry. The reason why 3 stores have been developed is because they are for different audiences, have different policies etc.
- The same widget in different stores would have paradata in different places. So they want to use the Learning Registry to bring together this paradata.
- Paradata includes downloads, ratings, embeds, reviews. It’s the same structure of information across different stores, so there are no ontology arguments. They are working on a shared architecture.
- Noted: there is a problem with how paradata is managed in Learning Registry. Their question is how to normalise paradata because the Learning Registry isn’t doing normalisation.
- They put in a bid to JISC OER Rapid Innovation for SPAWS.

Discussion:

Terry McAndrew mentioned that we need to identify good reviews of content rather than quick comments. There may be an opportunity for text mining in the middle, to bring out something more useful than Wordles. It worries him that there’s loads of info about resources: but it doesn’t include specifically meaningful stuff.

Scott responded with an example review: “I tried this in IE6 it doesn’t work”, noting that getting any comment is such a struggle, he’ll be happy with anything. If other organisations are interested in more in-depth stuff that’s fine, SPAWS just want enough to make it useful.

Owen Stephens noted that there are also stats about bias in reviews; that you could get many more bad reviews than good, but Scott felt that there is enough context around reviews for folk to interpret them. People don’t just say “oh 2 stars, it’s crap”.. you can leave some nuance to the reader.

Martin Hawksey said that as soon as he publishes anything, he gives himself 5 stars; so the system is open to gaming. Scott responded that if someone games it as positive and you use it and it’s rubbish, you are more likely to review it yourself; it balances itself out.

Phil Barker asked if there would be information on who is providing reviews; Scott noted that this will depend on the policies of the widget stores, whether they ask folk for permission to share their reviews; there may be some tracing of user activity. Some stores have to have control over user experience.

Amber noted that these issues of using paradata are fairly new and require some teasing out. It’s a different sort of architecture: it’s more distributed; it’s different from building a big website with the functionality inside it. We are separating it out; people don’t know how their reviews might be used. It’s still difficult to get our heads round it. People will need to re-calibrate how things are tweaked and used.

Terry McAndrew (TechDis): Learning Registry – Opportunities for Accessibility?

Terry shared some ideas at the Hackday in January about possibly using the Learning Registry to capture accessibility data about resources; perhaps simple reviews by people stating how well a given resource met their accessibility requirements as a start. This time, rather than showing a tool or app, or even a use case, Terry presented a “problem environment”: to enable participants understand what accessibility requirements might be.

- TechDis gives Inclusion Technology Advice across very broad range of education, including offender learning, so there are a broader range of issues and constraints than JISC projects even normally look at. The focus is on issues easily overlooked – accessibility.
- TechDis want conversations between developers, tutors and users about resources to be laterally available- live dialogue rather than end of year review, and available to all, to boost accessible development, and to expose issues around resource and practice early (including good and bad accessibility factors).
- Quality of data is extremely important.
- Terry asked whether we could get Learning Registry output through Google; noted the Xerte philosophy- doesn’t look glossy but gets lots of data through.
- Terry also noted that efforts around visualising OER use needs all the stakeholders’ invovlement. We need graphics that illustrate different types of users in different environments to see how the traffic is moving.

Discussion

Amber noted the validity of this view being wider: there’s accessibility, there’s usability, there’s mobile. Are there underlying characteristics that can go together? So not about boxing accessibility into one thing? Usability and mobile need similar things as a lot of accessibility.

Terry responded that this is where a controlled vocab would help, but Amber noted that questions of control and vocabulary happen before the data goes into the Learning Registry. The Learning Registry can then get the mass of developers around it to work on it.

Walt Grata from the Learning Registry: Harvesting Tool and Landing Pages

Walt talked about two things he has been working on for the Learning Registry to help developers: a harvesting tool and landing pages.

Harvesting Tool:

This tool can be used to harvest from any Learning Registry node that you want, and cache the data locally for analysis, mashing up, etc. Point the tool at a node’s HTTP endpoint, do a bit of config; it does a periodic harvest. You can get it all out in whatever format you want. He’s working on validation- and he noted that anyone who wants to try it out can raise an issue on GitHub and he will get to it. This work came about because he found he was writing bits of code for developers to use in Java, CSharp, JavaScript, etc., so he wanted to make all this available to just be plucked from GitHub.

Landing Pages:

Walt wanted to make available simple landing pages for resources, including backlinks for SEO, that are indexable by Google, so folk can find Learning Registry stuff through Google.

Discussion
There was some discussion around the relationship between the Learning Registry work and Google, touching on whether we are trying to create a parallel universe to Google, whether Google was interested in what the Learning Registry was doing, and to what extent the schema.org Learning Resource Metadata Initiative work might help (see Phil Barker at CETIS for more on that; Phil also noted that schema.org are coming out with other potentially useful vocabularies, e.g. to do with ratings). Lorna shared the interesting background that Google may have had a role in encouraging the Learning Registry project at the highest levels of the US administration: they were concerned about not being able to get to the swathes of educational content in silos.

There is also the question of how to identify unique resources in the Learning Registry given some resources will be available in different silos at different URLs. At the moment resources are only identifiable by having the same URL; Walt noted that they are currently looking at using RDF triples with “sameAs”.

Pat Lockley: Services Built Around the Learning Registry

See Pat’s slides here. Pat talked about two services he’s been working on:

- A Chrome extension for Google searches returning Learning Registry hits, that will display attributes about the resource in your search results. This could be what CC licence is applied to the resources, but as the Learning Registry develops, it could also include paradata, or data from the resource’s activity stream.

- A WordPress plugin that will let you display Learning Registry node content (accessed via “slice”, which is the Learning Registry search function) in your WordPress blog (I know, let’s try that in this blog! Watch this space).

Steven Cook from Jorum: Getting Paradata Out of Jorum and into the JLeRN Node

We’ve published all of Jorum’s metadata into the JLeRN node (that’s >15,000 resources), but we’ve also been looking at how to get paradata, initially just as some simple statistics and usage data, out of Jorum into the node. Steven Cook’s been exploring this as he is Jorum’s resident expert in extracting statistics.

Steven has been working on creating a CakePHP DataSource (“DataSources are the link between models and the source of data that models represent”) for the Learning Registry API, because CakePHP is being used to build the Jorum statistics Dashboard, and because CakePHP is designed for rapid development, is ideal for prototyping Web applications.

The Learning Registry DataSource abstracts away the connection requirements and the intricacies of the API and helps connect Cake developers directly to the Learning Registry, allowing them to simply “plug in” to a node and use the data obtained directly in their developments.

With the read capabilities of the DataSource he hopes to create a simple Web interface that will allow users to search the LR for resources. He also talked about “mashing up” this data with other sources across the internet including Topsy, Twitter, Names, Google, Klout. There are DataSources available for these other services and he has already written one for Topsy for the Jorum Dashboard.

Watch this space: Steven will be blogging on this soon here!

Discussion

When asked how scalable the CakePHP approach is, Steven responded that he probably won’t end up using Cake- but that it was a good way to gain understanding of how the Learning Registry works. He will probably port it to something else. He noted that the Learning Registry doesn’t give you exactly what you want and is not completely RESTful. In future he would cache on his side.

If You Could Have One Thing From or For the Learning Registry

We had a few minutes left at the end of the session, so I prevailed upon Lorna to ask everyone to say one thing they would like to see happen in the Learning Registry space. Here are the ideas shared, sans attribution! Contributors are welcome to expand on these, or add to them by commenting on this post!

  • Simple software libraries for a range of languages to interact with LR.
  • An OAI Explorer which allows you to poke at a node see what’s there.
  • The ability to get learning resources out by a simple search- e.g. of activity stream; what’s trending.
  • Would like to see my institution release Dynamic Learning Maps data for LR.
  • Developer how-tos.
  • How to deal with backdated paradata; website used for years, all that usage data- do I submit individual records for every single event? Don’t want to crush the node by submitting 50,000 uses.
  • Hide the hard stuff.
  • Can we get some good stuff from Google Scholar?
  • Accessibility dialogue.
  • How can we really make use of Google to get to LR content.

Finally, Lorna announced that JISC are running a DevEd event in Birmingham: a 2-day hackday focusing specifically on educational apps. Keep an eye out for further announcements, and put 29-30 May in your calendar!

Post Navigation

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: