The JLeRN Experiment

JISC's Learning Registry Node Experiment at Mimas

Archive for the tag “use cases”

Taster: some ideas for a use case on paradata and accessibility opportunities

Note from the JLeRN Team:

One of the tasks we’ve commissioned from our informal task group is some ideas around use cases, from JISC TechDis, on the potential of the Learning Registry and paradata for enhancing accessibility of OERs. Terry McAndrew, the author, will be presenting at the JLeRN Final Workshop on Monday 22nd October and completing it based on feedback and discussion there; but we’d be interested in any questions or points you have as well prior to the final version being published here. The taster below is Terry’s initial thoughts; he’ll be updating them between now and Monday.

So, here is your taster:

Paradata and Accessibility Opportunities – Draft

By Terry McAndrew, JISC TechDis.

The opportunities for the use of paradata for accessibility information regarding resources should not be missed. However, getting all the actors to produce and consume this information effectively clearly requires some thought for a realising implementation. Here are some of the issues we need to consider.

Firstly, the ‘actors’ in these scenarios need to be aware of what is important to capture and share, to enable all potential participants to make the best of the learning and teaching resources available. Generally all actors need to understand that multiple formats for information are necessary to communicate information effectively – it’s just that many omit to produce multiple outputs when delivering to an audience.

These actors can be thought of roles in the learning and teaching cycle and are the Creator, Tutor, Student, Publisher, Curator, Librarian, Accessibility Specialist, Manager, and Technical Infrastructure Manager. An actor can have multiple roles.

If you are thinking all these actors will have to co-operate to use paradata in the same way, this would be a mistake. You would be right in thinking though, that these should be aware of the others’ perspectives.

The actors and their roles

Looking at how the data is modelled in the JSON representation it may be that these actors can be aggregated under ‘educator’ or ‘student’ with additional attributes for the roles they are undertaking at the time of the interaction with the resource. For now, let’s use actors in the looser scenario modelling sense.

Creator: Produces original resource but may be unaware of the accessibility needs e.g. a diagram that lacks quality descriptive information for a visually impaired user in effect needs a ‘radio’ quality description; or a video which needs a transcript for users with hearing difficulties – the transcript provides more usability though; discovery, indexing and usually a better output if the speaker works from a script. The creator however mat wish to declare the purpose or intention of the resource.

Tutor: Uses resources to teach but encounters many different student needs and learning styles; if only they could feed these experiences back into the network to remind the creators of the issues, and how other students with various needs resolved problems. Tutors need to understand that disabilities are by nature often discrete – students do not have their disability “tattooed on their forehead” as many systems would like, just to make it easy to assign the appropraite data. Tutors therefore need to be informed of the nature of the disability.

student: Receives a resource in a context often defined by the tutor, but also as an independent browsing learner. The student need to be able to manipulate a resource independently if they have a disability, i.e. without tutor/demonstrator assistance to engage if possible (mobility impaired: the last thing they want is for someone to move the mouse for them). How, as the consumer, do they feed back this experience in the paradata? Is it gathered by the tutor or given directly – and do they have to declare the nature of the disability (glossory of learning issues) to validate it? Students with a support statement will usually be more active and highlight issues they could have done without. Students without support tend to be more passive, having lower expectations. Paradata could be liberating!

Publisher: Promotes resource into a community, probably using a generic description from interpretation of a complex one. If the paradata was tightly coupled to the resource such that users can easily access the usage information – including the accessibility issues – then the value of the resource may be more quickly assessed. The publisher may be the repository cataloguer.

Curator: Someone who collects interesting information to forward to others with similar interests. They tend to highlight the ‘back-story’ first, as often demonstrated on or, but the same resource will appear in different collections as it has different aspects to be highlighted. How useful would it be to be able to collect these views (why they qualify each other) and feed this into paradata?

LAMs: Library (and museum) staff are taking a greater interest in social metadata — content contributed by users — as it can assist with discovery and description to both augment and recontextualize the content and metadata created by LAMs. They need to be aware of the opportunity to collect accessibility issues and promote the benefits of online alternative where necessary.

Accessibility specialist: Translates information into advice and technologies. They would be ideally placed to bind or interpret common issues. May be occasionally working with staff developers.

Managers of professionals: Who need to recognise that engagement with paradata is part of professional scholarship. They allow time and resources to be made available to use paradata.

Infrastucture managers: Who enable the network and facilities to be utilised most effectively – if paradata recommends a resource in conjunction with a technology solution then that can make anticipatory provision based on this evidence.

So, how can we realise this interaction?

In oder to better understand the work going on with paradata I chose to inform myself, as many do, with the explanatory videos released by the LR team. Technical text needs context in order to understand it better adn get the impression of where it is heading, what elements are ‘concrete’ and others which are still under debate. From these I understood that ‘slice’ waas a practice which could enable the subselection of relevant information. I took this to mean that tools which only needed a smaller proportion of the data could be available to display or capture only the necessary information and therefore reduce confusion. In effect, not to be scared of the complexity that the system can cope with by any given actor role. For those of you without a nervous disposition…

On LR data services

I was struck by an example given for what was assumed to be a typical use – “state recommends a resource from the khan academy for teaching…” and became concerned that the state may not have considered the accessibility issues it has raised, and whether they would be one of the actor roles I envisaged above? Would they be ones to utilise accessibility paradata when making these recommendations? What tools would they need?

Not too long ago in the subject network (<10 years) we had many discussions about the metadata standards for cataloguing and interoperability – the dreaded ‘i’-word. Committment withered away eventually because all the actors who needed to use them would/could not significantly engage with them, and the supporting infrastructure to aggregate the services through each subject discipline needed consistent management buy-in. This needed to be better informed technically to appreciate the network solutions benefits. However, it was clear that significant work would have to be put into evangelising good practice with interoperable resource database to in effect, establish usage as a professional digital literacy. The community was shifting to less complex solutions – social networks and folksonomies.

Metadata itself maps to a procress already embedded in the business of cataloguing resources – it’s another way to capture and manage classification information so it is a small step to conceptualise this to operate the same processes in another landscape; what was done in libraries and understood by the lay person (who could physically observe books being catalogued and ordered), moved online. Those using online catalogues to search their library could visualise the translation even without technical knowledge, and retained confidence in the process. Those who are more aware of the ­­power of metadata could federate their searches through other agent software, and in effect be in many libraries at once. Paradata is a little more difficult to communicate as a concept; its the library of the conversation about all those books from the library, and more besides.

However, I don’t have to be a developer to get to grips with this – I just have to be able to understand some potential for development.

There is a problem when new technical solutions are facing a new attention challenge; it’s another practice to learn and the community that could benefit from them needs convincing there is a significant gain to be achieved. These potential users may have shifted attention to establishing their own online communities of interest through other free innovations which may be more satisfying to use e.g. curation tools and social bookmarking. They may be far from perfect but they can qualify information about resources and help one find allies with similar teaching problems. This shrinks the gap between the problem and the solution, making paradata capture and exchange less of a vital solution. Now that we have professional CPD from mailing lists to twitter, other innovative solutions need to have similar community benefits; finding similar others etc. Does the paradata networking promote this opportunity? If there are less efficient but far more effective alternatives then these will win out when the perspective of effectivness is from an individual’s perception, not an organisational one.

I followed the technical information to a something called Twitcurl – is this a potential method of harvesting the collection of tweets around a specific hashtag, or quickly poking information into a paradata network from ones own tweets? Could a hashtag and a shortened URI, coupled with account registration (to capture the twitter ID and map it to ones actor roles), be enough to record an interaction with a resource? This may be appealing to educators and students in the same way that using #fb posts status updates.

Another alternative may be a simple Widget to gather reflection by professionals  – something that could be available on all tutor’s and student’s devices to share interaction comments with a resource as it happens – but a context tag of accessibility info, would probably help. At this point a controlled vocabulary for disabilities may be required e.g. V.I., Blind, Deaf, Hearing Impaired, Print impaired, Mobility impaired etc. We are drifting back to metadata standards again. Oh dear!

It appears the accessibility content could be expressed as an ‘activity stream’ when sharing specific actions on what people did with an object, but the accessibility information is not simple transactional information – ‘teacher posted a transcript’ is not ‘teacher found it necessary to post a transcript to engage hearing impaired students’.

A resource could have accessibility paradata information utilised in parts of the JISC curriculum lifecycle – design, (re)develop, deliver, support, evaluate. Being able to collate evaluations from students and teaching staff could feed into the next design loop – being able to share a response to the evaluations would promote developments to other potential users (you asked for ……., we did ……..). Being able to filter this by disability may help refine the flow of information between actors and show how a resource is responding to feedback, but a culture change would have to occur to make this commonplace.

Our TechDis Accessibility Passport approach has been available for designers and developers to reflect on the features that may present issues so these can be addressed before a resource is released to the outside world. It works, but has not attracted large numbers of users as they are not finding a need to create a passport for their resources to be used elsewhere – when it is someone else’s risk.

‘One of the key challenges… is how to engage students, peers and tutors in creative and mutually beneficial dialogue characterised by innovative and reflective critical thinking – both in face-to-face, distance and work-based flexible learning contexts.’

Professor Peter Chatterton, Critical Friend to the JISC Transforming Curriculum Delivery through Technology programme

Should we look for accessibility information could be captured by proxies? Colleagues or attendees to presentations who could also see accessibility issues/opportunities – where do they pass comment? How could that be useful for paradata on the same topic? Where do you record your thoughts against this presentation? Does it have an event ID?

Video output online suffers from often suffers from poor quality comment streams – would reflections of its teaching potential be recorded through the nodes and then be harvested against the video by some other merged output.

Finally, I tried to think about how it might be expressed, just so I could visualise what tools might be available to create and use this paradata. Perhaps

“activity”: { “actor”: “email address”,”verb”: “recommends”, “disability”: “dyslexia”
“object”: http://resourceurl/resourceX/,
“content”: “individual recommends Xerte LO for dyslexic users to organise and plan”}

Some degree of identifer would help collate suitable resources and experiences with them. Does it have to be repeated for each type of disability? that seems wasteful and probably burdensome (repeated entries need repeated human input?)

Using appropriate attributes to capture roles, information could be coupled with type of disability and the purpose for the use of the resource. Here’s a modified example:


“actor”: {“role”: “teacher”,
“attributes”: [“elementary”, “math”],
“context”: [“site”, “NSDL”],
“action”: {“action-verb”: “viewed”, “count”: 2200, “context”: [“detail page”]},
“time-frame”: {“start-date”: “2011-05-01”, “end-date”: “2011-05-31”},
“object”: {“uri”: “xyz”, “attributes”: [“volcanoes”]}


The flexibility in the system exists, and it’s not my role to specify how this can be defined, but I see it can be. I believe it is worth attention to do so to capture an ‘accessibility context’ for any experience.

We have to think about how to engage all the actor “roles” with the process and visualise some scenarios. Perhaps pre-prepared widgets that may be available to all class participants as an experience review (quick feedback form) or class activity review. At primary school it could be part of the leveling processes that are recorded against curriculum performance, perhaps a tool for special needs support worker. In Higher Education it may work best as a reflective tool used by the student to reveal the issues each resource creates. The teaching staff (tutor and demonsrators) may add other observations against the resource but would need to be aware of the disabilities in the class, and how they manifest themselves – input into the accessibility comment may benefit from some professional qualification.

I am less of a fan of the ‘build it and they will come’ approach than I used to be: there are too many competitors for attention. The value of paradata needs some appealing tools for all sectors to slice this information they wish to use, in the way that suits them best. However, each of these interface tools should recognise the value of capturing accessibility information at the same time for the benefit of every potential user.


The Hackday: Report and Reflections

The JLERN Experiment and JISC CETIS co-organised a Learning Registry Hackday on January 23rd 2012, hosted by Mimas at Manchester University. Lorna Campbell from CETIS has already written a great blog post (which now includes some excellent discussion below the line) summarising some of the issues that were discussed.

I wanted to report from a JLeRN point-of-view what happened at the Hackday, and what we got out of this event. Our aim was to get the UK folk who were already doing stuff around the Learning Registry in a room with those who were starting to think about doing stuff, to talk about use cases. And we wanted to get some technical people in a room with the JLeRN developers to play around with publishing and accessing data through the JLeRN node.

Before JLeRN started CETIS were already on the case in forming a community of interested folk in UK HE. And Pat Lockley, when he was still at Nottingham University (he’s now at Oxford University), attended the first ever Learning Registry Plugfest in the US (see his report here). So we had a room full of early adopters and experts in the Learning Registry, alongside some new folk with great ideas.

Plan for the Day

We started the day with a blissfully brief presentation session, just an hour long before lunch. Lorna Campbell gave us an overview of the Learning Registry and JISC’s involvement with it. I talked everyone through what JLeRN is doing, with the help of this handout, which summarises some of the options from the Learning Registry specification: Communities, Networks, Nodes.  John Robertson of CETIS finished with an introduction to paradata.

After lunch everyone introduced themselves and what their interest in the Learning Registry was. This was where some interesting ideas and use cases started to emerge.

One Outcome: Concrete Proposals to JISC OER RI Funding Call

It also became apparent that several participants were intending to submit project proposals involving JLeRN and/or The Learning Registry to the JISC OER Rapid Innovation funding call. As it turned out, four proposals (that we know of) that were discussed here were submitted; we won’t know whether any of them were funded until March, but it was great to get so much concrete interest out of the day.

Who Came and What They Said

CETIS’s Scott Wilson was there; he was interested in the Learning Registry from the point of view of a project he’s involved in, which is developing “a Widget Store aimed at the UK education sector using a codebase shared across and sustained by a range of other EU projects and consortia”. Scott wanted to see if the Learning Registry could offer solutions for sharing paradata about educational widgets.

With our proximity to the national learning resource repository Jorum, also based at Mimas, we were delighted to see Ben Ryan, the new Jorum Technical Manager, and Steven Cook, one of Jorum’s developers. Jorum has a great interest in usage data and other forms of paradata, and had been promising to share data with JLeRN from the start. This event turned out to be an excellent opportunity for their new technical manager to get up to speed. Plus, Steven Cook, who is working on a framework and dashboard for Jorum usage stats, was inspired to start developing ways to get that kind of data out of Jorum, together with resource metadata, and into a Learning Registry node. Watch this space: Steven will be blogging here!

Tatiana Novoselova and Andrew Green came from the University of Liverpool, bringing with them lots of experience and ideas around sharing data about engineering learning resources. They had already been trying to grapple with the Learning Registry, and found the Hackday helpful in ironing out some technical issues they had been having, partly thanks to Pat Lockley’s presence and support, and partly thanks to the JLeRN technical team. They reported here.

Terry McAndrew came from JISC TechDis, with some great ideas about how the Learning Registry might allow the educational content community to finally start getting some traction on how to usefully share information about accessibility of resources. It’s always been very difficult to get metadata standards to accommodate accessibility data, which is often, basically, about a user’s experience of a resource, or about the features of a resource as they relate to a user’s individual requirements. Lorna’s post summarises this discussion.

Suzanne Hardy and John Peterson came from Newcastle University, full of ideas from the more complex end of the paradata spectrum: their Dynamic Learning Maps as educational context of use / curriculum data around resources. Watch this space: I’d like to get some more out of them about their ideas!

Julian Tenney came from the University of Nottingham with some great ideas about gathering good paradata from users via the Xerte content authoring tools. He felt it important to start to think about ways that content authors and educators will actually be able to contribute contextual data as part of their usual workflow. Another space to watch.

Richard Goddard, formerly of MrCute (Learning Objectivity) and Alastair Hole came from Worcester College of Technology with an idea about developing a tool that will help record actual educational usage of resources (as distinct from merely hits/downloads). Once this data is recorded, wouldn’t it be great to share it via the Learning Registry?

Jenny Gray from the Open University has had an interest in the Learning Registry since before JLeRN, from the point of view of the OU’s LearningSpace, came along and had some fruitful chats with Nick and Bharti, and Pat Lockley, about options for publishing LearningSpace data and paradata in the node. Like Jorum, she’s interested in how to get an OAI-PMH feed into a Learning Registry node. She reported back on her own blog here and followed up with some more thoughts, and questions, on sharing LearningSpace paradata here (to share or not to share?).

Matters Arising

Note: these action points and issues are not a to-do list for JLeRN, or even for the Learning Registry! They are just a note for when future plans are discussed, and for JISC and other stakeholders to take note of.

  • We need to more fully develop some concrete use cases. E.g. a simple Accessibility “Like” button that could start to capture accessibility info of use to users.
  • We need to develop some demonstrators. E.g. a desktop widget or browser plugin that captures info on user, resource, content, and context of use, with a simple interface.
  • We need to think about how we interact with a node to share and call complex paradata (e.g. Dynamic Learning Maps type stuff).
  • We need to start defining the structure of paradata for various use cases. E.g. we need to look at the concept of ‘actor’ as currently specified for paradata.
  • Note that running a node is a cost, and the JLeRN node will only run for the life of the project (until July 31 2012); need to think about business model and business case for node services; folk can build their own tools and services on top of nodes. People will not feel motivated to start building on something that is still experimental and has an uncertain future.
  • Might be an idea for JISC to take a similar approach to developing SWORD: a series of simple tools to put and get data into JLeRN.
  • Running a node on Amazon Web services (i.e. on a cloud service) has unpredictable costs; this is not generally liked by UK university finance departments!

Data sources for JLeRN

A final note to ourselves: some excellent suggestions from the floor, and thanks to Lorna for taking these notes!

  • Jorum
  • Xpert
  • OU
  • NHS eLearning Repository
  • DLM NBBS programme data
  • FE Data sources like NLN? XTLearn
  • Open Spires podcasts
  • Delicious / Diigo
  • Merlot
  • NDLR

Next Steps

JLeRN are about to attend the Learning Registry session at the CETIS Conference 2012, like, tomorrow. And we are still calling for entries to our Dev8D JLeRN Paradata Challenge (due Friday midday).

After that, we have five months to go until the end of JLeRN. What should we be focusing on? Discussions are ongoing, we’ll report back here, let us know your thoughts!

Post Navigation

%d bloggers like this: