elearning

New beta version of Pick&Mix benchmarking system released

by Paul Bacsich

There is a new release of Pick&Mix available. This is the beta 3 release of the new version 2.5 under development.

The release incorporates early input from the EU Re.ViCa project on Critical Success Factors and also the "headline" feedback from the Gwella phase of benchmarking with institutions in Wales in 2008-09.

We are particularly keen to hear from institutions currently undertaking benchmarking in case there are still gaps in the Supplementary Criteria.

The next beta aims to incorporate the outputs of several new concordance studies including on OBHE and several EU projects.

The release is available  here -  Download PnM-2pt5-beta3 - or in the Matic Media benchmarking repository at http://www.matic-media.co.uk/benchmarking/PnM-2pt5-beta3.xls.

Note that a copy of the latest beta is always available at http://www.matic-media.co.uk/benchmarking/PnM-latest-beta.xls

Happy Benchmarking!

I may be some time, I didn't say

Little did I think when I stepped out of my blog in September 2006 that "I may be some time", to use the inimitable phrase of Captain Oates. By the standards of blogger's rest cures, it has been on the long side. But in addition to the usual excuses of overwork, life, the universe and everything, I had the additional one that I was doing a lot of blogging - and fostering of blogging - just not on my own blog.

I also fell in love in the fallow period (personal-blog-wise) with my second web 2.0 technology, the wiki - and that affair, like many web 2.0 affairs I hear of, is only now cooling - or at least settling into a less passionate relationship. I still like wikis but I am somewhat more critical than in my wiki-youth.

So the period of being "somebody else's blogger" is over and wikis to me are now a tool - not a way of life.

Also given the range of things I do, more and more of which do not fit neatly into the timesheets of projects, there is greater re need for a personal channel.

And last, I am part of a team of midwives bringing to birth a very large and lusty wiki baby in the next  couple of months. Like all babies, it will need nurturing and fostering - and blogs are good for that.

So let's see if my resolve holds.

A particular challenge coming up is to see if I can blog while at a coference - and yet enjoy the socialising. My first test is at the Learning and Technology World Forum in January. To prepare I've even done some entries on the blog they provide for eacme as a delegate.

Maybe I'll see - or "see" - you there. More at http://www.latwf.org.

Paul

Technorati

Technorati Profile

Benchmarking e-learning bonanza

Further to the earlier posting on BENVIC, there is now a revised analysis of BENVIC. Some insights from BENVIC were incorporated into the new release of Pick & Mix (2.0) available in beta. 

In related work, a number of other reports on benchmarking are available, thanks to my work for the Higher Education Academy on the Concordance Project. These include:

1. MIT90s, including a literature search
2. Pick&Mix new material including a project report, the version 2.0 beta and a broad version of Pick&Mix 2.0 for learning and teaching (not only e-learning).

The work on ELTI is being urgently corrrelated with other input from pilot sites in order for colleagues to define the next release of ELTI.

The work on adapting eMM to the UK is now not so urgent since eMM is not being used in the Phase 1 round of benchmarking work in the UK (although it continues to be used at the University of Manchester).

Paul Bacsich

Coping in a world with software patents

Summary

This posting is designed to suggest that individuals and professional associations in the e-learning world consider seriously the steps they might take to cope more adequately in a world where software patents are increasingly more prevalent. There is a slight UK focus but it will apply all the more so in the US.

I wrote a version of this paper initially for the UK Association for Learning Technology (ALT) but was encouraged by reactions to it from people including Seb Schmoller to make it more widely available. His posting Should organisations now put public knowledge and knowhow on Wikipedia instead of publishing it themselves? Views requested is relevant.

The paper has been “inspired” (if that is the word) by the recent flurry of activity caused by the assertion by Blackboard of a software patent which appears to many to patent some key non-proprietary aspects of enterprise-scale e-learning, and clear indications that other large companies are following suit, not only in the US but increasingly in the EU and in Asia/Australasia. It is “informed” (or so I hope) by a number of recent pieces of historiographical work I have done (some paid, some unpaid) including:

• literature search on benchmarking e-learning
• history of the UK e-University
• historical literature search on the continuing relevance to e-learning of the MIT90s strategic framework developed in 1991
• labours in the Wikipedia vineyard on the history of virtual learning environments.

All these tasks were made much more complicated by the fragmentary nature of the information from fomer eras and the speed with which even recent vital information decayed from the web.

Disclaimers

1. The posting does not take a view on whether software patents are desirable, only that it is likely that they will be more widespread (and go more and more beyond the US) and more used. It argues that if the steps below are taken, then it will be much easier for defenders to find “prior art” when patents (or prospective patents) are being challenged, but also much easier for patent searches to be carried out by companies and their patent agents. Thus it aims to be patent-neutral when patents are researched and granted “properly” (but it is not “patent-troll-neutral”).
   Out of scope for this posting is the issue of whether professional associations should decide to have a policy regarding software patents (for - or against - or a more complex view).
2. I am not a lawyer and nothing in this posting should be construed as giving legal advice. Those seeking such advice should consult suitably qualified and accredited individuals.

Steps that might be considered

1. By individuals

This guidance is particularly oriented to individuals who have jobs involving research or other activities likely to generate specifications, products, systems etc of relevance to patent searches and patent claims in the general area of e-learning.

Much more than in the past, individuals move between jobs, including between academia and industry. Increasingly, individuals have portfolio careers and may have formally retired from their main employer yet still be active in research and consultancy.

1. Individuals should try to ensure that former employers are aware of their current contact details and as far as possible ensure (if they wish) that emails to a former email address are automatically bounced back to the sender with their current email address. (This is trivial to do using “out of the office” functions, but is remarkably rare. It is unlikely that more than a small minority of staff in any institution will wish this.)
2. Individuals should consider having a lifetime email address, which need not be public, but to which key emails and documents are sent – thus allowing a long-term archive. This is increasingly done by the “net generation”, but is still regarded as vanity or impertinence in many organisations. Selection of an email host and client software is crucial in order to have long-term guarantees of archiving.
3. Individuals who regularly develop or intend to develop “inventions” (taken in a wide sense) should ensure that they are more aware of the Intellectual Property aspects of these (copyright, moral rights, patents, etc) including securing an audit trail for the documents if they are ever needed again.
4. Individuals who develop material likely to be of interest in the patent world should take advice on storage formats suitable for long-term archiving. In many cases it will be prudent to create a version of the material with lower presentation quality but longer longevity. (This has been done with Internet specs for many years, creating their archival versions as text files.)
5. Individuals who regularly offer advice to associations of which they are members (either directly or via an employer) should ensure that they are more aware of the Intellectual Property aspects of any contributions they might make to association documents, and seek to assert them if appropriate.
6. In general, as a final catch-all, individuals are likely to have to empower themselves more and rely on organisations less, when considering long-term storage of material. It is accepted that this has cost and storage (including physical storage) implications for individuals’ homes. It is outside the scope of this paper as to how individuals justify such implications to themselves and those that they share their homes with [she made me write this].

2. By institutions

This section is phrased in terms of institutions from the university and college sectors, but could well apply more widely. There is some orientation to the UK but I expect that many aspects apply to Canada, Australia, New Zealand and the US.

Institutions are by and large getting worse not better at maintaining information. Changes of staff, reorganisations, public relations reasons, a desire to break with the past (e.g. caused by a change of name or role) and content management systems (when crudely implemented) all conspire to ensure that material is lost or its location is moved. The following are the suggestions for institutions:

1. Institutions should commit to maintain the email address of a former employee, on request (not to be unreasonably refused, and certainly by default for “faculty” (academic staff) who request it) for a period of years (10 is suggested) from the date of the employee leaving, and renewable on request. There are no storage implications, in that it is sufficient that email is rejected with a message such as “no longer at this email – believed to be at x@y.z”. There is public relations value that an organisation can gain from this, in terms of supporting collaborative research, etc.
2. Institutions should choose their email software for server and client with longevity in mind. There should be no deletion of messages and attachments based purely and automatically on time elapsed. (It is pathetic to watch the efforts that people go to in moving attachments from mail storage to file storage to evade quotas.)
3. Institutions should ensure that they are more aware of not only the Intellectual Property aspects of key documents (copyright, moral rights, patents, etc) but also the archival aspects, including securing a legally valid audit trail for the documents if they are ever needed again as “certified originals”.
4. Institutions should take advice (e.g. from JISC if they are universities or colleges in the UK) on storage formats suitable for long-term archiving. In many cases it will be prudent to recommend authors to create a version of the material with lower presentation quality but higher longevity. (IETF have been doing this for what seems forever.)
5. Institutions should have an IPR policy covering staff who regularly offer advice to associations of which they are members (via their employer) – such associations to include learned societies and software user groups (including for LMS vendors).
6. Institutions should consider having a time-out period after which rights in all e-learning material revert to the author(s). This time-out period may have to depend on the topic area, but for e-learning software and e-learning material a period of 10 years is suggested.

3. By Professional Associations in e-learning

This includes such bodies for individuals and universities as

• ALT and UCISA in the UK
• EDUCAUSE in the US
• ASCILITE, ODLAA and ACODE in Australia
• (fill in your own favourites)

If the above principles are accepted then in my view it is important that professional associations are seen as leaders in and earlier adopters of these principles. Thus:

Professional associations should put into practice the principles enunciated above

In particular:

1. As a gesture towards coherence of the community of practice and a service to Trustees / pro bono Directors (past and present), professional associations should extend to all such office-holders, on request, the facility of having a brief web page about them linked to their current details, in perpetuity. This again has public relations advantages for the associations, for example in showing how many previous “eminences” have advised them.
2. Professional associations should consider setting up their own history/archival SIG for e-learning or collaborating in a more general national history/archival SIG for e-learning (with a federated approach across countries). There is public relations potential for this and potential funding in some countries – European readers should note in particular the announcement by the Kaleidoscope EU project (which has several UK members) on Meeting Challenges of Creating an Open Research Archive in the Field of Technology Enhanced Learning.

4. By national bodies

This section covers suggestions as to what “national bodies” should do. By this I mean such bodies as JISC in the UK, SURF in Netherlands, perhaps EDUCAUSE in the US. In some countries where there are no national bodies of that sort, some of the “peak bodies” (to use an Australian phrase) might have to suffice.

1. A national body should maintain a formal database of all current and former researchers within (or who were within) its jurisdiction, including their current contact details (if alive, if known and if the researchers wish them to be publicly known). Possibly the practice could build on what the Research Councils (ESRC etc) do already in the UK – or what has been tried with limited success by the EU with Europa. It will be the onus of researchers to ensure that relevant national bodies know of changes in their contact details.
2. A national body should have a simple archival policy – no document should “ever” be deleted or the underlying file (logically) moved - even if (in rare cases) its URL has to be changed. It is assumed that storage costs will fall faster than the storage requirements for new material grows, so that old (thus compact) material is always a small fraction of current. By “ever” one is likely to mean at least the maximum period that IPR can remain extant for.
3. Without necessarily taking on the full panoply of “Open Archives”, national bodies should ensure that document names and a key subset of metadata (authors, dates) are adequate before loading any document. Searches reveal that many documents in many archives are called “Untitled” by “Unknown”.
4. For legal and other reasons, a small physical archive of paper copies of key documents may be needed. National bodies in key countries (UK, US, etc) should confer discuss with others the issues and benefits around the setting up of a physical archive of key e-learning documents in these countries.

Well, that's it! Comments welcome.

Paul Bacsich

BENVIC benchmarking in context

BENVIC was developed under an EU project, also called BENVIC (in full, Benchmarking of Virtual Campuses) in the era 1999-2001 There is a project web site still at http://www.benvic.odl.org/ – but it has not been updated since 4 February 2002. The consortium was led by the Open University of Catalonia (UOC) and had a strong set of partners (including UCL in the UK). However, for various reasons including retirement of key staff the work does not seem to have continued – anyway, follow-up work is not evident.

It has proved to be quite easy but also informative to correlate the BENVIC approach with the Pick & Mix system. This demonstrates how the value  of BENVIC can be preserved within the modern context of benchmarking e-learning. Indeed, it is planned that some insights from BENVIC will be incorporated into the next release of Pick & Mix, available August 2006. The full report gives more details of the correlation, its conclusions and implications for the development of Pick & Mix.

Paul

Cautious bloggers manifesto

Just occasionally I will reflect on what I am doing in this blog and why. Here are some founding principles of the cautious blogger.

(Some of you may have seen some thoughts like this before, on my "Ebenezer" pilot blog.)

• For me, blogging is not about changing the world. It is about helping my business and my research - and these come first.
• If blogging reduces the effort footprint for my web site, that would be particularly beneficial.
• There will not be a rigid schedule for blog postings - life is too complex for that.
• I will post rather less frequently than many bloggers - weekly at best
• I will try to make up for any infrequency of postings by increased depth of analysis.
• The needs of my (paying) clients come first. They will often get information faster and with more specific comments than general readers of the blog.
• I want to put information out (over the years I have edited several newsletters) but I want to get information back.
• I look to blogging to save me time (on an overall averaged basis) not to cost me time.
• I will not forgive slow, hard-to-use or buggy software just because it is open source.
• If the blog tools I use do not soon fit seamlessly into the Office paradigm, then I will change tools.
• There will be no sponsored postings on this blog.
• If you ever meet me at a conference and I say that I am too busy to talk because I have a blog posting to do, shoot me! (Or better, buy me a large Jameson [not a sponsored posting].)

[Disclaimer: all postings in any meta- category are the views of Paul Bacsich at the time, place and mental condition that they were posted in and do not necessarily reflect the views of Matic Media Ltd or any other employer of mine or any client of Matic Media Ltd or any other blog that I am contracted to post on.]

Paul

Secret histories 2: USOU

Historical analyses of e-universities seems to be just like buses down Oxford St [cultural reference: London, England] – you wait ages for one, then two come nearly at once. Shortly after the news came in about the analytic report on BCOU and Tele-U in Canada, along comes this EDUCAUSE article about the rather longer-dead USOU – the US branch of the UK Open University, which flourished from spring 1999 until summer 2002.

The article is written by Katrina Meyer. Some in the global audience may not recognise her but in the US she is a regular author and speaker on e-learning topics (including those of interest to us) and is heavily involved in WCET including having presented at most of the recent WCET conferences. (I went to several WCET conferences in the 1999-2002 era, which were excellent.)

As her methodological basis, Katrina decided to interview the first (and only) Chancellor of the USOU, Richard Jarvis. He has had a long and distinguished career in the higher reaches of university management – as his CV shows – and seems therefore to have the confidence to talk about one of the presumably less satisfactory episodes of his career.

Based on her interview with him, Katrina makes five key points about reasons for failure (which can as usual be inverted into critical success factors when applied to other institutions):

1. Loss of key advocacy and support from the base organisation (i.e. the UK OU)
2. Curriculum conflicts
3. Challenges of the US marketplace
4. Lack of accreditation
5. Business planning

How do these stand up? Declaring my interest, I worked at the UK Open University for many years until 1996. Thus I was not at the OU in the critical period for her study, nor do I have any inside information from that era. However, I feel I had a pretty good understanding of how the OU really worked, and remained in contact with the OU on various projects up to around 2004, including on other projects in the e-learning space, the UK e-University in particular. From the US standpoint I was involved with various US agencies including TLT Group and the Steering Committee of WCET during some of the critical period. So I believe I have something to add.

In terms of her five points, she personalises point 1 to some extent, noting that John Daniel left the OU and this led to a loss of top-level support. I am sure that this is true, but even before he left the OU, my observation from the US standpoint was that the OU were not necessarily putting their best people into supporting the USOU project. (Note that by summer 2001, in addition to any economic downturn issues referred to, the OU had a major e-learning distraction much nearer home base, the e-University – more of this later.) There were also some odd technical decisions that the USOU made, such as to not use the main e-learning system deployed at the OU (FirstClass, as then used in many UK, and many US providers) but rather to use Prometheus (which was shortly after taken over by Blackboard and discontinued – not much vision there).

In terms of points 2, 3 and 4, I feel that she is in general accurate. However, I think the point about the OU curriculum is perhaps overdone. Even when I left the OU in 1996, it had spent many years on internationalising its curriculum and had many areas (mathematics, science, computer science) where the differences between English-speaking countries would be in any case rather less.

However, there is a valid issue over the MBA, as Jarvis notes. Like most UK MBAs, the OU MBA was “European” in approach, rather different from the US approach to MBAs. Thus it would have been considerable work to adapt it for a US audience – and would they have accepted it without a US university partner being involved? In that era, I am sure that a US audience would not have accepted the existing OU MBA in any large numbers.

But on the issue of accreditation, I am surprised that the USOU did not try to finesse this issue in the way that several US providers (Phoenix etc) have done, by providing a range of non-accredited programmes; or as many UK distance learning providers have done by providing a range of MSc programmes (not MBA) where the (local) accreditation need is less.

On business planning, the OU is a cautious animal and the paradigm has not really changed since its founding. Indeed, it might be argued that the OU-Manchester alliance and the OU’s drive to open content are the first two major business model changes in the OU’s existence.

Likewise her point about student services is rooted in the issue that the OU has taken a (to some) rather leisurely approach to fully online learning – on the other hand, it is still in business whereas several of its “pure-play” rivals are not, and the pendulum (at least in the short term) has now swung back to rather more face to face than purists would like.

Inevitably when only one person is interviewed, however senior, I was left with a feeling that we had not heard a rounded story. It would be good to have the views of former colleagues, opinion-formers and also of key OU staff involved. In this context the study of Darby et al is a useful guide. This might be coupled with a documentary analysis of material referencing the USOU, such as speeches from John Daniel.

So what other factors might a follow-up study look into? From my own work on failed e-universities, I suggest the following:

• the role of the CEO in e-learning start-ups
• the views of US regulatory agencies and other opinion-forming bodies (such as WCET) on the USOU – and any sense of whether it was viewed as an “intrusion” (one has regularly seen such defensive reactions in the US in the last few years against various kinds of non-US commercial incursion)
• the reasons for and against the specific location of the head office (location of HQ is a lesser-known but crucial factor in the success of distance teaching organisations)
• what market research was done and (equally importantly) what was acted on
• the impact on USOU of the fact that the OU had a local e-learning competitive situation to deal with at home base, namely the UK e-University, announced in spring 2000, with many studies, negotiations and contracts in the next two years, several of which involved the OU (someone else can draw the necessary historical parallel with the Roman Empire – is this the place to admit that I am reading Romanitas?)

But all in all, I commend Katrina Meyer for “breaking the silence” on this so-far neglected institution as a case study of virtual university failure, and look forward to more on the topic.

On a subtler note, keen students of Richard Jarvis’s CV will note that his first degree was in geography. Insiders to the UK e-learning scene will note the signifance.

Paul

Secret histories of e-universities: the Canadian scene

The small but (we feel) select community of e-university analysts round the world were pleased to see recently that the long-expected study of Canadian e-university "changes" has appeared. This is a twin study of two interesting situations: the absorption of the British Columbia Open University (BCOU) component of the former Open Learning Agency by the new Thompson Rivers University (TRU) and the merger of tbe Tele-Universite de Quebec into the Université du Québec à Montréal (UQAM). The study was commissioned by the Commonwealth of Learning (COL) from Dominique Abrioux, who is President Emeritus of Athabasca University and and Professor at their Centre for Distance Education.

It is believed that COL is interested in supporting such case studies since many of the smaller countries of the Commonwealth cannot afford to make mistakes in their development of e-learning, and may be able to learn some lessons from the difficulties that several larger countries have had with their e-universities (Canada and the UK in particular).

The report is well summarised, so I will leave readers to read the summary and then form their own conclusions on the main report [PDF, 0.4 MB]. While there have been a number of postings and lots of gossip about the BCOU situation, that surrounding the Tele-Universite de Quebec is much less known, to the  English-speaking community at least.

Readers should not go away with the conclusion that this paper gives the last word on Canadian e-university "situations". There are still many aspects of the closure of TechBC and its absorption into the Surrey Campus of Simon Fraser University that could merit study - little material has appeared in public except an impassioned analysis from a former student.

Paul

Prehistoric benchmarking e-learning

Having at long last decided to organise a web site with details of my company Matic Media Ltd and lots of stuff on my main publications and presentations, I have spent some days going back in time through outputs from my work, and got to the mid 1990s just as I left the OU for Sheffield Hallam U.

I came across a paper (still online) on virtual universities which I remembered - but what I had not remembered that it has a benchmarking aspect. In the section on so-called Dimensions of Virtuality, I put together a table of criteria for a set of virtual universities, and scored them on a 1 to 10 scale. (Indeed, I dimly remember even runnning a participative workshop or two on this - most likely at Online Educa - with members of the audience shouting out suggested scores.)

So I shall have to rewrite my standard introduction to my benchmarking work, which goes something like this (as in the recent Pick & Mix update to 1.2).

The e-learning benchmarking methodology called Pick & Mix was developed in early 2005 and first used in a competitor benchmarking exercise for Manchester Business School. Since then it has been further developed in seminars and workshops in UK, Colombia, Germany and Australia. Though only named in 2005, the methodology draws on many years of Paul’s experience in benchmarking, competitor analysis, cost analysis, and change management, in particular on the benchmarking study on Ufi LearnDirect and on the pre-procurement report on “e-tools” done in 2000 for HEFCE (see http://www.heacademy.ac.uk/learningandteaching/eUniCompendium_chap16.DOC), which used 12 generalised criteria rather than “feature wars” scoring.

In other words, benchmarking started in around 2000.

But it didn't - not even for me.

So am I cleverer than I think  - or more forgetful?

To get a 10-year or so perspective on this, have a look at the old paper and then the very recent paper for ODLAA on critical success factors for e-universities (i.e. online virtual universities) with the UKeU as a specific case study.

Paul