Quality , enhancement and on-line distance education courses and programmes

As quality assurance systems become embedded, competitive institutions are seeing their futures in quality enhancement, in a continued commitment to improvement. This paper discusses quality signals that may be used to enhance course quality in on-line distance education. Analysis proceeds under five headings: pedagogies; promoting complex achievements; changing concepts of quality; designs for re-use; and new partnerships. Recurrent themes are: A rich and differentiated view of learning leads to a rich and differentiated view of quality; Quality courses are associated with the quality of affordances for learning; Conceptions of quality have been changing; e-learning is creating new conceptions of quality; Quality in higher education practice in a decade’s time is likely to be different again. Universities that cling to established views will be at risk from their global commercial competitors.


Quality and on-line learning
Is on-line learning better than face-to-face?Following Russell (1999), we might expect the answer to be that there is no significant difference and there is some support for this in the literature.Other studies, though, show some advantage for computer instruction on some sorts of task -typically recall, practice, understanding and near-transfer tasks -.In The College Effect, Pascarella and Terenzini (2005) find that '… compared with similar students taught by traditional instructional methods, the knowledge acquisition of students in computer-based courses is significantly better … or not significantly different ' (p. 98).Although theirs is a large book, this comment seems to be based on just 25 reports of computer-assisted instruction.
A third response recalls that, once upon a time, when school curricula were being reformed, Walker and Schaffarzick (1974) reviewed the literature to answer the question, 'Are new curricula better than the old curricula?'Their found that new curricula produce a better grasp of the things that new curricula emphasise and old curricula produce a better grasp of the things that old curricula emphasise.Not only did 'new math' include new content (tessellations, set theory, number bases), it carried with it new notions of quality in terms of design, teaching, learning and assessment.
So too, I argue, with on-line learning.Notions of course quality that applied fifteen years ago have not been replaced but, at the very least, they have been complemented.In the UK Open University, course (I'm using 'course' as a synonym for 'module' or 'unit') quality was largely defined by the quality of the print and broadcast materials -by the quality of our product.Service and support quality also mattered but product quality dominated.I want to insist that those notions of quality have not disappeared but I want to argue that new concepts of course quality are emerging and are embedded in good quality courses.I develop this claim through comments on five themes.

Pedagogies
Research based largely on students' course evaluations produces consistent findings about their views of teaching quality and of effective teacher behaviours.Administrators often encourage faculty to improve teaching by designing courses with these findings in mind, although there is little evidence that student preferences are associated with superior learning (Knight, 2002/5).Other research literature makes strong connections between student engagement, environments and learning.It gives good pointers to high-quality course design.Pascarella and Terenzini's (2005) review of the (mainly US) literature reports moderate effect sizes for learning for mastery, supplemental instruction, active learning, collaborative learning, cooperative learning and small-group learning.They add that student engagement is also vital: …a student's coursework and classroom experiences shape both the nature and extent of his or her acquisition of subject matter knowledge and academic skills [but] … what the student does to exploit the academic opportunities provided by the institution may have an equal, if not greater, influence … other things being equal, the more the student is psychologically engaged in activities and tasks that reinforce the formal academic experience, the more he or she will learn (p.119; see also Pascarella and Terenzini, 1991;Astin, 1997).
Similarly, there is a lot of agreement about 'teaching' quality in networked learning.For example, de Laat (2005) reports that In general, research in this theme suggest that teacher involvement and active participation is appreciated by students … [who] find communication with the teacher constructive and encouraging, especially where teachers support the students to set the right tone for the discussion (p.155) He adds that success involves teachers and students understanding a 'new' way of learning, which involves developing 'inter-metacognitive' knowledge and skills so that they can function as a networked learning community.We can take these findings as clear quality signals.
Interestingly, he goes on to say that the teacher's role will be different in networked learning groups depending on the course aim.If the aim is to have students learn through participation … the teacher will play the role of full participant in her domain.Finally, in this section, I argue that in this century it is necessary to have a view of course quality that helps students to remember, understand, apply, analyze, evaluate, create using modern technologies and in collaboration with people at a distance, if only because this is the world of professional work to which many of our graduates aspire.For many universities this view implies that they should be embracing the technologies with which we here are familiar in their face-toface operations.

Promoting complex achievements
In the last section I commented on pedagogic quality in relation to six cognitive processes: remember, understand, apply, analyze, evaluate, create.Higher education is, of course, concerned with much more-with fostering understanding, skilful practices, efficacy beliefs and metacognition (Knight and Yorke, 2004), or with knowledge, action and identity (Coate and Barnett, 2005).
In other words, good quality education is concerned with 'complex achievements'.Knight and Yorke (2004) argue that graduate employability depends on making convincing claims to these achievements, while Newman et al., 2004: 72/3), writing of the USA, say Academic leaders often assume that business leaders will argue for narrow vocational skills.Regularly, however, the call from the business community has been for skills that sound remarkably like what academics describe as a liberal education.
To be exact, these achievements are not 'skills'.Consider the findings of three studies of what employers want in new graduate hires.
UK, 1997: employers want graduates with knowledge; intellect; willingness to learn; self-management skills; communication skills; team-working; interpersonal skills (Harvey et al.).
UK, 1999: small enterprises especially valued skill at oral communication, handling one' s own work load, team-working, managing others, getting to the heart of problems, critical analysis, summarising, and group problem-solving.
Valued attributes included being able to work under pressure, commitment, working varied hours, dependability, imagination/creativity, getting on with people, and willingness to learn (Yorke).
Ten EC countries + Japan, 2001: initiative; working independently; working under pressure; oral communication skills; accuracy, attention to detail; time management; adaptability; working in a team; taking responsibility and decisions; planning coordinating and organizing (Brennan et al.).
These are not 'skills'.They are complex achievements: they cannot be determinately described; their development is a matter of months and years; development is uncertain and not easily attributable to any one source, educational or otherwise; and they resist measurement and assessment.Let me illustrate just the third of those points by referring to Pascarella and Terenzini's literature review which associates 'self-rated job skills' with interactions at university with other students.… an important additional contribution of the research of the 1990s has been a better understanding of the kinds of peer interactions that are most influential.The student-peer contacts that matter most appear to be those that expose students to diverse racial, cultural, social, value, and intellectual perspectives … Net of confounding influences, interactions with diverse peers have moderate but consistently positive impacts on knowledge acquisition, dimensions of cognitive development … principled moral reasoning, and self-rated job skills after college (p.615).
The implication is that if we want to enhance student employability, we should maximise interactions with a wide range of others.Good quality courses will be designed to do so.
This points us to something else that is prominent in Pascarella and Ternzini's book.While learning is influenced by instruction and the curriculum in which it is embedded, the university as an environment makes a great difference: … we know what factors do differentiate among educationally effective institutions … student involvement in the academic and non-academic systems of an institution, the nature and frequency of student contact with peers and faculty members, interdisciplinary or integrated core curricula that emphasise making explicit connections across courses and among ideas and disciplines, pedagogies that encourage active student engagements in learning and that encourage application of what is being learned in real and meaningful settings, campus environments that encourage scholarship and provide opportunities for students to encounter different kinds of people and ideas, and environments that encourage and support exploration, whether intellectual or personal (p.641).
They go further and argue that sub-environments are crucial to learning quality, saying that ' … the majority of colleges and universities in the American postsecondary system have important subenvironments with more immediate and powerful effects on individual students' (p.89).This line of analysis is compatible with the growing number of studies showing the extent of non-formal learning.Of course, much knowledge comes from formal instruction and curriculum but, when it comes to complex achievements, then non-formal learning, happening in activity systems, becomes more salient, as our work on learning to teach in higher education has shown (Knight, Tait and Yorke, 2006).
Let me take stock of the argument.First, there is an implied claim that good quality courses will attend to complex achievements, as well as the cognitive ones with which we are familiar.Second, is the suggestion that their complexity means that it is hard to talk, in familiar terms, about pedagogies to promote complex achievements.Third comes the suggestion that high quality provision to promote these outcomes would involve attending to sub-environments, such as subject departments, and to the affordances or opportunities that they lay out to people whose learning is both formal and non-formal.Here is a fresh view of quality, which associates it with the design of environments in which learning is evoked.
The fourth step is to say that these affordances must be on-line and distributed, as well as face-to-face.There is no shortage of software and services to support the formation of any and all of the complex achievements mentioned above.It might be said that online environments are less immediate, less personal, less compelling and less affective than ones in which there is real presence, notwithstanding the enormous difference made by personalisation and social presence enhancements.But that is exactly why it is important for good quality courses to have on-line and distributed affordances -it is because graduate work involves using on-line environments that may be, in some respects, inferior to face-to-face ones but which are also, in other ways, superior.The first proposition of The Cluetrain Manifesto (Levine et al., 2000) is that markets are conversations.By extension, good quality courses will expect students to use web services for interacting with others --instant messaging, VOIP, wikis, enhanced presence tools, P3 systems -and quality courses support the development of social practices of engagement.
In so doing the chances are increased that complex achievements will manifest themselves.(There are some interesting suggestions that they may manifest in the years after graduation.)

Changing concepts of quality
I have used some educational ideas about learning and awareness of fastdeveloping possibilities for interactivity to develop notions of course quality that differ somewhat from those in fashion in the early days of distance and on-line learning.For example, quality has been identified with a professionallyproduced 'course in a box' (or, in the case of the UK Open University, 'course on the box').There were two main elements here -a product (commercially published or specially created by the university) and some form of tutorial service.The goal was to help individuals successfully to perform a range of cognitive operations --remember, understand, apply, analyze, evaluate --on the target material.This remains an important goal and a great deal of formal education is given over to its pursuit -especially to remembering and understanding.Good quality courses still support the art of memory and the pursuit of understanding, using multimedia demonstrations to show how things work, simulations to show our misunderstandings, on-demand quizzes to identify areas for further attention, intelligent systems to suggest ways of remedying failings and pose directions for development.Ideally, courses will recognise that the quality of thinking tends to be higher when peers converse about problems and that understandings flow from such interactions.The strength of this model can be inferred from the UK Open University's position as the highest-ranked of all UK universities in the 2005 national student satisfaction survey (Times Higher Educational Supplement, 23/9/2006).
But in the age of 'Google knowledge' there is less need to remember data or information and, once base understandings of a topic are formed, it is significantly easier to extend them using on-line resources and multiple forms of connectedness.Quality course design recognises this, cuts the clutter, helps students understand the structure of the subject matter (Bruner, 1966) and teaches them to take change of extending base understandings.This creates space to foster cognitive processes that have often been marginalised --apply, analyze, evaluate, create.Quality courses use pedagogic routines appropriate to these four cognitive processes.
There are other changes in thinking about quality that can be connected with deep changes in higher education in the past fifty years.Growth has been propelled by the development of applied and professional studies at undergraduate and master's level -studies geared to improving commercial, social and other productive activities.When higher education is in the business of professional formation, its point of reference is what professionals (should) do.Most accounts of professional work emphasise the inter-personal nature of professional work and the centrality of non-routine, expert decision-making.And professional work happens in the sea of connectedness afforded by our technologies with their services and practices.Its practitioners need complex achievements to do their complex work.
Good quality courses will therefore tend to be concerned with a range of cognitive processes and complex achievements.Insofar as it is appropriate to talk of pedagogies, they will be differentiated because different goals call for different engagements.Yet, there is a strong sense in which quality will lie not so much in the pedagogies as in sub-environments and their affordances -in learning cultures; in the connections between learning intentions and assessment practices; in the quality of search that the environments favour; in the range of connectednesses it promotes; and in the interplays that arise between 'I' and 'others'.Here quality inheres in the set of arrangements that helps participants to customise services and create understandings, identities and connections for themselves.This is especially true of professional courses at postgraduate levels where participants already have knowledge or have access to it; quality courses are (supported) arrangements that encourage sensemaking and wider formation as a professional.This is central to the work of our Practice-based Professional Learning Centre (http://cetl.open.ac.uk/pbpl/), whose job is to make learning from practice a feature of a much wider range of on-line courses than is currently the case.In a few years we might be saying that quality online courses in the Open University will generally have a strong element of practice-based learning in them.
It is sometimes said that course quality used to be seen in terms of product quality but is now seen as service quality.Some go further and say that support quality is becoming more important (Zuboff and Maxim, 2004).I have suggested that this trajectory is too simple.Service quality has always been important and product quality still matters -very much so at some levels and in some subject areas.I have argued that the changes have been in the ways in which we understand product, service and support.I have complemented this with the claim that complex achievements have acquired higher priority, which means that quality courses, especially postgraduate professional courses, are replete with opportunities that are likely to evoke the rich blends of intended learning.Lower overheads for course team By the standards of last decade, PCAP is deficient -no glossy course readers, for example.By the standards of the years to come it is also deficient, not least because we have not had the confident to centre on the co-creation of multiple understandings in multiply-enabled networks of knowledge identity and action.Be that as it may, this decade's notions of quality are qualitatively different than last decade's.

Openness and new partnerships
I finish this analysis by wondering in what sense it will make sense, in ten years' time, to talk about 'courses', let alone about 'quality courses'.
The Open Educational Resource movement (OER) may revolutionise our thinking about the nature of quality courses.The best known of the OER initiatives is the MIT OpenCourseWare initiative, which comprises more than 1400 courses from 33 academic departments (http://ocw.mit.edu/index.html).These resources, visited over 1.2 million times a month, have been widely translated and make a significant contribution to the UN's 'education for all' goal.If high quality courses are available for free on the web, then why should local providers create their own?In what sense does it make sense to ask about course quality, except to assure oneself that the MIT brand, for example, does warrant that the course will be a good one?
Yet the MIT courses are not complete learning experiences.With the OU's Open Content Initiative probably in mind, Smith and Casserly (2006: 9) write: At the core of the first model would be a coherent sequence of academic or technical content organized into a course module that is roughly equivalent to a university offering.The regularly updated content would be similar to those of a traditional course and could be multi-media, lecture, or mixed mode, and the instruction could be enhanced with artificial intelligence or not.The quality of the material would be vetted through peer reviews and user testing.Moreover, since the content would be free and open, people around the world could review it continuously.
Our  (Batelle, 2005) will recede and the idea that the university is a content publisher will collapse.(It is far from certain that traditional publishers will survive either).Quality will be reconstructed.3.With blogs, word-processors, search spreadsheets and maps for free, everyone can become a publisher.GPS handhelds and locational software (Google Maps) brings place back into social networks, delivering Harvey's (1990) vision of postmodernity as a state of globalisation and locales.Educationally, does teaching get replaced by 'editing', in its broadest sense?

Unblinkering quality
Questions about quality tend to treat the term as unproblematic.I have suggested that we need differentiated views that are sensitive to the high degree of uncertainty attached to the formation of complex outcomes.I have also argued that those multiple meanings are also shifting meanings and argued that the notion of a quality course may itself become obsolete.
My account of 'quality' resembles Law's (2004) analysis of another great modernist concept, 'method'.I have modified one of his paragraphs by using 'quality' where he uses 'method'.… quality in social science (and in natural science too) is enacted as a set of nineteenth-or even seventeenth-century Euro-American blinkers.This means that it misunderstands and misrepresents itself.Quality is not, I have argued, a more or less set of procedures for reporting on a given reality.Rather it is performative.Quality helps to produce realities.It does not do so freely and at whim.There is a hinterlands of realities, of manifest absences and Otherness, resonances and patterns of one kind and another already being enacted and quality cannot ignore these.At the same time, however, it is also creative.It makes new signals and new resonances, new manifestations and concealments, and it does so continuously (p.143) Too often 'quality' has been used as blinkers.Challenged by Amazon, Google, Skype, Yahoo and corporations yet to emerge, we must now unblinker ourselves.
OCI venture (http://oci.open.ac.uk/) sits well with our adoption of an open source approach to the development of our VLE.Together, OCI and Moodle foreshadow a very different relationship between a university and the world, inviting new partnerships, including partnerships with 'for profit' operations, that are based on co-creation and shared benefits.In this world, quality, partnerships and networks are inter-seamed.Similarly, web 2.0 (O'Reilly, 2005) carries fresh ideas about quality and has the potential fundamentally to challenge distance and on-line education business models.The following table adapts O'Reilly's eight core competencies of web 2.0 companies and relates them to the business of higher education.The challenge, particularly with mobile learning is to design low-bandwidth 'courses' that challenge participants in supported social learning systems.Quality, in this sense, means a retreat from the high bandwidth banquets of traditional courses.2. As semantic web technologies become more robust, so existing limitations to search