Some recent work on teacher education (free and downloadable):
http://ro.ecu.edu.au/ajte/vol39/iss4/4/
And on creativity in design (paywalled, free version here):
Categories
Category: Research
Categories
Distributing determination of quality #2
In the previous article, as in the book chapter below, we discuss the idea of ‘distributing the determination of quality’. I think a better term for what we are trying to say can be devised, one that is less of a mouthful (any ideas welcome).
This is a short post to describe some parts of the animal, even if we’re not sure what beast it is yet.
– Peer assisted learning is a fantastic example of distributing the determination of quality in higher education. For example, having peers that took a course mark that course in the following year as a part of their second year studies. If implemented badly this is exploitation. If implemented well then all can benefit
– I attended an session with the Queensland Department of Education, Training and Employment (DETE) on the theme of doing research in education in Queensland. One idea that came up was the idea that instead of each school striving to “maximise their NAPLAN results” and being judged on this (as is the current practice), imagine if each school was judged on how well the four closest schools performed? The idea is that instead of encouraging competition and individualism, it encourages altruism and reaching out to help those schools closest. It was not a serious suggestion, but rather a nice sketch of how the world could be if KPIs were something distributed to provide external motivation for altruism, rather than concentrated to provide external motivation for self-interest.
More examples to follow (any suggestions welcome too).
I’ve recently published an ADFI Blog post here. This post is a cut and paste of the content from this article:
An image drawn from Buddhist philosophy is of a spider’s web covered with dew-drops. The significance of the image is that its beauty comes from the way in which each dew drop reflects all other drops. The droplets are interconnected in this complex way, and their beauty comes from these connections.
Image Source: http://en.wikipedia.org/wiki/File:Dewy_spider_web.jpg
(Licensed as Creative Commons: Attribution-Sharealike, image by http://en.wikipedia.org/wiki/User:Fir0002)
The image is a powerful one to introduce the theme of how we determine quality (of resources, people, information) in a society that increasingly relies upon the internet. The internet has changed and will continue to change many sectors of society by making it cheap (trivially so) and accessible (we all know how to do it) to duplicate and communicate data anywhere in the world. As a society we are still exploring the effects and potential of this.
This blog post is about one idea within this context; that much of the disruptive innovation driven by the internet has come about not simply through increased inclusivity but, rather, through innovation in ways to distribute the determination of quality.
Whilst it is often simple to ‘scale things up’ with the internet and get more people involved in something, the hard part is finding a way to similarly scale up the way that quality is determined. (For a subjective definition of quality we adopt the notion of ‘fit for purpose’)
Some examples are helpful for introducing the idea and distilling a common narrative (Kelly, Sie, & Suwer, In press):
- In the 90′s anybody could create a web page with HTML, and many people did. However, judging the usefulness of a given site was a difficult problem. Google changed all that with their use of eigenvector centrality (within the ‘PageRank’ algorithm) to quantify quality based upon the interconnected network of web page links (Page, Brin, Motwani, & Winograd, 1999). A site’s value is judged based upon the links to it from other sites, and these links are weighted based upon their respective value. By mining the graph of connections the value of each site can be determined. (A fantastic intuitive understanding of the complex nature of this calculation can be gained through this NetLogo simulation of PageRank).
- In research administration there is a desire to ‘manage’ the quality of research and hence to measure it. Many academics publish many articles, but which ones can be judged as high quality? The field of bibliometrics attempts to respond to this problem through content and citation analysis. Some of most popular metrics (e.g. the SCIMago Journal Rank or SJR as it is commonly known) utilise applications of the same eigenvector centrality measure used in Google’s PageRank.
- The company AirBnB has provided a platform such that anybody can open up their house for guests, thus allowing individuals to compete with hotels to provide accommodation. The difficulty in setting up this platform was not so much the ability to list their house (increasing inclusivity), but rather the challenge of ensuring quality in the listings – people wanting to stay in accommodation register by uploading ID documents, people listing houses need to provide accurate photos, and ratings and reviews play a part in providing constant community feedback.
- A new company Uber allows for anybody to use their car to provide ‘taxi’ services to others using the platform that requires this service. The online platform removes barriers to entry (inclusivity) allowing anyone with a car to compete in providing taxi services and it maintains quality in a way that is similar to AirBnB.
- eBay similarly allowed for increased inclusivity in online trading, allowing anybody to turn their home into a warehouse for selling goods. The challenge of ensuring quality involves legal protections, reviews, ratings and metrics such as number of goods sold.
These few cherry-picked examples have a common narrative: the internet makes it possible to implement processes on a grand scale. Wherever this occurs, a need is created to distinguish quality. The above examples can all be seen as disruptive innovations within their sector. It can be argued that the significant innovation in each case is the way in which quality is determined within the large-scale community.
The current fashion for MOOCs (the acronym for which is increasingly irrelevant) in higher education provides a useful case study. The model of students being taught by a lecturer and tutors on campus has been around for centuries and is difficult to scale. Putting digital course content online and making it open has been around in various forms for a long time but offered a different kind of education. Downes and Siemens introduced the term MOOC to emphasise the way that online courses designed in a specific way could be massive, open and, in this way, fulfil the aims of connectivist pedagogy (what are now known as cMOOCs). However, determining the quality of students within MOOCS (both formative to aid the students and summative for the ‘gatekeeping’ role of university education) is difficult within this expanded context. Innovations such as automated marking and peer-assessment have been used within the MOOC context but, as the two links show, also strongly criticised.
Recent work has begun developing a collection of ways in which quality is assured online and organising them into a taxonomy:
- Implicit measures: Users of a service to do what they would normally do. Quality is measured using implicit metrics, such as Eigenvalue analysis and content analysis (e.g. PageRank) or indicators of behaviours (e.g. seller ratings of eBay based upon volume sold, and measures of contributions on StackOverflow)
- Explicit measures: Users of a service or paid professionals take specific actions to ensure quality. Examples include paid staff rating contributions to the OER commons, ebay members rating their experiences, or Amazon buyers reviewing books.
A theory for quality in connected systems?
It is useful to identify and describe these measures of quality, but a more profound question is: Can we identify a technique that could aid this kind of innovation for determining quality?
An inspiring example is Shannon’s work in developing information theory (Shannon, 2001). Shannon saw that engineers were coming up with ways of sending and receiving signals with greater or less bandwidths and in the presence of noise (interference). Rather than contribute to ad-hoc, domain specific solutions, Shannon was able to develop a mathematical representation for the problem and consequent implications of this representation that make up the basis for much of the cryptography and communications that we use today.
What could such a representation, abstracted away from eigenvector centrality or specific measures look like? Or perhaps, at the very least, a general formulation of an approach that could be used in determining quality regardless of the medium or context.
This blog post is much more about questions than answers. The spiders’ web remains a beautiful symbol for our ever more connected world. Within recent centuries we have moved from a philosophy of the whole-in-the-parts to mathematical and applied representations of it. We propose that this way of thinking is particularly useful for distributing the determination of quality – there remains much more to be uncovered in applying this type of thinking.
If you would like to be involved in this research as a collaborator or higher degree research student, contact Dr Nick Kelly.
Image credit: Dewy Spider Web by User:Fir0002 Used under Creative Commons Attribution-Share Alike 3.0 Unported licence.
References
Kelly, N., Sie, R., & Suwer, R. (In press). Innovating processes to determine quality alongside increased inclusivity in higher education. In M. Keppell, S. Reushle & A. Antonio (Eds.), Open Learning and Formal Credentialing in Higher Education: Curriculum Models and Institutional Policies: IGI Global.
Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: bringing order to the web.
Shannon, C. E. (2001). A mathematical theory of communication. ACM SIGMOBILE Mobile Computing and Communications Review, 5(1), 3-55.
If you would like to be involved in this research as a collaborator or higher degree research student, contact Dr Nick Kelly.
References
Kelly, N., Sie, R., & Suwer, R. (In press). Innovating processes to determine quality alongside increased inclusivity in higher education. In M. Keppell, S. Reushle & A. Antonio (Eds.), Open Learning and Formal Credentialing in Higher Education: Curriculum Models and Institutional Policies: IGI Global.
Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: bringing order to the web.
Shannon, C. E. (2001). A mathematical theory of communication. ACM SIGMOBILE Mobile Computing and Communications Review, 5(1), 3-55.
Nobody really knows how exactly many teachers drop out of the profession. A couple of recent articles are keeping this issue in the spotlight, which is a good thing:
This from the conversation
And this a reference to the Gallant and Riley study (Monash).
50% in the first five years is a figure thrown around, 30% in the first three years is another. And unfortunately there is reason to believe that many of the leavers are good quality teachers.
How to keep them in the profession?
Some suggestions that have been raised: increase salaries of experienced teachers and finding a way for society to recognise the professionalism and technical skills required for the profession, giving them the social status they ought to have.
Many ideas about what it means to be a quality teacher are very well articulated in this article by Raewyn Connell.
Categories
ASCILITE Presentation
Presenting about support for beginning teachers at ASCILITE. Most useful thing about the conference was finding out about what the Keeping Cool people are doing in this area.
Categories
Academic WordPress Todo List
This is a short post about the joys of being an academic in the 21st century. There are more digital communities available than there are predatory open access journals these days and so what is useful online and what is not?
Clearly actually doing research and publishing is the important thing, but my thinking is that the following things are also useful:
- Slideshare is actually helpful. I’ve noticed some professors that put their slideshare on the last slide, and if they’ve been interesting then it makes it easy for a followup. I’ve also stumbled across collaborators by first coming across their slideshare in Google search
- Find one place to store all your papers. There are lots of options, like your institutional ePrints or Mendeley or Academia.edu or your personal website. I’m finding that currently I’m trying to keep three of these updated and it doesn’t work so well
I’m still figuring out if blogging is useful (the fact that my colleague’s blog was cited in George Siemens’ book is a good indicator that it probably is if done well) but the point of this website is to try it out and see. I’m sure there’s a clever way of doing this online presence thing that involves half an hour a week, so I’ll keep this updated as I figure it out.
I feel like the ideal kind of a pipeline would be:
- Write an article and publish it
- Deposit in the institutional ePrints
- Write a short blog post about why the world is a different place with this incredibly important finding and linking to the paper
- This blog post would then be automatically echoed in social media in some useful way. (What does this look like? Facebook? Twitter?)
I’m figuring all of this out (including the purpose of any of it, given that quality research is the hard part, but then I’m still figuring out the purpose of that is too, so bear with me for some time).
Categories
Research overview
I am a Research Fellow based at the Australian Digital Futures Institute (ADFI). This web site is about bringing together all of my research to communicate the ideas beyond publications and to listen to any comments.
A good place to start is with an overview of all the research that I’m actively involved in at the moment, with some links to the projects and the organisations responsible:
- Virtual Communities of Practice for Transition from Higher Education to Profession (ADFI)
Universities build strong communities amongst pre-service teachers. Often much of the community support is lost in the transition into practice. How could we design and foster a virtual community of practice within the university that could then support these teachers as well as possible in their first year of practice?
A collaboration led by Lina Markauskaite of the CoCo research centre using results from a study of students as they learn about the complexity of climate change with a model. The work examines epistemic fluency in model based learning.
More info to follow, but for now view papers here where you can see the published work that has come out recently.
- Meta-analysis of studying the process of learning (ADFI, CoCo)
- Models of Generate and Interpret in Design (ADFI, Design Lab, UNCC)
- Open Education Resources University (ADFI, OER Foundation)
A thread that runs through the work is creating communities for supporting learning – where learning is something takes place both inside and outside institutions.