Sunday, May 03, 2015

The Study, and Other Stuff

There are three separate threads in Siemens's response to my last post, all of which are fascinating:

  • The thread concerning whether or not the study he published was bad,
  • The thread examining the question of whether universities can be a valuable force for social equity, and
  • My own experiences of the university system.
Though the latter two threads are of endless interest, I'd really rather only focus on the first, for today.

Whether or not the study he published was bad

Siemens writes, "Stephen expands on his primary concerns which are about educational research in general." Let me be clear: I was making this statement about this study in particular. That's why I cited work from the study itself. Yes, I believe that educational work in general is pretty poor. But my focus was on this particular example.

I think he agrees with me, in part:

Educational research is often poorly done. Research in social systems is difficult to reduce to a set of variables and relationships between those variables. Where we have large amounts of data, learning analytics can provide insight, but often require greater contextual and qualitative data. ... The US Department of Education has a clear articulation of what they will count as evidence for grants. It’s a bit depressing, actually, a utopia for RCTs (Randomized Controlled Trials).

And he says:
Stephen then makes an important point and one that needs to be considered that the meta-studies that we used are “hopelessly biased in favour of the traditional model of education as practiced in the classrooms where the original studies took place.” This is a significant challenge. How do we prepare for digital universities when we are largely duplicating classrooms? Where is the actual innovation? (I’d argue much of it can be fore in things like cmoocs and other technologies that we address in chapter 5 of the report). Jon Dron largely agrees with Stephen and suggests that a core problem exists in the report in that it is a “view from the inside, not from above.”
So, from this, it appears that he agrees with my criticisms.

He nonetheless persists with his defense, focusing on the fifth paper in the study, first suggesting I don't find a lot to disagree with about it, and second, suggesting it is a vehicle for a conversation between two versions of myself. He also finds fault with some other criticisms:
The names listed were advisors on the MOOC Research Initiative – i.e. they provided comments and feedback on the timelines and methods. They didn’t select the papers. The actual peer review process included a much broader list, some from within the academy and some from the outside. 

Who selected the review committee? Who are the people 'from the outside' that were on it? Here's the best we have on the review process itself. Here are the project reports. All of this was set in motion by the committee I named in my previous post. If there's another list of names of people who were responsible for the outcome, they should be named. Otherwise, the people named are the people responsible. You can't name a list of names and then say it wasn't them.

In his defense of the fifth paper (he seems not to defend the first four studies, the 'histories', at all) he also writes:
In my previous post, I stated that we didn’t add to citations. We analyzed those that were listed in the papers that others submitted to MRI. Our analysis indicated that popular media influenced the MOOC conversation and the citations used by those who submitted to the grant.
I recognize this. What I am is saying is that it seems to me that the 28 winners of a major education research grant competition would have demonstrated more depth of understanding that is apparent from the summary study that resulted. Maybe I should not have expected more from what was essentially an automated and quantitative analysis of the papers (because there are individually some bright spots). But when we look at the citations - which is essentially what we were provided - the results overall are not reassuring.

That's it for Siemens's defense of the study. The core of my criticism, which is addressed mostly at the first four chapters, s is not addressed. Let me reiterate them here:
  • They all have very small sample sizes, usually less than 50 people, with a maximum size less than 200 people
  • The people studied are exclusively university students enrolled in a traditional university course
  • The method being studies is almost exclusively the lecture method
  • The outcomes are assessed almost exclusively in the form of test results
  • Although many are 'controlled' studies, most are not actually controlled for "potential confounders"
  • All these criticisms apply if you think this is the appropriate sort of study to measure educational effectiveness, which I do not.
I would not like to add that my criticisms are reinforced by two additional authors.

Although Jon Dron says "as such reports go, I think it is a good one," he writes:

For the most part, this report is a review of the history and current state of online/distance/blended learning in formal education. This is in keeping with the title, but not with the ultimate thrust of at least a few of the findings. That does rather stifle the potential for really getting under the skin of the problem. It's a view from the inside, not from above. 

And additionally, George Veletsianos writes,

One of Downes  criticisms is the following: “the studies are conducted by people without a background in education.” This finding lends some support to his claim, though a lot of the research on MOOCs is from people affiliated with education, but to support that claim further one could examine the content of this papers and identify whether an educational theory is guiding their investigations.

I don't think it matters whether the investigation is informed by an educational theory - all I care about is that the studies contribute in a useful, relevant and credible way to the field.

Finally, Siemens says, "The appeal to evidence is to essentially state that opinions alone are not sufficient."

It can be allowed that Siemens's use of "we" in the Chronicle article "is about the academy’s embrace of MOOCs." But as I pointed out, there's no mistaking his suggestion that the people outside the academy, the Alt-ac people, do not rely on evidence. This is what he says when he says, "Another approach, and one that I see as complimentary and not competitive, is to emphasize research and evidence."

I have never suggested that opinion alone is sufficient, and never would. But he has to cease characterizing the alternatives as not evidence based. Because I believe the opposite. I believe that the controlled trials offered in the study misrepresent what little evidence they provide, and I believe that the alternative approaches offer substantially more evidence than is allowed.

Siemens says, "While Stephen says our evidence is poor, he doesn’t provide what he feels is better evidence." I did once author a Guide to the Logical Fallacies, where I discuss the statistical problems. I've also talked about the same issue of evidence as it related to public policy. I've talked about research methodologies a number of times. And just the other day, I linked to a study I felt did pass muster (and indeed, over the years, I've linked to lots of things that I felt met the appropriate standards of research and evidence). And the body of my work, grounded in practical application and observation, stands as an example of what I feel constitutes "better evidence."

The Other Stuff

It's late and I don't want to longer on the off-topic stuff. But I also want to address a few things.

It's true that I am not a fan of universities and do not feel they support our common objective of " an equitable society with opportunities for all individuals to make the lives that they want without institutions (and faculty in this case) blocking the realization of those dreams."

This does not mean that I want to see them eliminated. And (contrary to Sebastian Thrun) I expect their numbers will multiply exponentially in the future.

But they need to be reformed, and they need to be brought around to the idea that social and economic equity are important. Because as it stands, they are one of the largest bastions in society standing against that idea. Here are a few of the ways:

- universities foster the perpetuation of a social elite, especially through exclusive institutions (Harvard, Yale, etc), legacy admissions, and perpetuation of a private social society consisting pretty much only of the one-percent

- universities bleed those outside the upper classes by consistently responding to society's demand for access with higher and higher tuition fees

- universities have fostered the creation of a low-paid academic underclass in order to support the students that pay these higher fees, and resist any suggestion that they should be fairly compensated, and actively resist unionization

- universities and professors continue to contribute to mechanisms which keep academic research behind expensive paywalls - indeed, they are so indifferent to these costs that they must be required by mandates and laws to open access to their research

- private universities operate tax-free, raise substantial endowment funds (sometimes in the billions), yet always plead poverty, and are typically the prime recipient of funding provided by governments and foundations attempting to support projects leading to the betterment of social and economic conditions

- they then waste that money, and a lot of other money, padding their own resumes and producing research such as the body of work I find myself criticizing today

Yes, perhaps universities could act as a force that promotes social and economic equity. They certainly have the talent and resources. But they don't, they don't want to, and they resist any attempt to make them do it.

It is true that I was badly treated by my PhD committee. But this is not a case of "today affirming that the Stephen in front of the phd committee made the right decision – that there are multiple paths to research, that institutions can be circumvented and that individuals, in a networked age, have control and autonomy." Why not? A couple of reasons:

On the idea that, individuals, in a networked age, (should) have control and autonomy: I have always believed that. I believed that long before I ever stood before a PhD committee.

On the idea that "the Stephen that today has exceeded the impact of members on that committee through blogging, his newsletter, presentations, and software writing."This may or may not be true. But I have never believed that I have been more influential because I have worked outside of academia.

I have been influential despite being outside academia. I have been influential despite not having a professor's wages, the support of grad students, a year off every seven, tenure, funding from foundations, grants and agencies, book contracts, and the rest. No university in the world would ever hire me, because they consider me unqualified. I don't regard any of this really as an upside.

Because that's what academia does. It wields huge sums of money and the support to achieve certain social and economic outcomes. I just wish it was wielding this power for good, rather than indifference. But I don't think it ever will.

Saturday, May 02, 2015

Research and Evidence

I wrote the other day that the study released by George Siemens and others on the history and current state of distance, blended, and online learning was a bad study. I said, "the absence of a background in the field is glaring and obvious." In this I refer not only to specific arguments advanced in the study, which to me seem empty and obvious, but also the focus and methodology, which seem to me to be hopelessly naive.

Now let me be clear: I like George Siemens, I think he has done excellent work overall and will continue to be a vital and relevant contributor to the field. I think of him as a friend, he's one of the nicest people I know, and this is not intended to be an attack on his person, character or ideas. It is a criticism focused on a specific work, a specific study, which I believe well and truly deserves criticism.

And let me clear that I totally respect this part of his response, where he says that "in my part of the world and where I am currently in my career/life, this is the most fruitful and potentially influential approach that I can adopt." His part of the world is the dual environments of Athabasca University and the University of Texas at Arlington, and he is attempting to put together major research efforts around MOOCs and learning analytics. He is a relatively recent PhD and now making a name for himself in the academic community.

Unfortunately, in the realm of education and education theory, that same academic community has some very misguided ideas of what constitutes evidence and research. It has in recent years been engaged in a sustained attack on the very idea of the MOOC and alternative forms of learning not dependent on the traditional model of the professor, the classroom, and the academic degree. It is resisting, for good reason, incursions from the commercial sector into its space, but as a consequence, clinging to antiquated models and approaches to research.

Perhaps as a result, part of what Siemens has had to do in order to adapt to that world has been to recant his previous work. The Chronicle of Higher Education, which for years has advanced the anti-technology and anti-change argument on behalf of the professoriate, published (almost gleefully, it seemed to me), this abjuration as part and parcel of its article constituting part of the marketing campaign for the new study.
When MOOCs emerged a few years ago, many in the academic world were sent into a frenzy. Pundits made sweeping statements about the courses, saying that they were the future of education or that colleges would become obsolete, said George Siemens, an author of the report who is also credited with helping to create what we now know as a MOOC.

“It’s almost like we went through this sort of shameful period where we forgot that we were researchers and we forgot that we were scientists and instead we were just making decisions and proclamations that weren’t at all scientific,” said Mr. Siemens, an academic-technology expert at the University of Texas at Arlington.

Hype and rhetoric, not research, were the driving forces behind MOOCs, he argued. When they came onto the scene, MOOCs were not analyzed in a scientific way, and if they had been, it would have been easy to see what might actually happen and to conclude that some of the early predictions were off-base, Mr. Siemens said.
This recantation saddens me for a variety of reasons. For one this, we - Siemens and myself and others who were involved in the development of the MOOC - made no such statements. In the years between 2008, when the MOOC was created, and 2011, when the first MOOC emerged from a major U.S. university, the focus was on innovation and experimentation in a cautious though typically exuberant attitude. 

Yes, we had long argued that colleges and education had to change. But none of us ever asserted that the MOOC would accomplish this in one fell swoop. Those responsible for such rash assertions were established professors with respected academic credentials who came out of the traditional system, set up some overnight companies, and rashly declared that they had reinvented education.

It's true, Siemens has moved over to that camp, now working with EdX rather than the connectivist model we started with. But the people at EdX are equally rash and foolish:
(Anant) Argarwal (who launched EdX) is not a man prone to understatement. This, he says, is the revolution. "It's going to reinvent education. It's going to transform universities. It's going to democratise education on a global scale. It's the biggest innovation to happen in education for 200 years." The last major one, he says, was "probably the invention of the pencil". In a decade, he's hoping to reach a billion students across the globe. "We've got 400,000 in four months with no marketing, so I don't think it's unrealistic."
Again, these rash and foolish statements are coming from a respected university professor, a scion of the academy, part of this system Siemens is now attempting to join. As he recants, it is almost as though he recants for them, and not for us. But the Chronicle (of course) makes no such distinction. Why would it?

But the saddest part is that we never forgot that we were scientists and researchers. As I have often said in talks and interviews, there were things before MOOCs, there will be things after MOOCs, and this is only one stage in a wider scientific enterprise. And there was research, a lot of it, careful research involving hundreds and occasionally thousands of people, which was for the most part ignored by the wider academic community, even though peer reviewed and published in academic journals. Here's a set of papers by my colleagues at NRC, Rita Kop, Helene Fournier, Hanan Sitlia, Guillaume Durand. An additionally impressive body of papers has been authored and formally published by people like Frances Bell, Sui Fai John Mak, Jenny Mackness, and Roy Williams. This is only a sampling of the rich body of research surrounding MOOCs, research conducted by careful and credible scientists.

I would be remiss in not citing my own contributions, a body of literature in which I carefully and painstakingly assembled the facts and evidence leading toward connectivist theory and open learning technology. The Chronicle has never allowed the facts to get in the way of its opinions, but I have generally expected much better of Siemens, who is (I'm sure) aware of the contributions and work of the many colleagues that have worked with us over the years.

Here's what Siemens says about these colleagues in his recent blog post on the debate:
One approach is to emphasize loosely coupled networks organized by ideals through social media. This is certainly a growing area of societal impact on a number of fronts including racism, sexism, and inequality in general. In education, alt-ac and bloggers occupy this space. Another approach, and one that I see as complimentary and not competitive, is to emphasize research and evidence. (My emphasis)

In the previous case he could have been talking about the promulgators of entities like Coursera, Udacity and EdX, and the irresponsible posturing they have posed over the years. But in this case he is talking very specifically about the network of researchers around the ideas of the early MOOCs, connectivism, and related topics.

And what is key here is that he does not believe our work was based in research and evidence. Rather, we are members of what he characterizes as the 'Alt-Ac' space - "Bethany Nowviskie and Jason Rhody 'alt-ac' was shorthand for 'alternative academic' careers." Or: "the term was, in Nowviskie’s words,' a pointed push-back against the predominant phrase, 'nonacademic careers.' 'Non-academic' was the label for anything off the straight and narrow path to tenure.'" (Inside Higher Ed). Here's Siemens again:

This community, certainly blogs and with folks like Bonnie Stewart, Jim Groom, D’Arcy Norman, Alan Levine, Stephen Downes, Kate Bowles, and many others, is the most vibrant knowledge space in educational technology. In many ways, it is five years ahead of mainstream edtech offerings. Before blogs were called web 2.0, there was Stephen, David Wiley, Brian Lamb, and Alan Levine. Before networks in education were cool enough to attract MacArthur Foundation, there were open online courses and people writing about connectivism and networked knowledge. Want to know what’s going to happen in edtech in the next five years? This is the space where you’ll find it, today.
He says nice things about us. But he does not believe we emphasize research and evidence.

With all due respect, that's a load of crap. We could not be "what’s going to happen in edtech in the next five years" unless we were focused on evidence and research. Indeed, the reason why we are the future, and not (say) the respected academic professors in tenure track jobs is that we, unlike them, respect research and evidence. And that takes me to the second part of my argument, the part that states, in a nutshell, that what was presented in this report does not constitute "research and evidence." It's a shell game, a con game.

Let me explain. The first four chapters of this study are instances of what is called a 'tertiary study' (this is repeated eight times in the body of the work). And just as "any tertiary study is limited by the quality of data reported in the secondary sources, this study is dependent on the methodological qualities of those secondary sources." (p. 41) So what are the 'secondary sources'? You can find them listed in the first four chapters (the putative 'histories') (for example, the list on pp. 25-31). These are selected by doing a literature search, then culling them to those that meet the study's standards. The secondary surveys round up what they call 'primary' research, which are direct reports from empirical studies.

Here's a secondary study that's pretty typical: 'How does tele-learning compare with other forms of education delivery? A systematic review of tele-learning educational outcomes for health professionals'.The use of the archaic term 'tele-learning' may appear jarring, but despite many of the studies being from the early 2000s I selected this one as an example because it's relatively recent, from 2013. This study (and again, remember, it's typical, because the methodology in the tertiary study specifically focuses on these types of studies):
The review included both synchronous (content delivered simultaneously to face-to-face and tele-learning cohorts) and asynchronous delivery models (content delivered to the cohorts at different times). Studies utilising desktop computers and the internet were included where the technologies were used for televised conferencing, including synchronous and asynchronous streamed lectures. The review excluded facilitated e-learning and online education models such as the use of social networking, blogs, wikis and BlackboardTM learning management system software.

Of the 47 studies found using the search methods, 13 were found to be useful for the purposes of this paper. It is worth looking at the nature of this 'primary literature':

(Sorry about the small size - you can view the data in the original study, pp. 72-73)

Here's what should be noticed from these studies:
  • They all have very small sample sizes, usually less than 50 people, with a maximum size less than 200 people
  • The people studies are exclusively university students enrolled in a traditional university course
  • The method being studies is almost exclusively the lecture method
  • The outcomes are assessed almost exclusively in the form of test results
  • Although many are 'controlled' studies, most are not actually controlled for "potential confounders"
This is what is being counted as "evidence"for "tele-learning educational outcomes." No actual scientific study would accept such 'evidence' for any conclusion, however tentative. But this is typical and normal in the academic world Siemens is attempting to join, and this is by his own words what constitutes "research and evidence."

Why is this evidence bad? The sample sizes are too small for quantificational results (and the studies are themselves are inconsistent so you can't simply sum the results).The sample is biased in favour of people who have already had success in traditional lecture-based courses, and consists of only that one teaching method. A very narrow definition  of 'outcomes' is employed. And other unknown factors may have contaminated the results. And all these criticisms apply if you think this is the appropriate sort of study to measure educational effectiveness, which I do not.

I said above it was a con game. It is. None of these studies is academically rigorous. They are conducted by individual professors running experiments on their own (or sometimes a colleague's) classes.The studies are conducted by people without a background in education, subject to no observational constraints, employing a theory of learning which has been for decades outdated and obsolete. These people have no business pretending that what they are doing is 'research'. They are playing at being researchers, because once you're in the system, you are rewarded for running these studies and publishing the results in journals specifically designed for this purpose.

What it reminds me of is the sub-prime mortgage crisis. What happened is that banks earned profits by advancing bad loans to people who could not afford to pay them. The value of these mortgages was sliced into what were called 'tranches' (which is French for 'slice', if you ever wondered) and sold as packages - so they went from primary sources to secondary sources. These then were formed into additional tranches and sold on the international market. From secondary to tertiary. By this time they were being offered by respectable financial institutions and the people buying them had no idea how poorly supported they were. (I'm not the first to make this comparison.)

Not surprisingly, the reports produce trivial and misleading results, producing science that is roughly equal in value to the studies that went into it. Let's again focus on the first chapter. Here are some of the observations and discussions:
it seems likely that asynchronous delivery is superior to traditional classroom delivery, which in turn is more effective than synchronous distance education delivery. (p. 38)

both synchronous and asynchronous distance education have the potential to be as effective as traditional classroom instruction (or better). However, this might not be the case in the actual practice of distance education (p. 39)

all three forms of interaction produced positive effect sizes on academic performance... To foster quality interactions between students, an analysis of the role of instructional design and instructional interventions planning is essential.

In order to provide sufficient academic support, understanding stakeholder needs is a main prerequisite alongside the understanding of student attrition (p.40)

I'm not saying these are wrong so much as I am saying they are trivial. The field as a whole (or, at least, as I understand it) has advanced far beyond talking in such unspecific generalities as 'asynchronous', 'interaction' and 'support'. Because the studies themselves are scientifically empty, no useful conclusions can be drawn from the metastudy, and the tertiary study produces vague statements that are worse than useless (worse, because they are actually pretending to be new and valuable, to be counted as "research and evidence" against the real research being performed outside academia).

Here is the 'model' of the field produced by the first paper:

It's actually more detailed than the models provided in the other papers. But it is structurally and methodologically useless, and hopelessly biased in favour of the traditional model of education as practiced in the classrooms where the original studies took place. At best it could be a checklist of things to think about if you're (say) using PowerPoint slides in your classroom. But in reality, we don't know what the arrows actually mean, the 'interaction' arrows are drawn from Moore (1989) , and the specific bits (eg. "use of LMS") say nothing about whether we should or whether we shouldn't.

The fifth chapter of the book is constructed differently from the first four, being a summary of the results submitted from the MOOC Research Institute (MRI). Here's how it is introduced:
Massive Open Online Courses (MOOCs) have captured the interest and attention of academics and the public since fall of 2011 (Pappano, 2012). The narrative driving interest in MOOCs, and more broadly calls for change in higher education, is focused on the promise of large systemic change.

The unfortunate grammar obscures the meaning, but aside from the citation of that noted academic, Laura Pappano of the New York Times, the statements are generally false. Remember, academics were studying MOOCs prior to 2011. And the interest of academics (as opposed to hucksters and journalists) was not focused on 'the promise of large systemic change' nearly so much as it was to ionvestigate the employment of connectivist theory in practice. But of course, this introduction is not talking about cMOOs at all, but rather, the xMOOCs that were almost exclusively the focus of the study.

Indeed, it is difficult for me to reconcile the nature and intent of the MRI with what Siemens writes in his article:
What I’ve been grappling with lately is “how do we take back education from edtech vendors?”. The jubilant rhetoric and general nonsense causes me mild rashes. I recognize that higher education is moving from an integrated end-to-end system to more of an ecosystem with numerous providers and corporate partners. We have gotten to this state on auto-pilot, not intentional vision.

Let's examine the MOOC Research Institute to examine this degree of separation:
MOOC Research Initiative (MRI) is funded by the Bill & Melinda Gates Foundation as part of a set of investments intended to explore the potential of MOOCs to extend access to postsecondary credentials through more personalized, more affordable pathways.
To support the MOOC Research Initiative Grants, the following Steering Committee has been established to provide guidance and direction:
Yvonne Belanger, Gates Foundation
Stacey Clawson, Gates Foundation
Marti Cleveland-Innes, Athabasca University
Jillianne Code, University of Victoria
Shane Dawson, University of South Australia
Keith Devlin, Stanford University
Tom (Chuong) Do, Coursera
Phil Hill, Co-founder of MindWires Consulting and co-publisher of e-Literate blog
Ellen Junn, San Jose State University
Zack Pardos, MIT
Barbara Means, SRI International
Steven Mintz, University of Texas
Rebecca Petersen, edX
Cathy Sandeen, American Council on Education
George Siemens, Athabasca University
With a couple of exceptions, these are exactly the people and the projects that are the "edtech vendors" vendors Siemens says he is trying to distance himself from. He has not done this; instead he has taken their money and put them on the committee selecting the papers that will be 'representative' of academic research taking place in MOOCs.

Why was this work necessary? We are told:
Much of the early research into MOOCs has been in the form of institutional reports by early MOOC projects, which offered many useful insights, but did not have the rigor — methodological and/or theoretical expected for peer-reviewed publication in online learning and education (Belanger & Thornton, 2013; McAuley, Stewart, Siemens, & Cormier, 2010).

We already know that this is false - and it is worth noting that this study criticizing the lack of academic rigour cites a paper titled  'Bioelectricity: A Quantitative Approach' (Belanger & Thornton, 2013) and an unpublished paper from 2010 titled 'The MOOC model for digital practice' (McAuley, Stewart, Siemens, & Cormier, 2010). A lot of this paper - and this book - is like that. Despite all its pretensions of academic rigour, it cites liberally and lavishly from non-academic sources in what appears mostly to be an effort to establish its own  relevance and to disparage the work that came before.

I commented on this paper in my OLDaily post:

The most influential thinker in the field, according to one part of the study, is L. Pappano (see the chart, p. 181). Who is this, you ask? The author of the New York Times article in 2012, 'The Year of the MOOC'. Influential and important contributors like David Wiley, Rory McGreal, Jim Groom, Gilbert Paquette, Tony Bates (and many many more)? Almost nowhere to be found.

Here is the chart of citations collated from the papers selected by the committee for the MOOC Research Network (p. 181):

 Here is the citation frequencies from the same papers (p. 180):

What is interesting to note in these citations is that the people who Siemens considers to be 'Alt-Ac' above - Mackness, Stewart, Williams, Cormier, Kop, Williams, Mackness - all appear in this list. Some others - Garrison (I assume they mean Randy Garrison, not D.D.) and Terry Anderson, notably, are well known and respected writers in the field. The research we were told several times does not exist apparently does exist. The remainder come from the xMMOC community, for example,  Pritchard from EdX, Chris Peich from Stanford, Daniel Seaton (EdX). Tranches.

But what I say about the rest of the history of academic literature in education remains true. The authors selected to be a part of the MOOC Research Institute produced papers with only the slightest - if any - understanding of the history and context in which MOOCs developed. They do not have a background in learning technology and learning theory (except to observe that it's a good thing). The incidences of citations arise from repeated references to single papers (like this one) and not a depth of literature in the field.

What were the conclusions of this fifth paper? As a result, nothing more substantial than the first four (quoted, pp. 188-189):
  • Research needs to create with theoretical underpinnings that will explain factors related to social aspects in MOOCs
  • Novel theoretical and practical frameworks of understanding and organizing social learning in MOOCs are necessary
  • The connection with learning theory has also been recognized as another important feature of the research proposals submitted to MRI
  • The new educational context of MOOCs triggered research for novel course and curriculum design principles
This is why I said in my assessment of the paper that "the major conclusion you'll find in these research studies is that (a) research is valuable, and (b) more research is needed." These are empty conclusions, suggesting that either the authors of the original papers, or the authors summarizing the papers, had almost nothing to say.

In summary, I stand by my conclusion that the book is a muddled mess. I'm disappointed that Siemens feels the need to defend it by dismissing the work that most of his colleagues have undertaken since 2008, and by advancing this nonsense as "research and evidence."

Thursday, April 30, 2015

Non-Research Citations in the Siemens Research Study

Defending himself against my criticism of his recently released research study on distance and online learning, George Siemens tweets:

Au contraire mon frère. There are many non-research articles cites, with a particular preference toward foundations, consultants, a few blogs and news and magazine articles. The non-research citations are as selective and ill-informed as the formal citations.

Personally, I have no objection to citing from supposedly non-research sites; I do it all the time.But I don't do it while claiming to not be doing it.

Here they are, collected from the references at the end of the various articles:

Allen, C. (2004). Life with alacrity: Tracing the evolution of social software. Retrieved from

Allen, I. E., & Seaman, J. (2011). Going the distance: Online education in the United
States, 2011 (Survey Report). Babson Survey Research Group. Retrieved from

Allen, I. E., & Seaman, J. (2013). Changing course: Ten Years of Tracking Online Education
in the United States. Sloan Consortium. PO Box 1238, Newburyport, MA 09150.

Allen, I. E., Seaman, J., & Garrett, R. (2007). Blending in: The extent and promise of
blended education in the United States. Sloan Consortium. Retrieved from

Azevedo, R. (1993). A meta-analysis on the effects of computer-presented feedback on learning
from computer-based instruction. The Department of Education, Concordia University

Beinkowski, M., Feng, M., & Means, B. (2012). Enhancing Teaching and Learning
Through Educational Data Mining and Learning Analytics: An Issue Brief (No.
ED-04-CO-0040) (pp. 1–57). US Department of Education, Office of Educational
Technology. Retrieved from

Belanger, Y., & Thornton, J. (2013). Bioelectricity: A Quantitative Approach.
Duke University. Retrieved from

Bordón, P., & Braga, B. (2013). Employer Learning, Statistical Discrimination and
University Prestige. Retrieved from

Clardy, A. (2009). Distant, On-line Education: Effects, Principles and Practices. Online
Submission, Retreived from ERIC Database. Retrieved from http://files.eric.

Coughlan, S. (2014, April 8). The irresistible urge for students to talk. Retrieved April 18,
2014, from

Dua, A. (2013). Voice of the Graduate. McKinsey & Company. Retrieved from http://

Eaton, J. S. (2001). Distance learning: Academic and political challenges for higher
education accreditation. Council for Higher Education Accreditation
Washington, DC.

Friedman, T. L. (2012, May 15). Come the revolution. The New York Times. Retrieved

GSV Advisors. (2012). Fall of the wall: Capital flows to education innovation. Chicago, IL:
Global Silicon Valley (GSV) Advisors. Retrieved from

Jaggars, S., & Bailey, T. R. (2010). Effectiveness of fully online courses for college students:
Response to a Department of Education meta-analysis. Retrieved from http://

Jordan, K. (2013). MOOC Completion Rates: The Data. Retrieved from http://www.

Laitinen, A. (2012). Cracking the Credit Hour. New America Foundation. Retrieved from

Learned, W. S., & Wood, B. D. (1938). The student and his knowledge: A report to the
Carnegie Foundation on the results of the high school and college examinations
of 1928, 1930, and 1932. The Carnegie Foundation for the Advancement of

McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for
digital practice. Retrieved from

McGuire, R. (2014). Hacking the hacker school: How the bootcamp is being taken to
scale outside the coding world. Retrieved December 20, 2014, from http://

Naughton, J. (2012, December 29). LinkedIn endorsements turn you into the
product. Retrieved January 16, 2015, from

OECD Publishing. (2013). Education at a Glance 2013: OECD Indicators. Retrieved from

Pappano, L. (2012, November 2). The Year of the MOOC. The New York Times.
Retrieved from

Paul, D. S. (2001). A meta-analytic review of factors that influence the effectiveness of
Web-based training within the context of distance learning. Texas A&M University.

Rainie, L. (2010). Internet, broadband, and cell phone statistics. Pew Internet & American
Life Project, 5. Retrieved from

Selwyn, N., & Bulfin, S. (2014). The discursive construction of MOOCs as educational
opportunity and educational threat. MOOC Research Initiative (MRI) - Final
Report. Retrieved from

Shapiro, J. (2014, February 17). Competency-based degrees: Coming soon to a campus
near you. The Chronicle of Higher Education. Retrieved from http://chronicle.
com/article/Competency-Based-Degrees-/144769/ Shedd, J. M. (2003). The
History of the Student Credit Hour. New Directions for Higher Education,
2003(122), 5–12. doi:10.1002/he.106

Siemens, G. (2012). MOOCs are really a platform. ELearnSpace Blog. Retrieved from

Siemens, G. (2014a, July 5). elearnspace › Activating Latent Knowledge Capacity.
Retrieved from

Siemens, G. (2014b, November 18). elearnspace › Digital Learning Research Network
(dLRN ). Retrieved from

Thomson, P., Saunders, J., & Foyster, J. (2001). Improving the validity of competencybased
assessment. National Centre for Vocational Education Research.
Retrieved from

Tucker, B. (2012). The Flipped Classroom. Education Next, 12(1), 82–83. Retrieved from

Wasserman, T. (2013, January 3). LinkedIn’s Endorsements Have Become Meaningless.
Retrieved December 20, 2014, from

Young, J. R. (2012, January 8). “Badges” earned online pose challenge to traditional
college diplomas. Chronicle of Higher Education. Retrieved from https://

Zhao, Y., & Breslow, L. (2013). Literature review on hybrid/blended learning. Retrieved

(Not inmcluding half a dozen Proquest results, no journal cited, can't access database)

(Also not including a large number of references from medical journals which had no apparent educational-based oversight)

Friday, April 24, 2015

Mark Surman on Open Eduction and the Open Internet

Article and photo by Stephen Downes

This is a summary of Mozilla CEO Mark Surman's talk at Open Education Global in Banff April 24 (today). It is a paraphrase with lots of direct quotation, but shouldn't be taken as word-for word literal. All errors are my own.

We need to help 5 billion people over the next 5-10 years become web literate.

Three quotes from great Canadian thinkers: "We are trying to do today's job with yesterday's tools and yesterday's concepts." "We drive into the future looking only into our rearview mirror."
- classrooms are organized around how monks talked.

The experience of living in a small town as the only punk rock kid shaped me. And we lived in the media culture hegemony, and also we lived in a time of very conservative politics with a daily fear of nuclear war. What punk rock showed me was that we could play a role in shaping the world we want. And I was a photocopier kid - a big part of punk culture was cutting things up and remixing them. Records, guitars, and a scene: this idea of our media, our ability to produce it, and a community. It's an ethos very different from the television world we grew up in.

The last 40 years has been technology that lets us reshape our world. When I got a tape recorder that I could record on, that was radical. These technologies and freedom inspire me. And I couldn't but help myself when the modem came along. And when Mosaic came out in 1994, I said that's what I want to work on.

Second Canadian: Harold Innis. "The Roman Empire and the city states were essentially products of writing." They could issue edicts and laws. How do we build the world we're trying to build? There's a connection between power and words, power and communication, and what we're trying to do is shift that, and make communication more open.

Mozilla: it says in our incorporation documents: "we exist to guard the open nature of the internet." Best job I ever had. That's what drove m to work on the Cape Town declaration. We said it can't just be OERs, it can't just be open content, it has to be learning, it has to be participation.

So I would argue that we have a common ethos around that idea. And I see Mozilla as being the David that can take on the Goliath with those ideas. And so we have won a number of battles, we have a lot to celebrate. Firefox itself is a big victory - we went from 98% Internet Explorer domination, and Microsoft was determining where the internet was heading. Firefox was a huge victory in shifting that. That was 10 years ago, we haven't won much lately. Reference to Sunday New York Times advertisement for Firefox 1.0 (I contributed to that: SD)

There is a shift, even in mainstream, toward seeing publishers as expensive and in the way. By contrast we have organizations like Lumen, David Wiley's company, getting traction and VC money. Similarly you've heard lots over the last few days, more and more public money has gone into ensuring that learning resources are open. For example, $2 billion for OERs in colleges.

Those victories don't just limit themselves to this room. We have those dollars to people who aren't having this conference explicitly. Eg. local tax grant in Missoula. We have people around the world coming to OERs and open learning, and doing real stuff. We see a bias toward action. Lots of victories, lots to be proud of.

We have won many battles... but we are losing the war.

We are losing the battle for openness, the open web, and in transforming education. These - Pearson - are the kind of people are going to win. They may shift from selling textbooks to capturing analytics and selling data, but they're still winning. Mozilla isn't anti-business but we're against oligopolies. I'm more afraid that this is going to be Pearson - 'Classroom'. As much as I use Google every day, it's increasingly a company that controls vast parts of the internet. India - Google is effectively a monopoly with Android in smart phones. But unlike Windows and IE, they control the OS, they control the money, they're taking over the carrier layer - this is a monopolist with an intent to take complete vertical control over our internet lives. That is losing the war.

How many think Uber is the good guy? We don't think of them as relevant, but it is likely the next big monopolists. Their goal and intent is to become the monopolist in the area of physical motion - to know everything about us, everything about the movers. That is then cloaked ina positive aspect of creating a new type of work.

"Millions of Facebook users don't even know they're using the internet." People don't even know what they're using. They don't really know what the affordances are in any of the most basic ways the way we know. There's a massive gap between the general purpose computers we have in our pockets and what people think they have.

We're seeing the growth of the empires that will shape humanity with a new set of values for probably the next few hundred years. The centre of that empire is pretty limited - it comes from Palo Alto, it comes from Silicon Valley. It's not that diverse a place. Its not the kind of empire I want to see. I don't want to see empire.

Fork in the road.

Do we want 'the next Steve Jobs' or do we want Edward Snowden. Do we want creativity and freedom, or control and a lack of agency. Are we going to choose openness, or are we going to choose the Matrix.

William Gibson, third (sort of) Canadian: "The future is here, it's just not evenly distributed." The future I have committed to is a future where everyone has the know-how to be internet citizens in full. That's where we want to go - how do we win the war?

Most of the people in the room used Mosaic, most were online before 2000 - the internet soon will be 5 billion people - that's where the battle for open will play out).

Three things we are doing:

First, web literacy. The challenge we have is to help 5 billion know how to wield that general purpose in computer in their pocket. We try to put it into Firefox, we try to put it into everything we ddo (cf Doug Belshaw's competency map). Participation, using the open web, is a bit part of this.

Second, we need to commit to learning and not just to open educational resources. That's what I took from my early work in Shuttleworth to what I'm doing now. The language we use to talk about our approach to pedagogy is: learn by making, make stuff that matters (that's a key idea OER brings to the table, we can work on real material that is stuff we need), do it together (social for us has to be a part of a radical open pedagogy).

Third, think of ourselves as bigger than just those of us around a single table, bigger than just this room - think of ourselves as people who want to take this open road (you are invited to Mozfest in November).

A movement, a different approach to learning (web literacy), can help us go down the open road if we do it ambitiously enough.

We've been doing this at Mozilla. Eg., the Maker Parties. We've had teach-ins,. to have people teach digital literacy to those around them. And this year we want to rally people to move litreracy on a massive scale - we don't know how to do that. Mozilla Academy? We will put whatever resources to bear on this, and help people do this. There are 300 organizations that make the Maker Parties happen - we want to do this together, get on the ball, and move it a lot faster.

This is important. We are at a Gutenberg moment. We are at an early phase in internet technology. What gets written today will determine the future.

Q. I'm struck by the fact that there are many Davids. How do you unite the Davids.

A. You have common cause though you have many approaches. 'Open' has been the rallying cry. But in that rallying we have become inward focused. The concept of 'open' isn't something that will get into the water necessarily. The key is to think practically, do things that will help people, rather than be evangelical. That may be a rallying point, but still around our ideas.

Q. Net neutrality - where the telcos are trying to determine what speed you will have and more. Steve Jobs was a master at creating beautiful golden cages. You cannot have OER and openness within a closed hardware environment.

A. Another hour-long talk. In general, in building this movement for openness, Mozilla very public takes a much more pragmatic approach on whether everything has to be free. Of course we all know all of the pieces we wish were there, they're not even not there is an way even open-advocates can live in an all-open world. Eg. should we be implementing the DRM standard in HTML 5. Of course we're against that. But if we don't implement it and the other three browsers do, then millions of our users won't be able to watch videos.

Which road do we choose, in order to remain relevant, and still keep a principled stance? Hardware and net neutrality are very important in that. Hardware is the biggest vector for network surveillance (I should have added Sczchen to the core of the new empire, on the hardware level). And it's a big question about how companies like Facebook play into net neutrality - Facebook is marketing itself in India as the free internet, don't bother with the rest of it.

Q. I can't help but think about Aaron Schwarz. Will civil disobedience become an appropriate response?

A. It already is. We don't hope what happened to Aaron will happen to others. But people like Anonymous - it's a tricky think to know what appropriate civil disobedience is. There may be reaal criminals in there. We don't all have the same agenda. Tricky questions.

Q. Would the internet be different if we had women making it?

A. Yes. And we need more of that. Mitchell Baker is a champion for women in technology and as leaders. But we're still very male-biased. We do need to have gender as an issue as we build, we're not as aggressive as we want to be yet, but it has to be a part of what we think.

Q. The web literacy is the closest thing to what I mentioned yesterday as digital citizenship. Who are the right people to engage on this?

A. We are thee stakeholders to first engage. Many great conversations here, eg., talking with Cable (Green) about getting a course on web literacy. And Cathy saying one way to do it is immersion. This is a good group of people to try to get some of those approaches into the mainstream.

There's a lit of other stakeholders we think about. The right part of business, for example, even some of the goliaths - eg., the phone companies, who have a set of interests counter to the core Silicon Valley values. Eg. they want people to make and consume local content.

Q. It's very common for us to conflate the web with the internet. To what degree is Mozilla interested in non-web parts of the internet.

A. As an activist, conflating the web with the internet is now a problem in my view. We think of the web as the human interaction layer, at least for now. The rest of the web isn't really usable by people. But increasingly not. We contrast the web with what's happening on the smart phone right now - the web is open, iOS and Android are much more bundled and controlled. But we have to pick our battled.

Q. Read-write-communicate has me thinking about openness - are you making the same pitch to other segments of the internet? Is it the same pitch?

A. he answer is, I'm about to. I'm trying to figure out a crisper pitch. This is spring training. I'm taking this to Quartz, and giving them the same pitch. The same in OE Africa in may. To see who we can bring along with us.

Q. I don't like your metaphor with the word 'battle' and the word 'war'. Cf. Hal Plotkin. He was entertaining us and also warning us with an example from the U.S. establishing a so-called 'free university' which failed because people became too militant.

A. Many people don't like those metaphors. I think we're too passive. Let's see if we can find a middle.

Q. Facebook is bringing free 'mobile internet' to people which is Facebook(+Google+Wikipedia)-only -

A. They're kind of BS. But they will be influential BS. Even at the board level, we talk about, do we play with or not? I've been into these sorts of discussions for years - the old Internet Advisory Council in Canada. It's companies saying "we will solve the problem of access." It's an exceptionally simplistic view. People will get access. The market will take care of access anyway. They want to be seen as on the forefront of solving that problem, and to capture customers while they're at it, with monopolistic strategies. But if we help where half the peopel only have Facebook, that's a bad outcome.

Wednesday, April 08, 2015

Johns Hopkins Academic Freedom Statement - An Analytical Representation

Drafted in January and just released, the Johns Hopkins statement on academic freedom will no doubt be widely cited.I cite the full text below.

This post is a version of the document designed to draw out and represent exactly what it says, and to examine the assumptions underlying the document.

Note: on detailed analysis, the document reads as though it were actually two separate documents forxced into a not-always-happy merger. My analysis treats the document as a whole, but maintains reference to the two parts as follows:

(*) Means the point was made separately in paragraph 5
(**) Means the point was made separately in paragraph paragraph 8

Academic Freedom - Analysis and Discussion


  • the liberty to speak and learn and invite others to do the same,
  • to create and pursue research, and
  • to participate, on and off campus, in public debate
  • they should be free to rebut or even condemn ... speech (*)
Questions: this definition is strictly limited to expression and research. Should matters of opinion and faith be included as part of academic freedom (the document references 'freedom of thought' but is vague on whether it should be explicitly protected)? What about assembly into classes, clubs, associations, and the like?  What about publication and distribution of research results? It is arguable that this is far too narrow an account of the freedoms protected in academic freedom.
  • not to obstruct, prevent, or punish (speech)(and research?)
  • Example: speech on academic, political, or cultural matters, even when deemed offensive to some, is not alone grounds for sanctions against any member of the university community
Questions: who does this force apply to? It is not clearly defined in the document. For example, does it apply only to the management and administration of an institution? Or to all members of the institution? If so, then what is its force with respect to a person not employed at the institution (eg., students, visitors)? Is it also intended to have impact on, and be respected by the wider community? Can a government, for example, be accused of violating academic freedom? A lot of thought has gone into the nature of the governance, but not nearly enough on who, and how, it governs.

  • promotes a diversity of views and perspectives, and   
  • necessarily tolerates the expression of views on a broad range of academic and political subjects that are thought by some to be wrong, distasteful, offensive or even hateful. 
Questions. There are numerous references to the protection of opinions that are thought to be offensive or hateful. But a far wider range of expressions could be said to be impacted by this policy. For example, does it apply equally to statements that are unpatriotic? Does it apply to expressions of political opinion, support for political parties? Would it protect an avowed belief in astrology and witchcraft? Does it protect climate change denialism, creationism, and other unscientific theses? Does it apply to calls for war or defenses of torture? There seems to be an over-emphasis on protecting hate speech, without an emphasis on protecting political, cultural and scientific speech.

  • to all faculty, students, and staff alike 
Questions: the only statement of application is to faculty, staff and students alike. Yet several statements in the document refer only to faculty and professors, thus creating the appearance, if not the reality, of a two-tier system. Additionally, it does not explicitly apply to other entities associated with the university, such as governing councils or boards, advisory committees, not does it apply to offices (such as the Registrar), societies and institutes within the institution. The document does not adequately reflect its applicability to the full membership of a university community.
  • Academic Freedom is the wellspring of a free and open university
  • the freedom of thought it protects is at the core of the search for truth, and its free expression lies at the very heart of our university mission,
  • a university must have breathing space for free and creative exploration and experimentation, and for the sifting and winnowing of the ideas that define its very purpose
  • A professional and respectful exchange of ideas is integral to creating a positive and professional environment for learning, teaching, and research (*)
  • On occasion, university officials, faculty, or students, may disagree with, and even be offended by, a statement or other expressive activity (*)
  • intellectual freedom and open inquiry is an important part of its history, and its legacy (**)
Questions: the statements made here go far beyond the statement and account of academic freedom. And yet they reflect a remarkably limited perspective. It is interesting that 'learning' does not appear until the fifth paragraph, and only as an aside, when presenting a justification of academic freedom. While there is perhaps no real reason to disagree with the (desired) attributes of a university that necessitate academic freedom, it may be relevant to list them here:
  • free and open
  • freedom of thought
  • search for truth
  • creative exploration and experimentation
  • sifting and winnowing of ideas
  • exchange of ideas
  • environment for learning, teaching and reserach
  • history and legacy
Are these all and only the properties of a university relevant to the establishment and maintenance of academic freedom?  Is it permissible as part of academic freedom to oppose the proposerties of a university enunciated as part of the justification of academic freedom?

  • the First Amendment to the U.S. Constitution
Question: is the basis for academic freedom really the U.S. constitution? Could there exist academic freedom in nations not governed by the U.S. constitution? The basis for academic freedom is not rooted in exceptional circumstances particular to the United States.

  • no right to defame or threaten
  • no right to deface or harass
  • no right to infringe on the privacy of others
  • no right to otherwise violate the law
  • reasonable (and) viewpoint neutral, restrictions on the time, place, and manner of expression (in order to) ensure the orderly functions of the university
  • no right to plagiarize or otherwise engage in academic or scientific dishonesty 
Questions: it appears as though this list of limitations is on the one hand too broad, and on the other hand too narrow. It is too broad in the sense that 'orderly functions of the university' may be very broadly, and disproportionately, defined. The definition of academic freedom should not be limited by 'reasonable' measures, only by extraordinary measures in extreme circumstances. Otherwise many manifestations of belief, such as political demonstrations, are exempt from academic freedom. It is too narrow in that it makes no mention of research and other ethics and standards. If academic freedom protects the freedom to research, it must define research ethics. Additionally, when the law requires or allows harassment, or the infringement of privacy, which prevails?

Additionally, and I understand that there is a cultural difference here, it would seem to me that academic freedom is no defense against racism, sexism, homophobia, and attacks of a strictly personal nature. These are forms of expression harmful to society as a whole, and a university cannot defend in its community the right to harm society. Many would also argue that the requirement of "a professional and respectful exchange of ideas" (see above) also prohibits the disparagement of culture, religion, background, appearance and language. You cannot on the one hand be "respectful" and on the other hand feel no restraint when being offensive to others. Academic freedom must not embrace a very narrow and (frankly) extremist view of 'freedom of expression' without regard to respect for others and impact on the wider community.
  • exercise of judgment on the basis of professional criteria and the highest intellectual standards:
    • in matters such as academic quality, and  
    • faculty and student performance evaluations
  • faculty / professors who express their personal views on controversial subjects in the classroom must make it clear that students may disagree with those views without penalty
  • when one is speaking on matters of public interest, it should be made clear that personal views do not represent those of the institution
  • the ... appropriate response to ... statements in an academic setting is objection, persuasion, and debate
  • nurturing that flame (of intellectual freedom and open inquiry) and passing it on (**)
Questions: these responsibilities (as suggested above) apply disproportionately to faculty and professors, and can be construed to give faculty and professors extraordinary rights over and above the academic freedom of staff and students. Perhaps this was intended. Nonetheless, 'professional criteria' and 'highest intellectual standards' are vague and could admit of wide interpretation, at the discretion of faculty and professors. The responsibility here should refer to some known, non-arbitrary, and neutral set of external standards not subject to malicious interpretation.

Additionally, it should be clear that the responsibilities listed here as being incumbent on faculty ought also apply to students; they should be enjoined not to sanction or punish each other as the result of the expression of opinion (this is an essential criterion for a free student press).

Additionally, there are many methods of persuasion that are presumably not sanctioned by academics, but which could be seen as allowed by this definition, for example, emotional or social pressure, boycotts and restraints of trade, physical force, ostracism and exclusion, and more. Presumably it is not the intent to explicitly allow these (or all of these) but the distinction is not properly drawn. What sort of non-rational forms of persuasion (strikes? boycotts?) are allowed, and which (torture?) are not? And on what basis? This document is unclear. The preference for rational forms of objection is clear, but the delinieation of permissible non-rational forms of objection is entirely absent.

Additionally, as noted above, there is no stated responsibility to adhere to any ethical or moral standard at all, including research ethics. Academic freedom must be exercised in an ethical manner.
  • (not limited by) contact with countries and cultures. and other institutions that do not share the same understanding of free speech and academic freedom principles. 
  • (not limited by) research, funding and other partnerships with external public and private entities
  • (not limited by) new roles and relationships with other organizations, many of which involve funding for university research and academic programs

Questions: this is probably the most important of the additions to traditional accounts of academic in recent years. I have employed the phrase 'not limited by' to stand for what was actually some very half-hearted language in the original document ("special care" is used twice, without any account of what "special care" entails). If academic freedom is a core value of the institution, it should not be allowed to be limited by engagement with other cultures, partners or funding agencies. This is especially the case regarding engagement with corporations and entities that profit by association with the institution.

I, personally, would go back to the drawing board, take a more ordered approach to the document, and try again.

Finally, and for the record, nobody simply grants you freedoms, academic or otherwise. Though not a contract, a freedom is a form of interaction between two parties, whether teacher and student, employer and employee, government and citizen. Each of these parties - and especially the weaker - must assert this freedom in order for it to exist. There are no natural freedoms, there are no contractual freedoms, there are only freedoms which live and breathe through everyday exercise to their full extent.

Johns Hopkins Academic Freedom Statement

Note: as the original document was released as an image file (!?) I took the liberty of subjecting it to OCR for presentation here; this may have resulted in some minor errors.

Academic Freedom is the wellspring of a free and open university. The freedom of thought it protects is at the core of the search for truth, and its free expression lies at the very heart of our university mission, Academic Freedom is the liberty to speak and learn and invite others to do the same, to create and pursue research; and to participate, on and off campus, in public debate, It promotes a diversity of views and perspectives, and necessarily tolerates the expression of views on a broad range of academic and political subjects that are thought by some to be wrong, distasteful, offensive or even hateful.

Although tenure may form its backbone, Academic Freedom extends to all faculty, students, and staff alike. A university must have breathing space for free and creative exploration and experimentation, and for the sifting and winnowing of the ideas that define its very purpose.

Like the First Amendment to the U.S. Constitution, on whose precepts academic freedom is based, however, Academic Freedom is not absolute. One does not have the right to defame or threaten, deface or harass, infringe on the privacy of others, or otherwise violate the law. Reasonable, viewpoint neutral, restrictions on the time, place, and manner of expression are legitimate ways to set the boundaries and ensure the orderly functions of the university.

Academic Freedom also entails academic responsibility. There is no protected right to plagiarize or otherwise engage in academic or scientific dishonesty. The exercise of judgment on the basis of professional criteria and the highest intellectual standards, in matters such as academic quality, and faculty and student performance evaluations, is both permissible and necessary. Faculty who express their personal views on controversial subjects in the classroom must make it clear that students may disagree with those views. When one is speaking on matters of public interest, it should be made clear that personal views do not represent those of the institution. Professors who express their personal views on a contested issue must make it clear that students may disagree with those views without penalty.

A professional and respectful exchange of ideas is integral to creating a positive and professional environment for learning, teaching, and research. On occasion, university officials, faculty, or students, may disagree with, and even be offended by, a statement or other expressive activity. They should be free to rebut or even condemn such speech, but not to obstruct, prevent, or punish it. Speech on academic, political, or cultural matters, for example, even when deemed offensive to some, is not alone grounds for sanctions against any member of the university community. The more appropriate response to such statements in an academic setting is objection, persuasion, and debate.

Johns Hopkins University is not a narrow enclave. Its mission, its influence, and its presence reach far beyond the traditional campus. This necessarily brings it into contact with countries and cultures. and other institutions that do not share the same understanding of free speech and academic freedom principles. In these situations, special care is required to maintain our standards.

Johns Hopkins continues to expand its connections to a range of research, funding and other partnerships with external public and private entities. It continues to develop new roles and relationships with other organizations, many of which involve funding for university research and academic programs. Some funding sources may seek to control data and research findings, or limit their dissemination. In response to such requests, special care must be taken to maintain the university's core principles of free and independent inquiry.

Johns Hopkins University was home to the very early development of the concept of Academic Freedom in the modern research university. The torch of intellectual freedom and open inquiry is an important part of its history, and its legacy. Each of us, in our time, as members of this community of scholars, bears a responsibility for nurturing that flame and passing it on. It is our heritage!

Saturday, April 04, 2015

My Viva

Patrick Dunleavy offers this list of ten typical questions that might be asked on your PhD oral exam. I always felt I would have aced my oral exam, but I never got to take it because my examiners did not want me to work on network theory.

So how would I have answered these questions? Dunleavy's post begs a response...

1. What are the most original (or value-added) parts of your thesis?

The semantics of distributed cognition.

In distributed cognition, there is no single location in memory where we might find an idea or concept. Rather, it is distributed across a set of connections between entities (a graph theorist might say it is constituted of a set of edges between nodes).

Like this:

We can say a few things about these distributed representations that are significant:
  • They are not representations at all - that is, they do not 'stand for' things or 'signify' things. This set of connections, for example, does not stand for the concept 'couch' nor the word 'couch'. 
  • They are not propositional - that is, they are not encoded in the form of a sentence, and we do not 'think' in words and sentences (there is therefore no 'encoding' that takes place when we communicate or perceive the world around us).
  • They are different for each person. No two activation networks are alike. Indeed, they are even different in the same person over time. There is no static constant that instantiates the concept 'couch' at all.
The best way to think of the web of connections is to think of it as being like the ripples that spread out when you throw a stone into a pond. The initial stimulus causes a cascade of interactions and water molecules bump against each other, a spreading wave of activation that follows the path of least resistance. The waves do not 'represent' anything, they are not 'about' the stone, and indeed, you cannot infer to the existence of the stone merely from the presence of the waves (although Kantian metaphysics is based on exactly that sort of inference).

To make life more complex, we only have one set of interconnected entities and connections between them, and exactly the same set of connections contains multiple concepts or ideas. So in addition to our thought of a couch (symbolized in red) we might have a thought of a dog (symbolized in green).

Like this:  

Again, these activations do not stand for anything; they are simply characteristic patterns of spreading activation that occur in the presence of a stimulus. It is typical for the green activations to overlap the red activations. This means that, in certain circumstances, the activation of 'dog' might, by association, cause the activation of 'couch', depending on the overlap of these and other associated concepts.

This is the basis for inferences. As Hume would say, "the far greatest part of our reasonings with all our actions and passions, can be derived from nothing but custom and habit." Our inferences from one thing to another, from cause to effect, from premise to conclusion, are based in repeated iterations of an associative logic, which is based on the mechanics of spreading activation.

There are different ways to talk about this. One way is to describe it mathematically, through the principles of network interactions and learning theories. Each individual entity has its own activation function, which determines how likely it is to be activated by incoming stimuli; each connection has its own weight, which determines how much signal it carries forward, and the creation and destruction of connections in a network, its plasticity, is determined by the learning theory, which derives these new connections from weights and activation functions.

Another way to talk about it (and the way I talk about it in my dissertation proposal) is to talk about it conceptually, by describing the relevant similarity between one concept and another. We might think of this as the degree of overlap between one concept and the next - a very loose statement of the idea would say that 'dog' and 'couch' have a similarity ranking of '3', based on the overlap depicted in the diagram.

It is this network semantics based on similarity that is probably my most significant contribution to the field. It follows the work of people like Amos Tversky in presenting a feature-based account of similarity, but analyzes this in terms of spreading activation in neural networks.

2. Which propositions or findings would you say are distinctively your own?


If you take the theory I've just outlined above seriously, you see that it would be inconsistent for me to say that everything is connected, and then for me to say that some proposition or finding is distinctively my own.

Indeed, I struggle with the idea that my thesis is based in propositions at all. This supposes an idea of a theory or model that is composed of a set of related propositions which are either all consistently and coherently maintained by some logical framework or all derived from a set of observations or measurements on the bases of some sort of inference calculus. And I doubt that either is the case.

At best what I offer is a perspective, a point of view, a set of outcomes as presented from the perspective of this entity given the experiences and observations obtained over a lifetime. But these are all influenced to a significant degree by interactions and communications with others.

My understanding of the word 'Paris', for example, carries with it the influence of every other occurrence of the word 'Paris' I have experienced in my lifetime. It can 'mean' nothing else. Indeed, if I were to say that the meaning of  the word 'Paris' were uniquely my own, there would be significant cause for concern, for either I would be asserting some sort of supernatural connection to Truth and Objects, or I would be asserting some sort of egotitocal priority of my own perspective above that of all others,

There is this totally false depiction of the PhD thesis, which is this:

which supposes that the individual researcher forges beyond the rest of knowledge on his or her own. But that's not how it works. We do all of our work entirely within the range of what might be called 'existing knowledge'. And when existing knowledge grows or changes, it does so on its own and not by virtue of one unique entity.

3. How do you think your work takes forward or develops the literature in this field?

Honestly, this question is just the same as the previous question, except that it refers to the localization of those propositions or findings.

The presumption in the question is that my new research builds on or extends previous research. It is a perspective or point of view that depicts knowledge as a mountain of propositions or facts, and it suggests that our PhD work is intended to extent that mountain (in an appropriately inferential of evidence-based way).

In my own case, if I had to characterize my contribution, it would be like this: same mountain, different view.

As I said in my answer to the previous question, there is nothing uniquely my own that has been added to human knowledge; I am working withing the same world, the same linguistic framework, the same logic and mathematics, the same sets of properties and qualities, as everyone else.

My contribution, if we must identify one, is that I see it differently from everyone else (of course, everyone sees it differently from everyone else).

One thing that I think is important is that I think that social knowledge and human knowledge are relevantly similar. Specifically, they share the same structure and the same logic. In both humans and societies, the structure of a concept or idea is the same: it is the set of connections between entities.

This general structure - what I called 'Learning Networks' and George Siemens called (more successfully) connectivism - calls into question even the concept of 'literature in the field', because knowledge is not divided between some privileged set of writings, 'literature', and everything else.  

At best, the 'literature' might be thought of as some stigmergic activity enabling each of us to contribute our own perspectives to a common object - a human intellectual anthill, of you will. But this thing that we build is distinct from the knowledge we have as a society, and does not hold any epistemic priority over other such network-based objects of knowledge, such as the 747 aircraft or World War II. It's a thing upon which each of us can reflect and obtain our own unique perspective take on collective knowledge.

So I am frankly not interested in developing 'the literature', except to offer the occasional contribution as a social gift, much the way I might contribute to Wikipedia or add to a cowpath in the grass by walking upon it. I don't think the objective of research or scientific enquiry is to develop the literature, or at least, not only to develop the literature. It is to engage, to contribute a life to society.

If I were forced to discuss there I think this is the greatest advance, I would probably say that it is the application of this thinking to education and pedagogy (keeping in mind that if it weren't me, it would surely have been someone).

Educational theory before connectivism is based almost completely on the idea that learning involves the recall of a set of propositions or facts, and is cumulative (much like knowledge in the literature) and stigmergic (analogous to a co-creation of knowledge). Dissuading ourselves of these propositions, and understanding that learning is network based, founded in the development of custom and habit, is the core of my work over the last ten years.

4. What are the ‘bottom line’ conclusions of your research? How innovative or valuable are they? What does your work tell us that we did not know before?

This is the same question again, but with a slightly different presupposition about the nature of enquiry. The presupposition actually takes two forms: first, as the 'bottom line' as a chain of inference or the conclusion of a logical argument; and second, as the 'bottom line' or net value (to society? to Bill Gates?) of the research.

Both presuppose a directionality to research. Both suppose that research works toward an outcome. And both, in their own way, focus on the utility or value of the research.

As we can no doubt infer what what I have discussed above, directionality is very much a matter of perspective. Sure, it is always possible to depict a progression or flow from one entity to the next to the next in a network. It is always possible to describe a series of activations, one after another, over a slice of time. But the idea that these lead anywhere is surely a matter of opinion.

So at best what this question is asking me to do is to imagine the perspective of some putative observer and to ask what my work looks like from their perspective (this is in fact the actual process I undertake when describing our research program in my current office).

For example, we could ask, "what is the value of knowing that learning is associative rather than propositional?" We can take the perspective of four distinct entities to draw out the implications:

- from the perspective of the individual student, it results in understanding that learning anything is based in a certain set of skills (which I call the 'critical literacies') formed around the idea that knowledge is not cumulative and constructed, but rather based in practice and reflection resulting in habitual recognition of relevant phenomena. In other words, it makes them better learners.

- from the perspective of the teacher, it results in the understanding that teaching does not consist in explaining or describing, because these depend on an already strongly developed association between the words and the concepts, but rather, that we c an at best show (ie., model and demonstrate) actual practice, and have them obtain direct experience and practice.

- from the perspective of the education technology provider, it results in the understanding that networking and interaction are essential components of learning, that new experiences must be based on past experience, which entails the development of personal and experiential learning environments.

- from the perspective of the employer seeking to address skills shortages it results in the shift from a formal class-based outcomes-based learning paradigm to an ongoing informal learning network, hands on support systems, and personal learning program.

These four might seem as large leaps from the answers given to the first and second questions, but they are not in fact so large - of course, depending on your starting point you might have to shift your thinking 180 degrees to get to this perspective, or it might click into place as immediately intuitive or rational.

5. Can you explain how you came to choose this topic for your doctorate What was it that first interested you about it? How did the research focus change over time?

Mostly by accident, by opting for what I thought was obvious, and by seeing opportunity.

The accidents are the vagaries of experience. Early exposure to science fiction in the local library stirred my imagination and led me to want to be a scientist; poverty led me (on the advice of my father) to investigate computers and technology; being overlooked for promotion led me to enrol in university to become a scientist; a full English section in my physics program led me to enrol in philosophy; my first course in philosophy exposed me to David Hume.

Why is this important? Because of my background in computers, I knew that logic is arbitrary, so I was sympathetic with Hume's scepticism. This led me over time to a district of cognitivist methods over time, and hence a Bachelor's thesis defending Hume's associationism and a Master's thesis questioning model-based semantics as arbitrary and unsound.

In my PhD years I worked on the idea of knowledge as based on relevant similarity, developing a logic of association, while at the same time creating huge conceptual maps of ideas, disciplines and a wall-sized history of philosophy. So I was ready with Francisco Varela discussed the essentials of network theory in a lecture at the University of Alberta, and between reading Rumelhart and McClelland and attending the Connectivism conference in Vancouver, had come to see that connectionism and associationism amount to essentially the same theory.

I came into the field of learning technology via distance education at Athabasca University. Given my background, it should be no surprise that I tried several things, including the creation of a Bulletin Board Service (BBS) and co-authoring of an academic MUD. At Assiniboine I created a website and online courses and eventually a learning management system. As I gained experience I found that network principles could be applied to learning technology. I explored the use of content syndication, developed the idea of learning networks, and with George Siemens created the first MOOC. These were all instances of connectionism applied to pedagogy.

My core academic interest lies in understanding knowledge and cognition -- the processes of learning, inference and discovery. Time and experience have refined my early thoughts on the matter, but I have always approached the subject from an empirical and scientific perspective. There are no magic symbol systems, there is no privileged access to nature. There's only experience and a very human - indeed, a very animal - way to comprehend it.

6. Why have you defined the final topic in the way you did? What were some of the difficulties you encountered and how did they influence how the topic was framed? What main problems or issues did you have in deciding what was in-scope and out-of-scope?

If I had to state in a sentence how I define my topic, it is in the previous paragraph: understanding knowledge and cognition through understanding the processes of learning, inference and discovery.

In other words, I am focused first on how we learn rather than what we learn. This is an epistemic choice; a cognitivist or rationalist approach will first describe what we know - "we know language, we know mathematics, we know who we are," etc. My view is that many of these knowledge claims are incorrect. We do not know universal truths, we do not have knowledge of ideal abstracts. We don't, I argue, because we can't.

As a consequence, through most of my career I have found myself in conflict with those who have very specific theories about what we know, and (therefore) how we know it. They depict these knowledge claims as givens and construct and derive theories of learning and pedagogy based on this.

For example, a common line of argument runs as follows: we understand scientific principles, therefore we have knowledge of abstract universals, therefore these must be codified in a physical symbol system, so learning is a process of acquiring and codifying these statements. This creates a view of knowledge and learning that is content-based and focused on the assimilation of a set of these statements by the most efficient means possible.

This is the dominant view, and the position I advocate meets opposition at each stage of the inference.

It results in the need to reframe knowledge and learning from the ground up. It becomes very difficult to decide that something is "out of scope" because each statement of an educational theory varies in importance and meaning depending on which of these perspectives you take. Even the idea of what constitutes a theory - and whether connectivism is one - depends on your perspective.

Throughout the last fifteen years or so I have assembled thousands of small items, hundreds of blog posts, and various talks and longer works. These do not lead from a basis of evidence to a single conclusion. Rather they are point by point observations on a welter of interconnected points, none of which is pure data, none of which is pure theory, all of which constitute an interconnected perspective on knowledge and learning that undermines, and advances an alternative to, the cognitivist view.

7. What are the core methods used in this thesis? Why did you choose this approach? In an ideal world, are there different techniques or other forms of data and evidence that you’d have liked to use?

As someone who wrote a presentation entitled Against Digital Research Methodologies I have to say I didn't use any core methods per se.It's not that I think that science is random. It's rather that, in the words of Paul Feyerabend, scientific method is whatever works.

I've given the talk several times and have tried to express this in different ways. I've talked about my work as being a process of discovery, in which I try things, explore things, and look for patterns that stand out, unexpected significance or meaning, patterns of change, and even observations and inferences.

In other cases I've talked about conduct by research and design (not, by this I do not mean "design-based research", properly so called). 

Maybe the best way to approach this question is to take on the question of data and evidence head on. Because the presupposition in a question like this is that the research methodology will either be inductive - that is, inference to a general principle or method based on evidence - or abductive - that is, inference to the best explanation of a body of data or evidence.

In my talks I argue that this depicts research as following a classical theory of science, one in which we express data as a set of "observation sentences" and derive from that a set of theoretical statements, which collectively form a model or representation of reality. Most people understand that in the end we cannot distinguish between observation statements and theoretical statements - this is why 'truth' is often defined as 'truth within a theory T'.

But they are less likely to agree with (or even understand) Quine's second proposition, that reductionismis a dogma. We don't in any way infer from evidence and data to generalization or method. Rather, the data show us only one of two things, either:
  1. that something exists, or
  2. that something is possible
Indeed, the bulk of my work takes the form: it can be done, because I've done it. My work, in other words, takes the form of modelling and demonstrating, of giving an example (nothing more) that others can use in their own thinking and their own reasoning.

It is often asked of me: if there are no universal principles or generalizations, then what are those statements that look like universal principles or generalizations? In response, I say that they are abstractions.

But then, continue my questioners, aren't abstractions themselves idealizations based on evidence? And my response is, no, abstractions (and therefore universals and generalizations) are not created by inference from a set of empirical data. They are created from subtractions from empirical data (sometimes even one piece of data).

There are many ways to create abstractions, but I'll illustrate just one: the elimination of extraneous data from two overlapping concepts. Consider the concepts of 'dog' and 'couch'., which I described above. If we keep the connections between those entities where the concepts overlap, and discard the rest, we have a new abstraction, which is whatever it is that dogs and couches have in common.

Like this:

How should we characterize this abstraction? This is where the characterization gets difficult. It might be the idea that dogs like to sleep on couches. It might the that both dogs and couches are things. What it means depends on everything else around it. The more we subtract, the less the overlap, the greater the range of possible things it could be.

8. What are the main sources or kinds of evidence? Are they strong enough in terms of their quantity and quality to sustain the conclusions that you draw? Do the data or information you consider appropriately measure or relate to the theoretical concepts, or underlying social or physical phenomena, that you are interested in?

This question doubles down on the idea that a thesis is created by assembling a body of data or evidence that supports a conclusion. Hence the thesis is evaluated by two criteria:
  1. the quality of the evidence (which speaks to the soundness of the thesis)
  2. the inference from evidence to conclusion (which speaks to the validity of the thesis)
I've dealt with inference above. But what are the criteria for good evidence?

Virtually all research that still uses this model will be based on some sort of statistical generalization; the days when we could reason inductively from evidence to conclusion are long since past, a relic of the world when Newton's theories held sway. Perhaps it is true, as Einstein said, that God does not play dice with the universe - but if so, then he is a mean poker player.

Evidence is based on two criteria: quantity and type. By quantity we are asking whether we have enough evidence to draw a statistical inference. There are some fairly well-established principles of probability that are at least as reliable as any other form of inference that will tell us that, for example, we cannot infer from 20 specific instances to a population of 7 billion instances.

But more significant is the question regarding the type of evidence, which is specifically focused around the question of whether the evidence is representative of the population as a whole. That's why you don't just ask your friends how they'll vote when you're predicting an election; chances are, your friends will vote like you do. It should also be why a class of 50 midwestern undergraduate psychology students should not be used as the basis for drawing conclusions about anything, but journals keep publishing the studies.

In my case, none of these matter, because I'm not generalizing (or, at least, I'm trying very very hard not to generalize).

In my case, the question is always: is this example an instance of the thing I'm talking about? If it is, we can say that it exists, and move on, trying to find associated phenomena. If it is not, then what was it that misled me about it? Either way, I learn.

In a world without universal (or even predefined, or even commonly understood) categories, answering questions like this is not a trivial matter. In some cases, it is sufficient to obtain agreement among a group of people that "this x is a y". In other cases, we have to take our time, be pedantic, and show that "this x is precisely a y". Most of the work involved unravelling confused, imprecise or inconsistent categorization. People assume that because we use the same words we mean the same things, while experience suggests that this is often not the case.

In my case, we consider the set of posts, papers and talks, etc., to be both the evidence and the conclusion (this is an instance of what I mean when I talk of 'direct perception'). There is no sense to dividing one set of my posts as the evidence and another as the conclusion. The same happens in the mind: the set of neural activations is at once the evidence and the conclusion. There is no thing over and above the set of neural activations that constitutes the 'thesis' being discovered or proven (or, to be more accurate, if there is, it exists only as an emergent phenomenon, and can only be recognized externally by a third party or observer).

So when we ask what the sources of the evidence are, and whether they sustain the conclusion, we are asking what is, in my mind, an incomprehensible question, or more accurately, one that embodies incorrect presuppositions about the nature of knowledge. Similarly, if one asks about the correctness of the evidence and the conclusion, we reach the same conclusion, that the question presupposes an incorrect epistemology.

I have developed and often talked about my response to this, which is what I call 'the semantic principle', and what is at best a set of methodological presuppositions based on some opinions about the most effective function of a network. So I ask whether in the entities and connections in my thesis are diverse, autonomous, open and interactive. There's a longer discussion to be had here (what does it mean to say a concept is autonomous, for example? what does it mean to say that the parts of a thesis are diverse?) But it gets, I think, to a deeper understanding of epistemic adequacy that a query about the soundness and validity of an argument.

9. How do your findings fit with or contradict the rest of the literature in this field? How do you explain the differences of findings, or estimation, or interpretation between your work and that of other authors?

I've discussed a lot of this in my discussion above. But I have as yet remained silent on the difference between my own approach and the major approach in learning theory, or perhaps I should say 'set' of approaches, under the heading of 'constructivism'.

If I had to put the matter in a phrase, I'd say that constructivism is cognitivist and connectivism is non-cognitivist. Of course some people will immediately respond that there are some varieties of constructivism that are non-cognitivist. I typically respond, only half in jest, that for any criticism of constructivism C, there is a version of constructivism not-C, for an indefinitely large set of criticisms {C1...Cn}.

In fact, though, theory-building, model-building and representation all had their foundation in philosophy and the sciences well before their appearance in education, and the emergence of constructivism in education is a not-surprising response to (for example) behaviourism and instructivism, just as they were responses to logical positivism in the sciences. Bas van Frassen, for example, offers a prototypical account of 'constructive empiricism' in the sciences. Other flavours of constructivism are found in things like Larry Laudan's 'Progress and its Problems'. Or Daniel Dennett's 'The Intentional Stance'.

They are at once responses to scientific realism, grounding science in logical and social structures (there's a strong strain of this as well in Kuhn), and at the same time are responses to the discredited idea of 'the observation statement', which was science's only response to the idealist and the rationalist. Scientific constructivism was a way to preserve rationality in science, without surrendering its basis in empirical grounds. Only these grounds would be served by proxy, through an actuive engagement with experience, a process of construction, the formulation of what Quine called 'analytic hypotheses', the presentation of tentative conclusions, models and representations which would be evaluated as a whole against experience, against reality.

It's a brilliant response, and I have no real quarrel with the overall approach. My major criticism is that while they jettisoned the 'positivism' of logical positivism, they kept the 'logical' part, and it was in the logical part that logical positivism actually foundered. Indeed, Quine should have called his paper Two Dogmas of Logicism, for that's what they were. Specifically:
  1. The analytic-synthetic distinction fails not because there are no observations, but because there are no observation statements
  2. The principle of reduction fails not because there's no empirical basis in fact, but because there are no logical principles of reduction
And this is where I find my difference with the constructivists. In (almost) all cases, they depict the creation of knowledge as one of construction, where we (intentionally) create models or representations, grounding them in a model or environment (literally: "making meaning"). Often this is depicted as a social activity (sometimes, it is depicted as only a social activity). As a stigmergic activity, as I discussed above, I can comprehend it. But not as a theory of learning.

And the reason it is not a theory of learning is simply this: in a person (in a network, etc) there is no third party to do the constructing.

The creation of a model or representation (or network or theory or whatever) that will be tested against experience (or reality, or a run computer simulations, or whatever)  is a representationalist theory which assumes a distinction between the model and whatever would act as evidence for that model (and, often, a set of methodologies and principles, such as 'logic' and 'language', for constructing that model).

But the network theory I've described, if taken seriously, entails the following:
  • the network is self-organizing; we do not 'create' sets of connections, these result naturally from input and from the characteristics of the entities and connections
  • the network does not 'represent' some external reality; 'evidence' and 'conclusion' are one and the same; the network itself is both the perceptual device and the inferring device
  • learning isn't about creating, it's about becoming
Someone recently said to me, "well that makes you a radical constructivist". Perhaps. But now the meaning of 'constructivist' has ceased to be anything that would be recognized by most constructivists. This is an outcome that could be predicted by my own theory, but not by most others.

10. What are the main implications or lessons of your research for the future development of work in this specific sub-field? Are there any wider implications for other parts of the discipline? Do you have ‘next step’ or follow-on research projects in mind?

I have several things in mind, though life may be too short for all of them:

  • I want to continue to develop in technology an instantiation of the concepts I describe in theory (understanding that there's no theory, etc., etc.). This is the basis for my work in MOOCs and my current work in personal learning environments, as instantiated in the LPSS Program. I want to see the various ways in which a learning network can grow and develop and help real people address real needs and make their lives better.
  • I want to draw together the various threads I've described in this post and offer a single coherent statement of connectivism as I see it. Related to this, but probably a separate work, I want to draw out and make clear the elements of 'critical literacy', which I believe constitute the foundations (if you will) of a new post-cognitivist theory of knowledge (the program above has funding but I have no funding for this work, so as a message to society at large, if you ever want to see this, consider some means of funding it).
  • I would like to see the principles of self-organizing knowledge applied to wider domains and to society at large, as (shall we say) a new understanding of democracy, one based not on power and control and collaboration and conformity, but one based on autonomy and diversity and cooperation and emergence. Society itself will have to do this; I can but point the way.
That's pretty much it, from an academic and professional perspective. But I also understand that the work is not possible in the confines of my own office working with texts and software. None of what I do today, nor indeed, have ever done, has been separate from the rest of my life and living. Each experience that I have, each society that I see, each new city and each new bike ride adds a nuance and a subtlety to my understanding of the world. It is a beautiful life and my greatest contribution to the future would be, I think, to continue living it.

That's my PhD oral exam. I'd like to say thank you on behalf of the group and ourselves and I hope we've passed the audition.