Reading creationism in the web archive

In recent years, anti-evolutionist thinking has attracted some attention in the news, mostly because of the role of some Christian free schools in teaching anti-evolutionist ideas alongside or in place of evolution. Anti-evolutionist ideas are however by no means new, and have been a durable minority view in some of the churches, picking up speed from the 1960s onwards. (Although the term ‘creationism’ is colloquially used to cover all the particular variants of this thinking, I use the more general term ‘anti-evolutionist’ here.)

It is not always easy to gauge the strength of the movement, but the archived UK web allows a new angle of view on the question. In theory, the web allows minority views to flourish in proportion with their intrinsic attractiveness and plausibility, no longer constrained by the high barriers to entry to traditional publishing. And in the absence of publicly available web usage statistics for the main sites, it is possible to analyse the structure of links to these sites as a proxy measure of attention (both positive and negative.)

Using the Host Link Graph dataset, available from the British Library, I extracted all the unique hosts that had been found linking to any one of four prominent anti-evolutionist sites at any point between 1996 and 2010. Then, using both the live web and of the Internet Archive’s interface at, I examined each host in order to categorise it, which I was able to do for 91% of the results. One immediate point to note is precisely how many “false” results there are. A large proportion of the hosts (34%) are categorised as Other, most of which were links associated with search engine and other directory-type sites, rather than from any host representing an autonomous actor in the field. Excluding these as well, the analysis of the remainder is shown below:


Of the remainder, 39% are the sites of individual congregations. A full analysis of these sites (39 in total) is yet to be done, but the majority are independent evangelical churches, with a handful of Baptist churches. They include very few indeed from Anglican, Roman Catholic or Methodist congregations. Given that at the time of writing the Evangelical Alliance has a membership of 3,500 individual congregations, the magnitude of these numbers suggests that anti-evolutionism is a minority view even amongst evangelical churches.

As might be expected, a significant proportion (17%) are other anti-evolutionist sites; a later post will explore the nature of this particular network. Interestingly, few inbound links are from secularist organisations, other than the British Centre for Science Education which exists to document (and counter) creationist ideas. Once data is available for the period after 2010, it may be that this interest grows as the schools controversy mounts. There are also very few links in from the mainstream media, which might also be expected to grow after 2010.

A complaint often heard from anti-evolutionists is that the scientific “establishment” does not engage with the critique of evolution which is being offered. That claim would seem to be confirmed here, as both the proportion and absolute number of inbound links from academic domains are also very small.

In sum, this data would suggest that between 1996 and 2010, British creationism was talking largely to itself, and was mostly ignored by academia, the media and most of the churches.

You can download the data, which is in the public domain, from here . Be sure to have plenty of hard disk space as, when unzipped, the data is more than 120GB. The data looks like this:

2010 | | | 20

which tells you that in 2010, the Internet Archive captured 20 individual resources (usually, although not always, “pages”) in the Church Times site that linked to the archbishop of Canterbury’s site.


(i) that a host “” held the same content as “”.

(ii) that the Internet Archive were no more likely to miss hosts that linked to these sites than ones that did not – ie., if there are gaps in what the Internet Archive found, there is no reason to suppose that they systematically skew this particular analysis.

(iii) that my sample of four target sites was reasonably representative of the movement as a whole. It is therefore possible that the profile of inbound links is very different for another hosts of the same type.

(iv) the analysis does not include cases where a site moved from one host to another during the time period. The host URLs used are those in current use, and so if another host linked to a previous host and that link was not subsequently updated, then that linkage will not be recorded in this data.

(iv) that the inconsistency in deduplication at the British Library noted here does not affect this analysis.

Review: Society and the Internet

Earlier this month I wrote again for the LSE Review of Books. Since the Review is admirably free in the reuse it will allow, I republish it here under a Creative Commons licence.

Society and the Internet: How Networks of Information and Communication are Changing our Lives.
Mark Graham and William H. Dutton (eds.)
Oxford University Press, 2014.

The word ‘revolution’ is at a discount when it comes to discussing the impact of the internet, but current reactions to what is undoubtedly far-reaching and permanent change fit a longer pattern. Societies in the midst of rapid technological change often perceive the change as both radical and unprecedented. Previous technological shifts in communication have before been greeted in the same way as the internet, being understood in terms of utopia and dystopia. For some, the internet is a new technology in the vanguard of the inexorable progress of such abstract nouns as Freedom and Democracy. It dissolves the power of old elites, putting the power to communicate, publish, mobilize and do business in the hands of any who should want it. For others, it provides dark corners in which criminality may flourish out of reach of traditional law enforcement. It undermines the business models of cherished institutions, saps our powers of concentration, and indeed threatens the alteration of our very brains in none-too-positive ways.

These two mutually contradictory narratives have one trait in common: a naïve technological determinism. Both stories radically overestimate the degree to which new technologies have inherent dynamics in single and obvious directions, and similarly underestimate the force of the social, economic and political contexts in which real human beings design, implement and use new applications to serve existing needs and desires. It is the great strength of this stimulating collection of essays that at every turn it brings such high-flown imaginings back to the bench of empirical research on the observable behaviours of people and the information systems they use. Given the rapidity of the changes under discussion – the commercialised internet is only now reaching the age of an undergraduate student, as it were, with social media still in junior school – this kind of very contemporary history meets sociology, geography, computer science and many other disciplines in a still fluid interdisciplinary space.

The volume is very much the product of the Oxford Internet Institute, with all but six of the thirty-one contributors being associated with the institute in some way. The twenty-three essays are arranged into five thematic sections: everyday life; information and culture, politics and governments; business, industry and economics; and internet regulation and governance. Whilst the grouping is convenient as an orientation to the reader, the effect of the book is best experienced as a whole, as several themes emerge again and again. In this review I examine just three of many such themes.

One such is the complex geographies of the web. Gillian Bolsover and colleagues examine the shifting geographic centre of gravity of internet use. The proportion of total users who were located in the United States fell from two thirds to one third in a decade, and the proportion in Asia grew from a tiny 5% to nearly half over the same period. Bolsover and colleagues find that this shift in numbers is accompanied by distinctive geographic variations in the uses that users make of their internet, and attitudes to its regulation. Reading this chapter in conjunction with that by Mark Graham would suggest that these patterns of use map only loosely onto patterns of knowledge production (the “digital division of labour” between nations). These patterns of production in turn relate only inexactly with patterns of representation of places online; the “data shadows” fall unevenly. That said, the Global South both produces a small proportion of the content online, and is itself underrepresented as the subject of that content.

Many businesses, and media businesses in particular, have found the last ten years a time of particular uncertainty about the impact of the internet on long-established ways of doing business. Economists will be interested in two chapters which seek to address some of these issues. Sung Wook Ji and David Waterman examine the recent history of media companies in the United States, and point out a steady fall in revenues, and a shift from a reliance on revenue from advertising, to direct payment by consumers. Greg Taylor’s valuable essay examines the ending of the traditional economic difficulty of scarcity of goods by the advent of an almost limitless abundance of content online. This has created a different theoretical problem to be understood: the scarcity of attention that consumers can pay to that content.

Perhaps the most coherent section in the book is that on government and politics. Several governments (mostly amongst those western nations that were the early adopters of the internet) have placed considerable hope on online delivery of government services, and on social media as new means of engagement with voters. At the same time, both the chapters by Margetts, Hale and Yasseri, and by Dubois and Dutton examine the uses made by individuals of electronic means to organise and to influence government independently of, and indeed in opposition to, the agenda of that government. Governments have often expected greater benefits and lower costs from e-government; and political activists have tended to lionise the role of the self-organising ‘Fifth Estate’ of networked individuals to which Dubois and Dutton point. These five chapters situate all these hopes firmly in empirical examination of the interaction of politics, culture and technology in specific contexts.

Individually, the essays in this volume are uniformly strong: lucid, cogent and concise, and accompanied with useful lists of further reading. As a whole, the volume prompts fertile reflections on the method and purpose of the new discipline of Internet Studies. The volume will be of great interest to readers in many disciplines and at all levels from undergraduate upwards.

Slow scholarship and fast blogging

Recently there has been some debate on whether academic blogging is good for you; part of a wider debate about the speed and pressure of contemporary academic life. (You can get a sense of the debate in the articles here and here.) Some of this is prompted by the widely-circulated Manifesto for Slow Scholarship, which points to the supposedly inevitable superficiality of academic interaction in social media channels. It calls for a return to a more leisurely, measured, considered mode of thinking and writing, that produces writing that is fully baked. (There are some sage comments on the manifesto at large on

This month marks the second birthday of this blog (at least in its current form and location). And so it seemed a good time to look back and see whether there is any evidence here of ‘fast scholarship’ which has been too fast. After an elapse of two years, are there posts here that in retrospect I might wish not to have published ? If so, there would be some justice in the charge.

In that two year period, I counted up some 74 individual posts. Some of these were reports of work that was appearing in print, or in other outlets, including extracts. Some were explicitly partial and forward looking, such as this post inviting comment on an abstract for a forthcoming conference. These have a natural shelf life.

Along with these, there are perhaps 45-50 posts which have the character of an essay: thinking that had not previously been published, and which were an expression of a reasonably settled view. How have these fared ?

Some which contained comment on live issues at a point in time have been overtaken by events and changing circumstances, such that they speak to issues that have either been settled, or have transmuted into something different. An example is this post on the Church of England and women bishops, and this, on disestablishment. Also in this category are various posts on the policy environment for Open Access in the humanities, in which policy statements from government and funders have come thick and fast. Others are in the character of reviews of fast-developing web services, such as Google Scholar Updates. That said, I think these remain reasonable and cogent points to have made at that time, and so I don’t intend to remove them.

But what of the others ? There are areas in which my thinking has deepened since the first time I posted about them. But (crucially) that growth in thought has not been away from the initial post, but deeper and wider in the same soil. This is indeed what  one would hope would happen – the act of first essaying something here is the stimulus to further thought.  And so, I don’t think there are any posts here which I now wish were not here, and not in the archived version in the UK Web Archive. From the evidence of this blog, at least, there is no contradiction between slow scholarship and fast blogging.



The ethics of search filtering and big data: who decides ?

[Reflecting on discussions at the recent UK Internet Policy Forum, this post argues that societies as moral communities need to take a greater share in the decision-making about controversial issues on the web, such as search filtering and the use of open data. It won’t do to expect tech companies and data collectors to settle questions of ethics.]

Last week I was part of the large and engaged audience at the UK Internet Policy Forum meeting, convened by Nominet. The theme was ‘the Open Internet and the Digital Economy’, and the sessions I attended were on filtering and archiving, and on the uses of Big Data. And the two were bound together by a common underlying theme.

That theme was the relative responsibilities of tech providers, end users and government (and regulators, and legislators) to solve difficult issues of principle: of what should (and should not) be available through search; and which data about persons should truly be regarded as personal, and how they should be used.

On search: last autumn there was a wave of public, and then political concern about the risk of child pornography being available via search engine results. Something Should Be Done, it was said. But the issue – child pornography – was so emotive, and legally so clear-cut, that important distinctions were not clearly articulated. The production and distribution of images of this kind would clearly be in contravention of the law, even if no-one were ever to view them. And a recurring theme during the day was that these cases were (relatively) straightforward – if someone shows up with a court order, search engines will remove that content from their results, for all users; so will the British Library remove archived versions of that content from the UK Legal Deposit Web Archive.

Monitor padlock
But there are several classes of other web content about which no court order could be obtained. Content may well directly or indirectly cause harm to those who view it. But because that chain of causation is so dependent on context and so individual, no parliament could legislate in advance to stop the harm occurring, and no algorithm could hope to predict that harm would be caused. I myself am not harmed by a site that provides instructions on how to take one’s own life; but others may well be. There is also another broad category of content which causes no immediate and directly attributable harm, but might in the longer term conduce to a change in behaviour (violent movies, for instance). There is also content which may well cause distress or offence (but not harm); on religious grounds, say. No search provider can be expected to intuit which elements of this content should be removed entirely from search, or suggest to end users as the kind of thing they might not want to see.

These decisions need to be taken at a higher level and in more general terms. It depends on the existence of the kind of moral consensus which was clearly visible at earlier times in British history, but which has become weakened if not entirely destroyed since the ‘permissive’ legislation of the Sixties. The system of theatre censorship was abolished in the UK in 1968 because it had become obvious that there was no public consensus that it was necessary or desirable. A similar story could be told about the decriminalisation of male homosexuality in 1967, or the reform of the law on blasphemy in 2008. As Dave Coplin of Microsoft put it, we need to decide collectively what kind of society we want; once we know that, we can legislate for it, and the technology will follow.

The second session revolved around the issue of big data and privacy. Much can be dealt with by getting the nature of informed consent correct, although it is hard to know what ‘informed’ means; difficult to imagine in advance all the possible uses that data might be used, in order both to put and to answer the question ‘Do you consent?’.

But once again, the issues are wider than this, and it isn’t enough to declare that privacy must come first, as if this settled the issue. As Gilad Rosner suggested, the notion of personal data is not stable over time, or consistent between cultures. The terms of use of each of the world’s web archives are different, because different cultures have privileged different types of data as being ‘private’ or ‘personal’ or ‘sensitive’. Some cultures focus more on data about one’s health, or sexuality, or physical location, or travel, or mobile phone usage, or shopping patterns, or trade union membership, or religious affiliation, or postal address, or voting record and political party membership, or disability. None of these categories is self-evidently more or less sensitive than any of the others, and – again – these are decisions that need to be determined by society at large.

Tech companies and data collectors have responsibilities – to be transparent about the data they do have, and to co-operate quickly with law enforcement. They also must be part of the public conversation about where all these lines should be drawn, because public debate will never spontaneously anticipate all the possible use cases which need to be taken into account. In this we need their help. But ultimately, the decisions about what we do and don’t want must rest with us, collectively.

Introducing Web Archives for Historians

WebArchivesforHistoriansIt was a great pleasure last week, after several months, to be able to unveil Web Archives for Historians, a joint project with the excellent Ian Milligan of the University of Waterloo.

The premise is simple. We’re looking to crowd-source a bibliography of research and writing by historians who use or think about the making or use of web archives. Here’s what the site has to say:

“We want to know about works written by historians covering topics such as: (a) reflections on the need for web preservation, and its current state in different countries and globally as a whole; (b) how historians could, should or should not use web archives; (c) examples of actual uses of web archives as primary sources..”

Ian and I had been struck by just how few historians we knew of who were beginning to use web archives as primary sources, and how little there has been written on the topic. We aimed to provide a resource for historians who are getting interested in the topic, to publicise their work and find that of others.

It can include formal research articles or book chapters, but also substantial blog posts and conference papers, which we think reflects the diverse ways in which this type of work is likely to be communicated.

So: please do submit a title, or view the bibliography to date (which is shared on a Creative Commons basis). You can also sign up to express a general interest in the area. These details won’t be shared publicly, but you might just occasionally hear by email of interesting developments as and when we hear of them.

You can also find the project on Twitter @HistWebArchives

In defence of pseudonymity

Pseudonymity has had a bad press recently. A moral consensus seems to have formed that there is a problem not merely with behaving badly online while not under your real name, but with adopting a pseudonym at all. Pseudonymous authors “hide” behind their noms de plume; they lack the courage of their convictions; they are in some sense cowardly.

I don’t want here to get into the powerful imperatives of self-preservation that make pseudonymous writing a necessity when resisting a tyrannical government. I’d like to explore the particular reason why I myself blog elsewhere, and tweet, under a pseudonym (which I am clearly not about to disclose here).

I write for a living, more or less. I publish academic works, and blog here and elsewhere on the areas in which I am either directly professionally concerned, or on those subjects in which I am expert enough to make some observations. As more and more historians start “doing history in public”, this hybrid model of communicating what we as scholars do will become more and more important. And, as more and more of the web is routinely archived by the Internet Archive or the UK Web Archive, all of that communication which might previously have happened in person or in conferences, will now persist in the digital record. And since all these various utterances are linked together in various ways, such as Google Authorship, it will become easier for readers to trawl back through them all, and to put them together.

Given this, it is increasingly difficult to keep open a space in which to express an opinion just as a citizen without it becoming part of a professional profile. There are many issues in contemporary life on which I have views, but without any particular expertise. I’ve written elsewhere on the reasons I just write, and some of those thoughts are clarified in my own mind by the discipline of putting them online. And so my “other blog” gives the space to work out those off-piste ideas without them becoming mixed with my more “professional” writing.

And (incidentally) this is why I am not a “public intellectual”. The concept, at least in the UK, seems to involve the bringing to bear of a general intelligence, honed in one field, to matters of more general interest. Stefan Collini and others have already pointed out the tensions in the role. I have no view on whether or not the specific professional reputation of (say) David Starkey in relation to Tudor England is compromised by taking part in The Moral Maze. My point is simply that maintaining a separation between professional and pseudonymous selves in public means the question does not arise.

Reading old news in the web archive, distantly

One of the defining moments of Rowan Williams’ time as archbishop of Canterbury was the public reaction to his lecture in February 2008 on the interaction between English family law and Islamic shari’a law. As well as focussing attention on real and persistent issues of the interaction of secular law and religious practice, it also prompted much comment on the place of the Church of England in public life, the role of the archbishop, and on Williams personally. I tried to record a sample of the discussion in an earlier post.

Of course, a great deal of the media firestorm happened online. I want to take the episode as an example of the types of analysis that the systematic archiving of the web now makes possible: a new kind of what Franco Moretti called ‘distant reading.’

The British Library holds a copy of the holdings of the Internet Archive for the .uk top level domain for the period 1996-2010. One of the secondary datasets that the Library has made available is the Host Link Graph. With this data, it’s possible to begin examining how different parts of the UK web space referred to others. Which hosts linked to others, and from when until when ?

This graph shows the total number of unique hosts that were found linking at least once to in each year.

Canterbury unique linking hosts - bar

My hypothesis was that there should be more unique hosts linking to the archbishop’s site after February 2008, which is by and large borne out. The figure for 2008 is nearly 50% higher than for the previous year, and nearly 25% higher than the previous peak in 2004. This would suggest that a significant number of hosts that had not previously linked to the Canterbury site did so in 2008, quite possibly in reaction to the shari’a story.

What I had not expected to see was the total number fall back to trend in 2009 and 2010. I had rather expected to see the absolute numbers rise in 2008 and then stay at similar levels – that is, to see the links persist. The drop suggests that either large numbers of sites were revised to remove links that were thought to be ‘ephemeral’ (that is to say, actively removed), or that there is a more general effect in that certain types of “news” content are not (in web archivist terms) self-archiving. [Update 02/07/2014 – see comment below ]

The next step is for me to look in detail at those domains that linked only once to Canterbury, in 2008, and to examine these questions in a more qualitative way. Here then is distant reading leading to close reading.

You can download the data, which is in the public domain, from here . Be sure to have plenty of hard disk space, as when unzipped the data is more than 120GB. The data looks like this:

2010 | | | 20

which tells you that in 2010, the Internet Archive captured 20 individual resources (usually, although not always, “pages”) in the Church Times site that linked to the archbishop’s site. My poor old laptop spent a whole night running through the dataset and extracting all the instances of the string “”.

Then I looked at the total numbers of unique hosts linking to the archbishop’s site in each year. In order to do so, I:

(i) stripped out those results which were outward links from a small number of captures of the archbishop’s site itself.

(ii) allowed for the occasions when the IA had captured the same host twice in a single year (which does not occur consistently from year to year.)

(iii) did not aggregate results for hosts that were part of a larger domain. This would have been easy to spot in the case of the larger media organisations such as the Guardian, which has multiple hosts (society,,, etc.) However, it is much harder to do reliably for all such cases without examining individual archived instances, which was not possible at this scale.


(i) that a host “” held the same content as “”.

(ii) that the Internet Archive were no more likely to miss hosts that linked to the Canterbury site than ones that did not – ie., if there are gaps in what the Internet Archive found, there is no reason to suppose that they systematically skew this particular analysis.