Reading old news in the web archive, distantly

One of the defining moments of Rowan Williams’ time as archbishop of Canterbury was the public reaction to his lecture in February 2008 on the interaction between English family law and Islamic shari’a law. As well as focussing attention on real and persistent issues of the interaction of secular law and religious practice, it also prompted much comment on the place of the Church of England in public life, the role of the archbishop, and on Williams personally. I tried to record a sample of the discussion in an earlier post.

Of course, a great deal of the media firestorm happened online. I want to take the episode as an example of the types of analysis that the systematic archiving of the web now makes possible: a new kind of what Franco Moretti called ‘distant reading.’

The British Library holds a copy of the holdings of the Internet Archive for the .uk top level domain for the period 1996-2010. One of the secondary datasets that the Library has made available is the Host Link Graph. With this data, it’s possible to begin examining how different parts of the UK web space referred to others. Which hosts linked to others, and from when until when ?

This graph shows the total number of unique hosts that were found linking at least once to archbishopofcanterbury.org in each year.

Canterbury unique linking hosts - bar

My hypothesis was that there should be more unique hosts linking to the archbishop’s site after February 2008, which is by and large borne out. The figure for 2008 is nearly 50% higher than for the previous year, and nearly 25% higher than the previous peak in 2004. This would suggest that a significant number of hosts that had not previously linked to the Canterbury site did so in 2008, quite possibly in reaction to the shari’a story.

What I had not expected to see was the total number fall back to trend in 2009 and 2010. I had rather expected to see the absolute numbers rise in 2008 and then stay at similar levels – that is, to see the links persist. The drop suggests that either large numbers of sites were revised to remove links that were thought to be ‘ephemeral’ (that is to say, actively removed), or that there is a more general effect in that certain types of “news” content are not (in web archivist terms) self-archiving.

The next step is for me to look in detail at those domains that linked only once to Canterbury, in 2008, and to examine these questions in a more qualitative way. Here then is distant reading leading to close reading.

Method
You can download the data, which is in the public domain, from here . Be sure to have plenty of hard disk space, as when unzipped the data is more than 120GB. The data looks like this:

2010 | churchtimes.co.uk | archbishopofcanterbury.org | 20

which tells you that in 2010, the Internet Archive captured 20 individual resources (usually, although not always, “pages”) in the Church Times site that linked to the archbishop’s site. My poor old laptop spent a whole night running through the dataset and extracting all the instances of the string “archbishopofcanterbury.org”.

Then I looked at the total numbers of unique hosts linking to the archbishop’s site in each year. In order to do so, I:

(i) stripped out those results which were outward links from a small number of captures of the archbishop’s site itself.

(ii) allowed for the occasions when the IA had captured the same host twice in a single year (which does not occur consistently from year to year.)

(iii) did not aggregate results for hosts that were part of a larger domain. This would have been easy to spot in the case of the larger media organisations such as the Guardian, which has multiple hosts (society,guardian.co.uk, education.guardian.co.uk, etc.) However, it is much harder to do reliably for all such cases without examining individual archived instances, which was not possible at this scale.

Assumptions

(i) that a host “abc.co.uk” held the same content as “www.abc.co.uk”.

(ii) that the Internet Archive were no more likely to miss hosts that linked to the Canterbury site than ones that did not – ie., if there are gaps in what the Internet Archive found, there is no reason to suppose that they systematically skew this particular analysis.

Where should the digital humanities live ?

Don’t get me wrong. The cluster of work that bears the label ‘digital humanities’ is important; very important. I’ve spent the last decade or so of my working life in the gap between historians and application developers, trying to make sure that digital tools get designed in the ways historians need them to be designed. Projects digitising books; collaborative editing platforms; institutional repositories; Open Access journal platforms; web archives: I’ve done a similar job, more or less well, in each case. As well as that, I was (and remain) founding co-convener of the Digital History seminar at the Institute of Historical Research, which looks to showcase finished historical scholarship that would have been impossible without the digital, broadly defined.

But there is a problem with how we understand the term, I think. I receive the term as signifying a community of practice, of scholars employing new technological means to achieve the same ends as they did before ‘the digital’. And as that community of practice grows, one would naturally expect a degree of self-consciousness within it as to the distinctiveness of what we’re all doing. This is inevitable, and almost certainly helpful, as new journals, conferences and online spaces appear to in which work can get published that might be too innovative for traditional channels to handle, and for discussions about method to take place safely.

My worry is over the institutional location of this activity. Several universities have spotted the potential of locating DH people together, and so there are several Schools or Faculties or Departments of Digital Humanities, all centres of real excellence, in universities in the UK and elsewhere. It’s an institutional means of nurturing something important, and it seems to work. My concern is with the long-term.

As in all large organisations, the internal structures of universities have their own force in determining the shape of the work that goes on within them. Structures shape cultures and cultures influence behaviours. It’s nobody’s doing, but the effect is real.

A department has a head, who usually sits at the same table as the head of History, or Philosophy; and funds run down these channels, and reporting lines back up. And my concern is that this Digital Humanities, this enterprise that starts to be treated (in institutional terms) as a discipline in its own right, could become a silo. The unintended consequence of creating a permanent space in which to foster the new approach is that Dr So-and-So in English, or Philosophy, can say “Oh, a digital approach, you say ? You want DH – they’re over in the Perkins Building.” Enterprising individuals and projects can and do bridge these gaps between departments; but the effect of the existence of the silo on the general consciousness has to be reckoned with, and mitigating the effect takes time and effort.

Put it this way. When Microsoft Word came within the reach of university budgets, no-one proposed that a Department of Word-Processed Humanities be set up – although word-processing was a technology that became ubiquitous in a short space of time, and had profound and widespread and general effects on a crucial element of academic practice – just like the digital humanities. And right now, there are not Schools of Social Humanities, to foster communities of practice in the most effective use of Twitter for dissemination and impact. Both these were disruptive technologies which were (and are) promoted across departments, faculties and whole institutions until they needed (or need) promoting no longer.

The end game for a Faculty of DH should be that the use of the tools becomes so integrated within Classics, French and Theology that it can be disbanded, having done its job. DH isn’t a discipline; it’s a cluster of new techniques that give rise to new questions; but they are still questions of History, or Philosophy, or Classics; and it is in those spaces that the integration needs eventually to take place.