Existing Web archives: an orientation

Web archives are fast becoming the fundamental source with which the history of the Web is written. Scholars coming to them for the first time are in need of some orientation, however, since those archives are brought into being by many different organisations for varying purposes and by different means. Their scope and structure also vary widely, as do the means of first locating and then using them.

My chapter in the new Sage Handbook of Web History aims to provide just that orientation.

It begins with a brief historical sketch of the development of Web archiving over the last 20 years, which I discussed at greater length here. It then moves on to outline the different means by which these archives are created, and what implications those differences have for how they must be interpreted. It outlines the varied kinds of collections in existence, and the different questions of method that this variety raises for scholars. Finally, it details the means by which scholars may first locate archived Web content, and (once located) how it may be used.

Along the way, it raises several points of necessary critical engagement for Web historians regarding the archived Web as a new class of primary source. Some of these issues have their analogues in print, manuscript or other sources; a scholar needs to understand who produced an object, whether it be a book, a manuscript, a painting or a PDF. But some of the issues presented here are peculiar to the archived Web, and must be thought through afresh.

The technologies that are used to create archived Web resources fundamentally shape those resources, and so understanding those technologies is a prerequisite to understanding the archive. Crucial also is an understanding of how the archive is structured: along national lines, by the institution or sector that created the content, by format or by a more general subject.

Finally, users must also understand something of the means by which they discover, search within, view and analyse archived objects, since those means are both relatively new and in a state of flux and development. That thinking will be greatly enabled by close collaboration between scholars and archivists: a partnership of mutual benefit which shows welcome early signs of growth.att

[See also, in the same volume, ‘Religion in Web history‘, my essaying of an agenda for the religious history of the Web.]

Religion in Web history

My chapter in the new Sage Handbook of Web History is now published. I summarise it here.

The literature on the phenomenon of religion in computer-mediated contexts is now very large, having built up over two decades. That literature is also produced both in, and in the spaces between, more than one discipline: Internet Studies, which concerns itself with the nature of the medium); the sociology of religion; and from scholars of religious studies concerned in particular with the relationship between religion and the media in general. The disciplinary labels vary between countries, but however it is named, little of this writing concerns itself directly with the kind of questions that most preoccupy historians.

This essay surveys the current state of Web history as it relates to religion, and falls into two halves.

Its first half attends to some debates of particular historical and methodological note with which the emerging history of religions on the Web may fruitfully be brought into conversation. These include debates concerning both the Web itself as a technological system, and religious responses to technological change in general.

It then sets out some points of contact between Web history and three key themes in contemporary religious history: secularisation; religious radicalism; and the place of religion in civic life and the law. It also argues for a fresh integration of the Web, and the archived Web in particular, with the study of offline religion, in pursuit of an ideal state in which the archived Web is merely one of many kinds of primary sources with which historians work.

The second half then takes a fourfold schema of different aspects of religions as they may be studied. The first of these is doctrine and religious knowledge: the symbols and forms of words that describe the divine, the world, the human person and their interrelations. Second are religious organisations and their representatives (clerical or lay). Third is religious practice: communal and solitary activities of prayer, worship and other rituals. Finally, the section on religions and the Other deals with the modes in which religious people and organisations encounter those outside: as potential proselytes, as discussion partners about wider social issues, and as antagonists. In each case, I identify the current state of research and set out elements of an agenda for future Web history research.

[See also, in the same volume, my introduction to existing Web archives.]

The silence of the archive. A review

[A review that appeared first in the LSE Review of Books.]

David Thomas, Simon Fowler and Valerie Johnson.
The Silence of the Archive.
Facet, 2017.

In the past two to three decades, the archival profession has been caught between two currents of cultural and technological change: simultaneous, largely unrelated, both apparently inexorable. Largely confined to the academy, but resonating beyond it, has been a radical scepticism about the stability of meaning in language resulting from the postmodern turn in historical thinking. Coupled with this epistemological scepticism has been a hermeneutic of suspicion of the power relations that are embedded in the creation, description and accessing of archival records. This has been bound up with the emergence of a wider politics of identity, and the assertion of the experience of marginalised groups as being equally worthy of documentation and study as those more ‘official’ voices that have traditionally dominated archives.

At much the same time, the transition from paper to digital in records management and archiving has presented the profession with challenges of exceptional scale and complexity, as laid out by David Thomas, former Director of Technology at the National Archives of the UK, in Chapter Three of this fascinating book. This transformation has fundamentally changed the ways in which live records are created and managed by organisations, with the significant added risk of mis-description as frontline staff are pressed into becoming their own archivists, and also of discontinuity in working IT systems such that data is lost or rendered uninterpretable. As these records pass to the archive, new and intractable challenges of scale come into play as archivists must select content for archiving and appraise it, presenting the difficulty of finding effective ways of describing these records and designing access systems that meet the needs of users.

For most working historians, much of the ferment of the discussion that these changes have prompted amongst archivists and theorists has been largely obscure; most of the literature that the authors (all three of them present or former TNA staff) synthesise here is to be found in the journals of the archival profession, into which historians rarely look. For those scholars whose only contact with archives is in the search room, this book will likely come as something of a revelation of just how far-ranging and radical some of that thought has been in the last ten years, and should be widely read for that reason alone. One might expect it also to find its way onto reading lists for introductory courses in the methods of archival research. It is therefore a matter for regret that the book, even in paperback form, is priced at a level that makes it unlikely that it will find its way into many private collections.

As a whole, the book has two major themes, one of which is acknowledged by both the title and the back cover, and another, equally if not more important, which is everywhere implied but rarely stated (to which this review returns below). Firstly, the theme of the title: the silence of the archive. The authors, along with Anne J. Gilliland in the foreword, identify an image that has formed in the public imaginary of the archive as a comprehensive repository of all known facts about the past. Scholars will differ on how potent and pervasive that image is, but the authors set out to show firstly that archives are neither comprehensive in this sense nor purely objective, even supposing such a state were possible.

Chapter One, by Simon Fowler, deals with ‘enforced silences’, whereby organisations conceal, amend or destroy records before they reach the archive, or where (as an unintended consequence of freedom of information legislation) records are never created as business is transacted informally. All manner of decisions are then made as the archivist selects which records should be preserved, appraises those records that are selected and removes material in the process, and then catalogues records in ways that bring certain aspects of a record to the fore while effectively silencing other voices. In addition, neither the transient quality of everyday life nor the lives of the majority of the population often come under the gaze of the state and so leave few traces (Chapter Six by Valerie Johnson is instructive on the ways in which marginalised communities may be intentionally brought into view and their stories documented as a result.)

Professional historians are of course accustomed to engaging critically with the ways in which their archival sources come into being, but they will benefit nonetheless from this wide-ranging survey of the particular issues. In several places, however, a strangely critical note is struck: a suppressed frustration with the users of archives and their apparent inability to understand the issues. In Chapter Two, ‘Inappropriate Expectations’, Fowler quotes the historian Nicholas Rodger on the distaste of staff of the Public Record Office when asked to provide subject indexes: to do so ‘would imply that the Office had a duty to provide something the public wanted, instead of the public having a duty to shift for itself and leave the archivists in peace’ (54).

While this whole book is a testament to how far those kinds of attitudes have been eclipsed, glimpses occasionally show through. Archivists, we are told on page 45, are familiar with being ‘bombarded’ with questions which cannot be answered, by users who ‘struggle to understand’ the issues (60). Johnson writes (after Lisa Jardine) of the ‘longing of historians and researchers to find that golden key which will unlock the secret they are investigating’, which in some cases leads to false assumptions about evidence that does not in fact exist and (at the extreme) to the sorts of conspiracy theorising, fictionalisation and fabrication that Thomas explores in Chapter Five. Whilst some researchers can and do cross this line, the experience of this reviewer, at least, is that such cases are rare, and are perhaps overstressed here. Most historians are able to control their longing. That said, archive users, for their part, have no doubt been guilty of failing to appreciate the role of the archivist as something more than a mere fetcher and carrier of files as Johnson notes (146): there is work to do perhaps on both sides of the relationship.

To a certain extent, the book is let down by its title and chapter headings, since the focus on what is not possible obscures a more hopeful and arguably more important thread which appears explicitly only on page 141. Johnson asks where the responsibility for the documentation of society lies, and answers: ‘it has been the implicit argument of this book that we are all responsible, whether as creators of records or professional curators of those papers, or as users, researchers, historians and informed citizens.’ At this point, this reviewer must declare an interest, as one working to facilitate precisely this better working between archives and the users of their digital services.

Nonetheless, The Silence of the Archive is throughout a call for a new relationship between archivists, the ‘archival subjects’ (those whose lives are documented) and those who use the archived record. Johnson writes of the process whereby those archival subjects are engaged in the process of creating the archive of their existence, thus becoming co-creators with the archivist (149-53). Thomas points out the acute need in a digital archive for close engagement with end users, both in the selection of material and in the design of the interfaces that make those records first discoverable and then usable (70-72). It is a shame, then, that this call for change – necessary and urgent – is somewhat muted here; indeed, in general, the authors have a tendency to quote and expound the work of others rather than elaborate an argument, and could have been bolder. However, it is a case that should be widely heard. Records managers, archivists, historians and other users of archives should read this timely and important book.

New article: Users, technologies, organisations – towards a cultural history of Web archiving

This article is now published, in Web 25. Histories from 25 Years of the World Wide Web, edited by Niels Brügger. It is published by Peter Lang, in hardback, paperback and ebook formats. A postprint version is available to download here.

From the Introduction:

If 2015 marked the elapse of 25 years since the birth of the web, 2016 marked the 20th anniversary of web archiving: of systematic attempts to preserve web content and make it accessible to scholars and the public. As such, the time is ripe to make an initial assessment of the history of the movement, and the patterns into which it has already fallen. This chapter represents the first attempt to document the subject at length. It concentrates on what might be termed the cultural history of the movement. It does not address the question of how web archiving has been carried out, but why, by whom, and on whose behalf.

Historians have for long known that, in order to interpret archival materials properly, it is first necessary to understand how that archive came into being. Why is a particular object to be found, and not another? What does the archive seek to document, and whose interests does it serve? The last very few years has seen a very welcome growth in interest in the archived web among scholars. However, that interest is not yet accompanied by the necessary familiarity with how the archived web came into being, and to be thus familiar is arguably even more important in this context than for traditional paper-based archives. Older distinctions with which historians are familiar — between published document, ‘grey literature’ and institutional records — have become blurred, as have those between personal and institutional publication. As a result, it has become less clear where the responsibility for preserving which types of content lies among the established institutions in the library and archives field. In addition, the archived web resource is unlike the live version from which it was derived in subtle and complex ways that do not apply to print publications or to manuscripts. If this chapter serves to orient users as to some of the questions they should be asking of their sources, and of the institutions that provide them, it will have achieved its aim.

It falls into the following sections: The Internet Archive / National libraries / The corporate record / Research-driven archiving / Activist archiving / Users and the future

Reflections on Web Archiving Week 2017

Once in a while, the unplanned turns out to be as good if not better than the planned. It had not been the intention that the annual Web Archiving Conference of the IIPC should be combined with the second conference of ReSAW (the Research Infrastructure for the Study of Archived Web Materials). However, they came together in London last week, with intriguing results.

One of the great pleasures of the event is the diversity of both speakers and delegates: the institutions represented by the IIPC were there in strength, but also present were the largest assemblage of researchers I have yet seen. These include not only people from computer science and related fields – a group that has been engaged in this space for a while – but also an enlarged contingent of scholars of media and communications and several of the humanities disciplines. At the Archives Unleashed datathon on Monday and Tuesday there was a particularly creative meeting of scholars, technologists and archivists – the crucial nexus of relationships for making successful tools and services. The whole week was marked (for me) by a refreshing openness to the perspectives of others, a frankness about difference, and a collegiality without hierarchy which (if it can be sustained) bodes very well for the future.

If I compare this discussions last week with those in this community perhaps three or four years ago, a number of differences stand out. As I’ve tried to show in my short history of Web archiving, direct engagement between archiving institutions and researchers came relatively late in that twenty year history, and even four years ago there was still a sense that researcher engagement was still only very exploratory. We now seem to have reached the stage where substantial attention is being paid to understanding the needs of users as a preliminary step to developing new tools and services (of which there were also many exciting examples). Here I’m bound to mention the research study that I (as Webster Research and Consulting) carried out for the Parliamentary Archives, which Chris Fryer and I presented, but I also have in mind papers on citation practice (Nyvang et al), the research data management issues involved (Zierau and Jurik), and what users need to know about the materials they use (ie. what to do about descriptive metadata), a theme taken up variously by Bingham, Dooley et al, Maemura et al. The variety of different use cases both discussed in the abstract and demonstrated in concrete reminded me of how varied the user base for web archives is (or could be) and how much we need as fine-grained an understanding of those different users as possible. As Ben Steinberg of Harvard noted ‘How we [ie. the providers of services] think archives should or could be used may not be as pertinent as we imagine…’

Another theme for researchers that surfaced several times at the first ReSAW conference in Aarhus two years ago was the need to understand the offline as context for the online. In Aarhus the particular point was about the need for oral history and for analysis of print and manuscript sources to understand how web materials make it online to begin with, and the theme was taken up last week by Federico Nanni and (in passing) by Gareth Millward and Richard Deswarte. There were also reminders here that a full history of the Web will need to take account of the history of computing more generally (Baker and Geiringer), the interaction between the Web proper and other content delivered online, notably social media (Castex, Schafer et al, Day Thomson), as well as the wider social and intellectual context in which the Web is embedded (Schroeder, and my own paper on the religious language of the Web) .

What of the future? Delegates who followed the same tracks as me may have come away with a sense of the diversity of analytic approaches to the study of the Web, and impressed with the depth at which scholars are now seeking to understand the methodological challenges they face. The aim, however, must be to build on this reflection to a point at which the Web archive becomes simply one type of scholarly source amongst many in the production of substantive scholarly insight on history, sociology or literature as Gareth Millward noted. I look forward to the day when I can go to mainstream historical conferences and hear contemporary history written using the archived Web.

There is also, I think, a challenge to the community at large in navigating a path through the diversity of new technical development and analytical need on display here, to decide which elements best serve users in particular situations, and so should brought forward and made part of ‘business as usual’ operations. Some will be incorporated by web archives themselves, others maintained by communities of interested scholars, others probably commercialised. The IIPC has a part to play here, while remembering that a significant part of this new thinking is taking place outside the membership. At least one person on Twitter thought a combined conference like this was worth repeating, and it would certainly be a way of developing the listening process between archives, users and developers that is required.

Finally: I celebrated the diversity of the conference when viewed in terms of professional background, but in another sense there is still much to do in terms of geography. I counted some 17 or 18 nationalities represented here, a joyous thing in a fragmenting world, but nonetheless overwhelmingly from Europe and north America. The archiving and study of the Web, a global medium, still remains dominated by certain countries.

My thanks are due to all those involved in organising such an excellent event: Jane Winters as host at the School of Advanced Study (University of London), and Olga Holownia of the IIPC and my former colleagues at the British Library which also contributed most significantly. It was my pleasure to be a part of both the IIPC and the ReSAW programme committee, and to hear such a fine set of papers.