'All Models are Wrong; Some Models are Useful' by Martin Smith

All Models are Wrong; Some Models are Useful: the power of simulation in scientific research and teaching

As someone working in the sciences it would be easy to conceive of oneself, and also to be regarded by others, as a hard-headed, cold-hearted, logical positivist, with a pronounced lack of romance in their soul. In regard to my work at least, there is hopefully some truth in such a view, as otherwise I would be unlikely to perform well in my job. As scientists, our work must be firmly rooted in verifiable evidence and rests entirely on what we can observe, measure and substantiate, as opposed to the comparative freedom that might be ascribed to artists, who are at liberty to imagine anything they wish. My subject involves the analysis and interpretation of evidence obtained from human remains deriving mostly from archaeological settings. In most cases this involves preserved bones and teeth, although in some instances we are further fortunate in having the opportunity to learn from other body tissues as well in the form of mummified remains.

In undertaking research into the human past we are attempting to inquire about the nature of events that are often no longer directly observable. To be able to say anything about past events requires us to accept the ‘uniformitarian principle’. This is simply to say that we must start from the assumption that the natural laws and processes we can observe now, have always worked in the same way, and also that the patterns of material evidence left by current actions will mirror those that would have been produced by the same actions in the past. If this basic premise is correct (and we have no reason to think that it is not) this places us in a position where we can investigate the past through simulation -i.e. by undertaking experiments to establish whether the material signature of a given event or process is consistent with a particular form of evidence preserved in the archaeological record or at a scene of forensic investigation. In designing experiments that simulate a particular event or set of conditions, it is always the case that only a portion of relevant conditions will be somewhere close to correct. All simulations are flawed by nature as human design can only ever exert control over a limited range of aspects in ‘recreating’ a given event. It is therefore the case that all such experimentation entails acknowledgement of the shortcomings of models as inherently imperfect imitations that will always fall short of reality. At the same time good experimental design, focusing on the most important aspects under investigation can produce results that help address questions which often cannot be tested by other means. In this sense the commonly cited axiom that “all models are wrong, but some models are useful” underlines the extent to which one of the greatest strengths of the scientific approach is in fact its willingness to accept imprecision in the pursuit of accuracy.

Simulations as Research Tools:

Some work I have undertaken in recent years that involved the creation of simulations in order to answer research questions began with questions raised as to whether particular patterns of damage seen in preserved archaeological skulls were signs of past violence as opposed to having been produced by some more mundane cause, such as the respective bones being damaged accidentally when the excavators recovered them. This interest initially grew from examining the remains of Neolithic people, from the time of the first farmers dating from around 3,600 BC. Contrary to the view we had of this period a generation ago, it now seems, on the basis of careful study of human bones that this was a stressed and challenging time, when conflict between communities was frequent and where trespassing into the wrong territory could cost someone their life. In particular, I was interested to know whether elliptical holes penetrating some Neolithic skulls could in fact have been caused by stone weapons. This was resolved through shooting flint tipped arrows at the scapulae (shoulder blades) of cattle and pigs, with the results resembling the defects in the human skulls very closely.

Whilst these experiments were effective in answering this specific question, similar queries persisted regarding the effects of more powerful projectile weapons such as crossbows and pre-modern firearms in causing head injuries. These latter queries could not be answered by simply shooting at flat pieces of animal bone as the nature of the head as a dynamic, closed ‘sphere’ containing liquid and soft tissues means that such powerful impacts have more complex effects and produce patterns of injury to bone involving distinctive fracture patterns, rather than simple holes in the skull. An obvious solution might be to use skulls or rather heads (with skin, brains etc.) as experimental samples. But of course, such a choice would bring obvious ethical problems -I would certainly have no wish to use donated human cadavers for example, whilst other mammal species have heads that are very different from our own due to the large size of the human brain and so the results would be invalid. In looking for an alternative I came across a synthetic bone material made of brittle polyurethane plastic that was developed specifically as having similar properties to human bone for use in training orthopaedic surgeons. The same material was also available as hollow spheres, representing the cranium, which my students and I were able to make use of by filling the spheres with ballistic gelatine (constituted to the same density as human soft tissue) to mimic the contents of the skull. These simple and imperfect simulations in fact worked very well in reproducing the kinds of fracture seen in modern gunshot wounds, with various results that were specific to the crossbow and black powder musket we used in these investigations.

Other simulations I have conducted with my research students have involved similarly spectacular arrangements such as the re-creation of Roman style funeral pyres and accidental modern house fires. These latter have used whole deer carcasses (culled due to over-population and not killed for the experiments) as proxies for humans in order to investigate the extent to which patterns of burning in the recovered bones differed and so could indicate whether a body was burned deliberately as opposed to being that of an accidental fire victim.

A third kind of simulation I have been involved in over recent years is of a virtual nature. It was realised in the 19^th century that comparative measurements taken from offset photographs of the same object or landscape could be used to mathematically calculate the shape of the subject in the images. This application has undergone a great degree of development over the following centuries and has now matured as digital photogrammetry. This technique involves the comparison of multiple photographs taken at differing angles in order to generate a three-dimensional model of the object in view. The process involves a high degree of computer processing power but can produce impressive results, where complex objects can be captured as virtual replicas. The human skeleton presents a variety of complex three-dimensional forms which have always presented a problem in terms of recording. Two dimensional photographs and drawings have strengths of their own, but are also limited in terms of the information they can convey. The human skull in particular is a highly complex three-dimensional structure that is difficult to represent even using multiple views from differing angles. Moreover, photographic representations can suffer from distortions of perspective which are made worse when close up views are used to try and show greater detail. The option to create highly detailed virtual copies of parts of the skeleton offers exciting possibilities, both in terms of providing a new and more versatile form of permanent record for cases when such remains are later reburied, and also for being able to share such 3D ‘virtual’ simulations easily with others. Student projects I have supervised applying this technique have ranged in size from the documentation of mass graves (again simulated for research purposes), to the recreation of skulls to check measurements against the originals, to the detailed recording of sword cuts in the bones of a group of 10^th century Vikings, buried together in a former Roman quarry pit, apparently after a raiding party had fallen foul of the locals.

Simulations as Teaching Exercises:

The ultimate aim of teaching is to produce competent, versatile individuals with a range of skills that position them well to commence a career in their chosen field. Engaging in practical activities as part of a course of structured learning allows students to make connections and build aptitudes in ways that are seldom achievable simply through reading or being told how to do things. In this sense our subject is often at a disadvantage, as the events we wish our students to experience cannot simply be created or accessed to order. The excavation of human remains from archaeological sites is a frequent occurrence in itself, that can form a useful aspect of student training, but remains a type of evidence that cannot easily be predicted or planned in advance, nor can we guarantee a burial for every student to excavate. More complex arrangements of such remains as encountered in scenarios of natural disasters or the investigation of modern conflicts and genocide require further specialist skills to investigate. However, these latter present even greater challenges to an instructor wishing to provide realistic and relevant training experiences. An aspect common to both archaeological and forensic investigations is that these are unrepeatable exercises. A buried body or skeleton cannot be excavated twice, once bones or artefacts have been removed, the spatial relationships between them no longer exist, whilst a court will not permit a witness to go back and present the same evidence a second time. This presents a problem in that for such crucial exercises the notion of ‘learning on the job’ and ‘picking it up as one goes along’ is simply not an option.

As the above occurrences are impossible to predict or arrange in advance, an alternative is to create simulations of these phenomena, allowing students to experience them in a form that is both sufficiently realistic to be useful, but also safe, both for participants and the ‘evidence’ they are set to investigate. For our forensic archaeology students, who are studying to acquire the skills needed to locate and recover modern buried remains, we therefore spend time each winter digging ‘graves’ into which plastic, replica skeletons, normally used in anatomical teaching are placed, complete with clothes, personal items and objects that might assist with dating the ‘crime’. These arrangements serve to simulate shallow burials following a murder, with the students then working in teams first to locate the graves (months later when the vegetation has regrown) and then to excavate them systematically, producing drawings, photographs and written records at a standard submissible to a court. A further team exercise executed on a larger scale replicates the kind of mass graves that are a feature of the modern world in too many regions affected by conflict, genocide and human rights violations. Again, replica skeletons are buried, furnished with clothing and personal effects. Once placed and ready for the grave to be backfilled, the visual effect of this imitation arrangement is in fact very considerable. On one occasion the exercise was visited by a photographer who had worked for the United Nations in documenting mass graves relating to war crimes in the former Yugoslavia. He was quite struck by just how closely the simulation resembled real examples of such graves and commented that the greatest difference was simply that the simulation lacked the odours associated with the real thing. During this exercise the emphasis is on replicating the working circumstances for a team challenged with such a task, requiring considerable physical labour, leadership, specialised job roles (excavator, photographer, surveyor and others) and a high degree of collaborative team working. These recovery exercises are followed by an assessment of a different skill set in which students present the ‘evidence’ they have gathered in a simulated courtroom. The most striking feature of the latter for us as staff has always been the extent to which this scenario exerts a strong psychological effect on the participants, who often find the courtroom assessment to be a nerve wracking experience, despite being fully conscious that it is only a simulation. The situation is fictitious, yet the reactions are entirely real.

Spanning the Great Divide: seeing what isn’t there

Returning to the wider field of biological anthropology (the study of human populations over time), aspects of the subject that focus upon excavated human skeletons are constrained by a range of perennial challenges. Rather than examining single skeletons, we more frequently arrive at conclusions about past life through studying populations. Such investigations therefore principally focus on cemeteries rather than individuals. The remains of past people that are accessible to us are frequently broken and fragmented, and subject to a range of processes that have hindered their preservation in the ground. However, not only are individual skeletons often incomplete, but also the wider groupings in which they were buried are often only partially represented by the remains recovered through excavation. Such excavations can only access the bones of people who were buried in a given location rather than never buried or interred elsewhere. Nor can they retrieve bones that were simply not preserved by the local soils, those that have been destroyed by previous building or quarrying etc., or those that simply lie beyond the limits of the area being excavated. When the results of a cemetery excavation are published, the authors are therefore in fact presenting only a ‘sample, of a sample of a sample’ (etc.) from which conclusions are drawn regarding the wider population that lived in a given region and period. It is an inherent characteristic of the field that we strive to produce the fullest picture, with the greatest resolution that we can achieve from only a fraction of what was once present.

Photography is by definition a selective endeavour that presents the viewer with a sample of visual information, chosen from an infinitely wider set of possibilities. In this sense photographic art is always as much about what is not in the image as what is. Our eye, and by extension our mind is taken beyond the frame of the picture to conceive and ask questions about the wider setting in which the image was produced. In addition to being of explicit value in their own terms, photographic images therefore also hold wider implicit potential in presenting a window onto the wider world in which they were created. This effect is particularly powerful in the case of images of exotic or distant regions that we may never visit, but also with images from the past. Photographs from fifty or a hundred years ago take the viewer into a world which no longer exists, providing a framework onto which we can build a wider mental model of the setting in which the image was originally created. In this sense a photograph offers a sample of information from which a greater whole is then imagined. Sampling, modelling and simulation are intrinsic aspects of the process. In this regard photographic art and scientific inquiry can actually be seen to sit very closely together in conceptual space, rather than being at the opposite ends of a philosophical gulf. The concept underlying archaeological excavation of achieving ‘preservation by record’ also has strong parallels with approaches in art and especially photography which continually endeavour to capture fleeting and ephemeral aspects of the world which would otherwise be gone.

Whilst their overall objectives might bear more similarity than is obvious at first, the greatest differences between art and science arguably relate to their respective approaches to creating new insights. The scientific method is necessarily constrained by evidence, with a constant tension between imagining what is possible and demonstrating what is not. The commonly cited notion that science can never ‘prove’ anything, rather only having the power to disprove hypotheses, rests very much on the principle that doubt is healthy whilst casual acceptance is dangerous. By contrast, artists can be seen as enjoying an unfettered ability to create new realities, constrained only by imagination. An example from my own institution came when a colleague from another faculty (Media and Communications) brought his Creative Writing students to study in our human osteology laboratory. Here the lab was used to provide an alternative environment to a standard classroom, whilst some of the remains we care for were used to provide inspiration. Amongst the human remains we curate are 150 skeletons excavated from the cemetery of the former Royal Naval Hospital at Plymouth (in use from 1764 to 1806). On the one hand as anthropologists we subscribe entirely to the view that every skeleton represents the material vestiges of an individual human story. However, we are not at liberty to deviate from the evidence we can verify, as opposed to the students in this instance who were encouraged to start from the same proposition but to each imagine what the ‘story’ and experiences of one of the Plymouth sailors might have been. Although these approaches differ along both practical and theoretical lines, science and art are not without parallels in a broader sense as both are ultimately concerned with asking questions. In this sense, the default position of science being that ‘we do not know’ is in fact closely echoed by the creative arts, which continually seek to look at the world in new ways in order to reveal insights that were previously inaccessible rather than simply accepting the dogma of established views. In these senses I have been very privileged to work with Lieven Lefère who I regard very much as a colleague engaged in similar work to my own rather than as coming from the other side of a great divide.

Martin Smith, 2021