Category Archives: Digital Film Historiography

Formalising Digital Formalism: An Interview with Adelheid Heftberger and Matthias Zeppelzauer about the Vienna Vertov Project (2015)

The following post is a pre-print version of an interview I made back in 2015 with film scholar and archivist Adelheid Heftberger (then Austrian Filmmuseum, now Bundesarchiv) and Matthias Zeppelzauer (then Vienna University of Technology, now St. Poelten University of Applied Sciences).

Because this interview is not easy to find and, I think, still topical in terms of understanding the developing relations between film archives, film studies and digital scholarship, I thought now was a good moment to make it available on my blog. The interview is preceded by a short introduction that discusses the interview’s aim and place in the research I was doing at the time (and continue to do).



In recent years, film and media historians have increasingly made use of quantitative approaches and computational techniques to analyze and visualize patterns in digitised archival sources. This development has given rise to new visual methods and forms of data visualisation – diagrams, graphs and interactive maps – which scholars deploy to produce evidence in their research. In the historiography of film style, methods such as Cinemetrics and ImageJ/ImagePlot are becoming increasingly prominent for visualizing filmic structure and narration.1 And in socio-economic cinema history, GIS technologies and digital cartography are opening new avenues for network analysis.2 These methods forge collaboration between fields that traditionally have remained separate from each other. In particular, it sees the fields of film archiving, film historiography, computer science, information science and artistic research intertwine to a still greater degree.

This relatively recent development invites reflection on digitisation’s consequences for the epistemology of film historical research, similar to discussions in the discipline of History where the emergence of computational techniques of analysis have received critical attention for decades.3 There is a need to attend to the processes of formalisation which underlie contemporary digital research methods as we still know little about them. In this regard it may be considered an obstacle that film and media theory has predominantly tended to regard digitisation as a ‘deauthorization’ of established notions of history which renders its sources increasingly manipulable and dynamic.4 Or, as media scholar Steve F. Anderson has critically remarked, digitisation tends to be perceived as resulting ‘in the loss, rather than the reconfiguration, of history’.5 Such perspectives have arguably neglected the multifarious processes through which scholars reimagine new methodological avenues for their research tradtions to historicize film and media objects with computational techniques.

To grapple with the challenge of understanding this development’s consequences for film historiography, my own research has focussed to a great extent on the socio-technical operations that underlie digital methods to elucidate how quantitative techniques and data visualizations produce evidence for film historians and reconfigure their traditions. To this end, my research includes theoretical perspectives from philosophy of history, science studies and media archaeology to study how historical concepts and techniques, both digital and analogue, amalgamate in the digital era to create new representational practices and scholarly research dispositifs.6 In this regard, I consider digital film history as the product of both poetic and scientific gestures or – as Michel de Certeau once labelled computational history – as a ‘science-fiction’ which combines historical narration and metaphors with technical practice.7 In doing so, I do not wish to (re)instate a normative view of film history nor of best practices. Rather, I hope to elucidate the epistemological underpinnings of digital film history’s methods to invite scholars to use and critically discuss them and develop new, quantitative as well as qualitative, uses of data visualizations to their own ends while being aware of the inherent contingencies of digitised archival sources and the techniques of visual analytics and visualization strategies.8

In the context of my research I have been particularly interested in the interdisciplinary collaboration and use of scientific visualization software within the Austrian research project Digital Formalism. Digital Formalism was a research project on filmmaker Dziga Vertov’s work and theory which ran from 2007-2010.9 It involved media scholars at the University of Vienna, archivists from the Austrian Filmmuseum and computer scientists from the Vienna University of Technology.10 Though not an official partner, the Cultural Analytics project of media theorist Lev Manovich’s Software Studies Initiative joined forces with the project’s participants in 2009 to create visualizations of Vertov films, based on data collected by the Digital Formalism research team.11

In the conversation below I asked Adelheid Heftberger – project participant in Digital Formalism and then film archivist at the Austrian Filmmuseum – about aspects of interdisciplinarity, scholarly tradition and choice of visualization formats in producing a new historical interpretation of Vertov’s works. Furthermore, Heftberger was joined by computer scientist Matthias Zeppelzauer to answer the first part’s questions on interdisciplinary collaboration. With the interview I hoped to yield a better understanding of the underlying, methodological procedures through which Digital Formalism was formalised focusing on the ways in which it used, imagined and assembled different media technologies to create a new, historical perspective on Dziga Vertov’s seminal filmmaking.

Digital Formalism’s Institutional Background and Interdisciplinary Collaboration

CGO: Could you tell me a bit about the background of the Digital Formalism project? What were the circumstances that made such an ambitious project on Dziga Vertov possible?

AH: The tradition of showing Dziga Vertov’s films at the Austrian Film Museum goes back to its founding years. The first screening was already in 1966.12 Since then, the dissemination of Vertov’s work has been of high priority on the Film Museum’s agenda, so far four of his films and a book about the collection have been published. These activities result from the scholarly engagement with the collection which was nurtured by Rosemarie Ziegler over the years in close collaboration with Vertov’s widow Elizaveta Svilova. When I came to the Film Museum around 2006, the process of building a database with the aim of publishing Vertov’s documents online, was already underway and soon after it became my first task there. That was around the time when the Digital Formalism projects had been accepted in 2007.

Long before that, Professor Klemens Gruber from the Theatre, Film and Media Department at the University of Vienna had been involved in the dissemination and analysis of Vertov’s oeuvre in Vienna. In 1996 he, together with the New York University, organised a symposium named „To the 100th anniversary of Dziga Vertov“, a book publication with the same title followed where rare documents were translated and published.13 Therefore, it is not an exaggeration to say that Vertov and Vienna have formed a very strong connection for some time already, and consequently it was not surprising that the idea of setting up a research project came up.

Moreover, Vertov is known for his highly formalized style of filmmaking and although one gets a sense of that whilst watching his films, it was tempting to see if automated video analysis could provide us with more insights into how he actually composed and structured his work. This is how the personal connection of Professor Gruber with Professor Christian Breiteneder from the Technical University and his team came about. The two departments, together with the Austrian Film Museum, developed this innovative research project which was eligible for funding from the WWTF – a funding body in Vienna for technical undertakings and interdisciplinary projects.

CGO: What was your role in the project and how did it relate to your background?

AH: My background is in Slavic (Russian) Studies. At that time I had just finished my MA thesis on Dziga Vertov, in which I had analysed three of his films discussing him as a documentary filmmaker and focusing on aspects of propaganda in his films. Through Thomas Tode, a German film scholar who was working at the Austrian Film Museum at that time, I was invited to help with the work on the collection, which I gladly did.

When the Digital Formalism project was accepted, it wasn’t clear first if my role would be at the Film Museum or at the Theater-, Film and Media Department. Personally, I was keen to be affiliated with the Museum because it suited me best at that time. I wanted to learn more – hands on, so to speak – about film material, prints and film handling, and it was clear that within the DF project the in-depth analysis of the analogue material would be central. At this point I was only just becoming an academic researcher and film archivist, so I was broadly interested in the whole field. I soon began on the annotation of the videos, whilst comparing the digital material with the original analogue material in order to determine aspects such as for instance the original reel splices. I also learned a lot about edge marks, film damages and traces of film printing on the films. A highlight in the project was certainly the restoration of Man With a Movie Camera (Soviet Union, 1929) in collaboration with Eye Filmmuseum and namely with Senior Curator Mark-Paul Meyer. We compared a vintage print on nitrate film stock with our print and later also the digitized versions of both prints.14

Apart from that, my contribution mainly consisted in annotating the films manually using the Anvil Software, in close collaboration with the colleagues from the Technical University. Furthermore I was involved in close readings of selected films, with the aim of correlating Vertov’s writing with the filmic realisation. Together with Michael Loebenstein and Georg Wasner from the Film Museum we published a DVD, which contained not only montage list comparisons and visualizations, but also a video essay, Vertov in Blum, about the re-use of Vertov’s footage in German director Viktor Blum’s work.15 The visualizations that I worked on after the project was finished are in a way a natural next step and after the annotation and the collection of the data, it was logical for me to arrange the results in a useful way. My book Kollision der Kader. Dziga Vertovs Filme, die Visualisierung ihrer Strukturen und die Digital Humanities (Edition text + kritik, 2016) aims to sum up this long process.

CGO: The project is quite unique as a digital humanities project in film studies. There are relatively few projects of this scale and complexity in film studies that make use of new visualization techniques to discern, among other things, relations between archival prints. Furthermore, it is special because it explored a collaborative work form involving researchers from diverse fields; computer scientists, film archivists and scholars. With this in mind, how would you describe the group’s collaboration? What challenges did you need to overcome to develop a shared understanding of archival, scholarly and computational issues?

AH: Nobody was talking about Digital Humanities then, at least not in Austria. It all transpired after the project was finished, when I started to talk to Yuri Tsivian about writing a PhD based on Digital Formalism.

MZ: We learned that when different scientific disciplines collaborate it is essential to define a common terminology, a common language. For computer scientists it is especially important to get precise definitions of film-specific terms and entities, such as for instance shot, take or scene, to enable the development of computer algorithms which can operate for example automatic shot and scene segmentation. Another challenge was to distinguish between syntactic attributes which can be defined in a formal correct way, for example shot, and semantic attributes which always exhibit a certain amount of fuzzyness and requires interpretation. By fuzzyness I mean that there is place for interpretation and the resulting analysis depends on the subject that performs it. An example is ”scene segmentation”. Asking three different people to segment a film into ”scenes” will usually result in three different segmentations, as there is no unique and formally complete definition of the term ”theme”. This makes the development of computer algorithms for such tasks difficult.

AH: I think Matthias decribed the challenges quite aptly. This is a very difficult topic altogether – how to annotate the film material in terms of material characteristics and maintain the specific place on the respective film reel. Everyone who has ever tried to do a proper sequence protocol knows how time consuming this work is and especially when one for example has to mark the single frame or specific place where an occurence has to be annotated, for example a stamp, a tear, a splice, a note from an editor etc. So, the computer scientists had to learn about specific, material attributes. But the process could also be frustrating, among the film archivists and the scholars especially in more fundamental aspects. The scholars for instance had to accept that there exist different versions of films, sometimes mutilated without knowing who made the changes, and that records in international archives are hard to find. It requires patience and stubbornness to dig into archives and collect, evaluate and bring together film material and related information.

One thing we all had to understand, was to respect different scholarly pratices. I think, generally speaking, film scholars and archivists probably take their time to discuss things and concepts, share experience and knowledge as a vital part of the process. Conversely, technicians are trained to quickly get aquainted with the issues at hand, define the task and get moving and work independently, meeting only at neuralgic points in the project.

CGO: But the project’s computer scientists seem to have developed a quite advanced understanding of the archival films’ material characteristics.16 In the articles coming out of the project they show awareness of philological, material aspects in line with the scholarly reflections emerging in the 1970s and 1980s.17 How did you merge your knowledge with their’s?

MZ: We inspected the material and had numerous discussions with our colleagues from the Austrian film museum about the material’s characteristics and the challenges they pose for automated analysis. From these activities we developed a comprehensive understanding of the specific film material. Vice versa, the film archivists developed a better understanding of how a computer ”sees” images and videos and how artifacts, such as instability and flicker, which a human viewer can compensate for are highly disturbing in automated film analysis. In this regard, the interdisciplinarity added an important value to the project and improved the mutual understanding of computer scientists and archivists.

CGO: Could one say that a specific group took the lead within the project to shape its direction and objectives?

MZ: At some point, interestingly, the project’s direction became influenced by the demands and requirements of the computer scientists much more than anticipated. The reason was that the computer scientists required precise and complete annotations of the films for quantitative evaluation of the algorithms, and these annotations did not exist. Subsequently, they were generated by the film scholars and the archivists in the project who provided the necessary background information and knowledge about the film material. This stimulated research in the computer science domain and led to a shift from qualitative analyses to quantitative evaluations. Especially the archivists recognized the great potential of the annotations for visualizations and for developing novel perspecives on the material.

AH: I came later to the project but immediately got interested in annotating the films. The technical side of it triggered my interest in combination with the opportunity to watch Vertov’s films very carefully and measure them. It helps to go through a film frame by frame. Consequently, I worked closely with the computer scientists, which made me realise that they had developed skills in formal film analysis as well, so it was mutually beneficial. I also think Matthias is spot on when pointing out how the annotation work triggered interest in producing visual representations. This was not initially defined as a task within the project.

CGO: What role did existing Vertov scholarship play in these annotations? It seems that the Vertov studies by Vlada Petric and Yuri Tsivian are prominent, in particular their formal analysis of his films were a strong source of inspiration. In an article by project participant Stefan Hahn one can see for example that Vlada Petric’s extensive segmentation and annotation of for instance camera movements in Vertov’s work, included in his monograph Constructivism in Film: The Man With The Movie Camera – A Cinematic Analysis (Cambridge University Press, 1987) was part of the annotation scheme devised for the digitised films.18 Could you elaborate on how Digital Formalism in general leaned upon existing scholarship in developing its analytical scope?

AH: Yes, that is right. For our first conference we invited Tsivian, who had met some project members at a conference in Siegen before, as a keynote speaker. Our scholarly input came very much from Slavic studies. With Barbara Wurm we had a promising, well connected young scholar on board. With me and her being Slavicists and knowing Russian, that helped a lot. Manovich, who joined very late, has also written about Vertov before. For the Slavicists there appears to be a natural, combined interest in the Avantgarde, Formalist movement and films and formalist film analysis. Quantitative film analysis is not far away then. My point is that Vertov’s way of filmmaking comes from a very formal approach – he had to organise his material, he didn’t have a lot in the beginning. Many others have pointed to that as well, Manovich for example. In this regard, we used Petric’s segmentation and studied an unpublished manuscript by him as well. Tsivian provided that and was thrilled that we could now measure Man With a Movie Camera more accurately than anyone else before.

As for film studies I am not sure and must confess not being a specialist in Film Theory. Famously Bordwell and Thompson have written about Neoformalism, but I think quantitative film analysis has more in common with other desciplines than film studies, like quantitative linguistics or statistics. A reference point in that regard might be Herbert Birett and the German movement he was part of in the 1970s where scholars got very interested in measuring shots and argued its usefulness for identifying and analysing films.

Imagining Visualizations and Digitality with Vertov

CGO: The famous essay ”WE. A Variant of a Manifesto” (1922) is foundational for Vertov’s and the kinoks documentary approach and conceptualization of rhythm and metrics in cinema. It is also central to Digital Formalism’s interpretation of the textual dynamics and elements of Vertov’s films – or what Vertov refers to as ‘phrases’.19 It appears crucial, as a guideline for your annotation of shot types and the visualization of their relations. Could you tell a bit about the place of this text, or Vertovian documentary theory more broadly, in Digital Formalism and the role it played in your annotations?

AH: Vertov’s film theory is not very concise, stringent or complete, but as you probably know, everyone points to this essay to explain it. Barbara Wurm pointed out that there tend to be a lot of misreadings of it. A reason for this is, that it initially contained a graphical representation of Vertov’s system, which is basically the only one we have. But the wide-spread translations were published without it. Furthermore, the translation’s wording is really very tricky, and we should also keep in mind that he wrote this at the beginning of his career, when developing his way of working with film. So, we talked a lot about this graph, looked at it, and tried to analyze it to understand Vertov’s filmmaking as a system of phrases. However, in terms of classification of sequences it does not give a lot of guidance concerning shot types for instance. His wording is still very ambiguous… you can’t really pinpoint it. Therefore, many of our categories were really defined pragmatically in the end, influenced by very simple, say, conventional film theory and film-formal vocabulary. Although we did try to connect it back to Vertov’s own words. I am not quite sure our interpretation stands but it was good that we tried.

CGO: Can you elaborate on why you think it doesn’t stand?

AH: I think there are several problems in the definition of phrase. We probably over-interpreted its meaning. I think Vertov just means episodes, a wording he also uses, and I am not quite sure that he has a typology of phrases really. We stopped following it quite early, because after doing it with one film and moving on to the next, we got the impression that Vertov tried to work differently with each film. So, even if he had a vocabulary of phrases, it would be very different. We did something very inspiring, you know, but I would not subscribe to it after working a lot with his films.

CGO: Within Digital Formalism you explored the annotations with different types of visualizations and analytics originating from different fields to show patterns in shot lengths and sequences. Yuri Tsivian’s Cinemetrics developed primarily for film scholars and used for visualizing Average Shot Lengths (ASL) and Median Shot Lengths (MSL) in statistical style analysis was for instance used to visualize the internal dynamics of cutting rates in Man with a Movie Camera‘s different reels/acts in various archival prints (see Fig. 1).20 But likewise, the scientific software MatLab was also explored as a way of showing the interplay between different types of segments, ImageJ as well as Lev Manovich’s development of the ImageJ software, ImagePlot, was used to work with the films as image sets and sequences without having to rely on reduced, statistical representations.21 To begin with Cinemetrics, what role did it play in Digital Formalism and what did you find to be its strenghts and limitations in representing Vertov’s formalism?



Fig. 1 Cinemetrics visualization of Man With a Movie Camera’s first reel.


AH: Yes, this is a very valid question. I almost completely forgot about Matlab, because it was so early on. We were of course eager to share our research and to include Yuri Tsivian because of his pioneering work in semiotics and interpretation of Vertov’s signs, also from before he became interested in quantitative analysis. There was overall agreement in the project that Cinemetrics was a great way to look at the films, and we started our collaboration by working together on Man With a Movie Camera.22

What the DF project could offer, for the first time, was a frame-accurate annotation of the rapidly edited film in Excel form. On the Cinemetrics website the previous, quite heroic attempts, are still saved, which were carried out by using the traditional timing method (basically watching the film in real time and pressing a button when a shot ends). Vertov sometimes cuts shots together which only consist of single frames, so how could a human possibly be accurate here? However, the Anvil software allowed exactly that to be done. Once we had uploaded our data into Cinemetrics, we were rewarded with a visual representation of the film’s structure. This really added something to the project, for instance it helped me a lot to understand at a glance, how the films were edited and if there were trends in the editing of shot lengths. Only by looking at the Cinemetrics graphs, I could compare the different films and see general trends. This is also something which I explain in greater detail in my book where I use Cinemetrics graphs as visual evidence to suggest that one film – Entuziazm (Simfonija Donbassa) (SU, 1930) – did not survive in its original form.

Another small study we carried out with Cinemetrics was a historical analysis of the original reel structures in Man With a Movie Camera. After the examination of the analogue material in the Austrian Film Museum and research on prints in other archives, we could determine the original structure of the film’s five reels. Again, the visual representation of the single reels allowed for a useful visual comparison of the different structure of each reel.

CGO: I would like to elaborate a bit on this latter point. I recall from reading a conversation between you and Tsivian that by analyzing the different reels with Cinemetrics, you could visualize and provide evidence for, what you phrase using a very scientific terminology, an ED-rule – ED meaning Event-Driven – in Man with a Movie Camera.23 Thus, the visualization illustrated how the pace of Vertov’s montage did not support the film as one coherent, narrative structure, but as independent reels with openings and closures and shifting tempi, which tried to capture the rhythm of the depicted events?

AH: This was perhaps a weak point in our discussion, I have to say, because here you could start to use the graphics for developing broader concepts about how some films are created and we did not pursue this further, which is a pity. I myself did not feel confident enough, because I had a lot of things to dive into myself. So, I would leave this for other people to consider. I have the feeling there should be done a lot more, looking at the structure to find or develop broader concepts for describing such film dynamics. You know, Vertov is just not a very representational case for story-driven cinema, so, you would have to have a much bigger sample to develop this.

CGO: For editing analysis, I see the clear relevance of Cinemetrics, but on the other hand I also thought the Matlab visualization was interesting because it is almost like a graphical notation (see Fig. 2), a musical score, which charts the different elements so that you can look further into them and discern relations between events and cutting rates. However, in the end, it seems the ImageJ/ImagePlot montage visualizations became the preferred visualization type within Digital Formalism, especially for the analysis of Dziga Vertov’s Odinnadcatyj (The Eleventh Year, USSR, 1928, see Fig. 3). They are the ones that circulate most widely, and which scholars associate with the project to illustrate the potential of data visualizations for film and media studies. Could you tell a bit about why you preferred this visualization type, or why – according to you – they are the ones most frequently associated with the project?


Skærmbillede 2020-03-17 kl. 14.24.33

Fig. 2 Matlab visualization created by Adelheid Heftberger showing the different types of ‘phrases’ in Vertov’s The Eleventh Year distributed on the y-axis and their temporal duration on the x-axis.


AH: I think you are familiar with what Manovich claims, that we should explore visualization without reduction and use the image information. In a way this is very tempting, and I think it makes sense with film. There is of course also the point that now we can do it, computer power was just not sufficient previously. We used abstract visualizations on the one hand, plots and graphs and then ImageJ. I think they can of course be complementary. For what I researched later in my PhD, concerning montage, I could analyze and draw conclusions on shot length distribution, using additional statistics that I have, which I think only makes sense with Cinemetrics.

With ImagePlot/ImageJ you really don’t do that. You arrange them in whatever way you want them, and play around with it. It is an explorative way of working and basically there are two ways. Either you can visualize it in a way that you would think would make sense, looking for patterns in, say close-ups, it could be anything. Or, you really visualize a feature you are more specifically interested in. With reduced visualization I think you really need to know what you are looking for. It doesn’t make sense really to fool around, the question has to be a lot more precise. I think you perhaps got the impression we were generally more interested in ImageJ/ImagePlot, because of the euphoria around it, that suddenly you could do it. It is perhaps also because it is a bit like a memory of the film, a condensed memory. If you know a film really well, have a structure in your head already, it reminds you what is happening in it and makes you want to explore it.



Fig. 3 Montage ImageJ visualization of frames – the second from each shot – from Dziga Vertov’s Odinnadcatyj (The Eleventh Year, USSR, 1928) organized in sequential order from left to right from the upper left corner.


What I would underline is that I certainly preferred the montage visualizations for looking at films as time-based structures rather than for content analysis. I want to see the time structure, discern relations and connections between frames. I know that Manovich probably sees it the other way around. Perhaps because in what he is doing now, exploring Big Data sets, using image per image, for example these Time magazines, or Instagram photos, it does not matter really in what order they are taken, there is not a similar sequential connection. I was also not really sure if it made sense to go into the content analsis he did for films. For black and white there is not so much you can do, you know if you analyze greyscales and arrange them according to variations, really you kind of subtract, you calculate the pixels, and you get an image where you see where the greyer areas are, and you have a kind of snapshot of the whole film. I always felt a bit uneasy about that, but maybe this has to be a lot more methodologically supported.

CGO: Well, thinking in terms of methodology one of the reasons the montage visualizations appear successful, as I perceive it, is because they seem to realize some perhaps tentative ideas about style analysis from the 1970s. In that period, Vlada Petric for instance wrote about how the ties between academia and film archives should be strengthened, stressing that scholars need access to editing tables to understand filmic structure more adequately and develop accurate profiles of directors’ works or films. They should develop a ‘visual/analytic’ history of film he wrote.24 In that regard, the montage visualization really allows you to grasp cinematic structure in a way that, although emanating from a different material setting, shares features with the cutting table’s regime of vision. In their organization they also resemble classic sequential, scientific cinematography’s breakdown of movement. You can literally grasp film structures and compare them, and also make philological observations about film prints. This, I think, is also what you do in your use of the montage visualizations in the DVD edition of Sestaja čast’ mira (A Sixth Part of the World, SU, 1926) and Odinnadcatyj (The Eleventh Year, US, 1928) – see for instance the ‘Blum affair’ explained in the release’s Rom-section – by colour coding them to say what footage appears several times in his films or in films by other directors.25 It allows you to discern philological relations that traditionally you could only do in the film archive and to circulate these insights with these visualizations.

AH: Yes, I think the philological function is obvious and that you characterize this very well.

CGO: My next question concerns how Digital Formalism also seems to historicize and place Dziga Vertov as a predecessor to digital forms of image appropriation and visualization. As Seth Feldman has pointed out, since the rediscovery of Vertov in the mid-1950s in the post-Stalinist era, there has existed a branch of Vertov scholarship in which contemporary media imaginaries are mapped onto Vertov’s film practice.26 For instance, Georges Sadoul’s translations and discussions of Vertov’s documentary theory was instrumental to the conceptualization of documentary cinema as ’cinéma vérité’ in the 1960s – a literal translation of the title to his kinopravda newsreel series. More recently, and arguably also more widely known today, Lev Manovich’s The Language of New Media (2001) develops its understanding of contemporary database culture on the basis of Vertov’s reuse and montage of his own and/or stock footage. In your project, you suggested that Vertov’s ideas on machinic vision forecasted visual, computer-generated analysis to motivate your visual analytics.27 How did Vertov’s theory guide your conception of your analytical intervention and understanding of digital techniques?

AH: First of all I should say, that probably a lot of what was written, also the article you refer to, appeared at a very early stage where we tried to define the ‘digital’ in Digital Formalism. It is funny to think back at it now, because none of the people behind that article knew a lot about Vertov at the time. Probably we should not go into that argumentation too much.

Personally, I am not quite comfortable with seeing Vertov as digital avant la lettre. It’s very tempting, but I think we should consider his context and remind ourselves that among his peers at that time there was a strong interest in film’s formal structures and seeing where it could go. You might be familiar with these charts he drew, this famous chart of zero and ones, often taken as an evidence of it and which spawned this reaction of ‘Oh my God, this is so digital, I can’t believe it!’ (see fig. 4). But if you look closer I think you need a different explanation for that. Vertov used formal methods to convey a message, which was the worst strategy at the time, it really was his end.28 That was one of the first things that attracted me about Vertov, his documentary practice, it’s so far away. It really says a lot about us now and our view on what film should be. Probably it was just being a filmmaker, and more importantly a found footage filmmaker that defined his work.



Fig. 4 A shot chart from Man with a Movie Camera suggesting Vertov as a pre-digital filmmaker. On the chart is written: “Excerpt from a montage phrase from Part 4 of the film Man with a Movie Camera”. Courtesy of the Vertov Collection, Austrian Film Museum, Vienna. Archival number V 80.


In our project we were primarily interested in Vertov’s way of working from a filmmaker’s point of view, a filmmaker making documentaries more contemporary than anything we see today. Yet, within the project, we did spend a lot of time, discussing how exactly or if Vertov was digital. This is something that drove the computer scientists crazy. You also have to keep in mind that the digital humanities was just not on at the time, so we kind of had to invent the digital!

Visual Evidence in Digital Film History – Between Science and Art

CGO: In Science and Technology Studies, processes of negotiating the visual properties of scientific evidence in research settings are often studied critically, as ’speech acts’ that expose the underlying assumptions of scholarship and reflect inherent contingencies of scientific evidence.29 To take that perspective for a moment, how did you experience the processes of choosing a visualization within Digital Formalism in relation to your research objectives and could you tell about, if relevant, the contradictions and discussions that emerged in this regard?

AH: I think this discussion is needed, especially now after we have created this. You can probably talk about why a certain visualization could help or not, or what you want to do with it. But in the project, including myself, we all accepted something as image evidence – recurrences in motif or cuts – which could be summarized in a visualization. The most difficult discussions were about abstract concepts, like rhythm, montage and political interpretations. I guess we all started from the same point, even coming from very different fields and having our differences we all trusted each other and could talk about things. But in the end this was mainly my own process. I could just explore and do what I felt I wanted to do and I myself started very much from bottom-up.

CGO: If we consider the digital humanities on a more general level, one could also argue, lending the words of Johanna Drucker, that the field’s visual forms of representation can be regarded as a ‘trojan horse’. It uses the natural sciences’ representational practices and may, in doing so, consequently introduce reductionist, scientist notions to the historian’s practice by atributing data visualizations a strong evidentiary role at the expense of complexity and ambiguity. To refer again to Manovich this also brings to mind a question raised in an introduction he has given to his introduction to Cultural Analytics which asks “What will happen when humanists start using interactive visualizations as a standard tool in their work, the way many scientists do already?”.30 Elsewhere, you have briefly touched upon this, observing that the ‘two cultures’ of the arts and sciences (referring to C.P. Snow’s 1959 Rede lecture ’The Two Cultures’) are coming together with the digital humanities, but that a lot remains to be done.31 Could you elaborate on this?

AH: I can see the criticism, I just don’t feel strongly about it. Maybe that is just the way I’ve always worked, to try and combine things or at least try different approaches at the same time. Although I can see that it is important and makes our research clearer and more comparable to stick to methods from our own field, I also think it perhaps reflects an undercurrent of institutional structures which need to be followed in order to receive jobs or grants. I don’t want to point fingers, that is just an observation which could also be wrong. I think as a way out we should probably develop convictions and not just waves, you know, science vs. humanities.

As for Johanna Drucker’s reasoning, I think she indeed has a valid point in reminding the humanities of the fact that we have a rich tradition in methods aside from positivist or scientist ones. We could discuss this with regard to data mining, visualization or even interface design, etcetera. For me, this means, that if we take Drucker’s criticism seriously the humanities might want to contribute to the design of interfaces or the navigation and search on websites by drawing on longstanding experience in knowledge design and the expression of complex semantic issues. Drucker foregrounds the interpretative subjective aspect of the humanities, so why not develop individual search entries? Or explore narratives or artistic approaches in this context? Again, I understand the criticism, but then I am curious to hear what could be the alternatives, to see suggestions for a combination of positivist and hermeneutical methods. This is an area still very much to be explored, especially in a theoretical sense.

CGO: If I then understand correctly, what you mean by bringing the ‘two cultures’ together is perhaps simply that these visualization tools can be embraced by humanities scholars without too many preconceived ideas about what they produce, and that we should not be afraid of being lured into the positivist corner?

AH: Yes, probably that is it. There is a very polemical line in which Manovich said that humanities spent so much time discussing what they can or shouldn’t do, and you just can do it, you know, it doesn’t take too much time in the end. I always try to encourage people to just try something and see. Visualization invites that, it is very intuitive I think.

However, while I tend to stress the explorative factor I would say that there is also much visualization with a question behind it though. When I try to produce something I really have to think of what I want to do, which question I want answered. So, in order to use the tool you also develop a theoretical approach. I think it shortens the process if you have to communicate it to someone else. If we focus what we thought we could do with these tools, which questions we really have, I think we would gain a lot more.


Albera, François and Maria Tortjada, ”The 1900 Episteme”, in François Albera and Maria Tortajada, Cinema Beyond Film. Media Epistemology in the Modern Era. Amsterdam: Amsterdam University Press, 2010.

Amann, K., and K. Knorr Cetina, ”The fixation of (visual) evidence”, in Michael Lynch and Steve Woolgar (eds.), Representation in Scientific Practice. Cambridge, MA: The MIT Press, 1990, 90.

(de) Certeau, Michel, ”History: Science and Fiction” in Michel de Certeau, transl. Brian Massumi, Heterologies. Discourse on the Other. Minneapolis: Minnesota University Press, 1986.

”Cultural Analytics”. See: Last accessed 27 September, 2015.

Feldman, Seth. ”Vertov after Manovich”, in Canadian Journal of Film Studies – Revue canadienne d’études cinématographiques, vol. 16, no. 1 (2007)

Gruber, Klemens (ed.), Maske und Kothurn, special issue ”Dziga Vertov zum 100. Geburtstag”, Vol 42, No. 1. Wien, Köln, Weimar: Böhlau Verlag, 1996.

Hahn, Stefan. ”Filmprotokoll Revisited. Ground Truth in Digital Formalism” in Maske und Kothurn, vol. 55, no. 3 (2009)

Heftberger, Adelheid, Michael Loebenstein and Georg Wasner, ”Auf Spurensuche im Archiv. Ein Arbeitsbericht”, in Maske und Kothurn, vol. 55, no. 3 (2009)

Heftberger, Adelheid, Yuri Tsivian and Matteo Lepore, ”Man with a Movie Camera (SU 1929) under the Lens of Cinemetrics”, in Maske und Kothurn, Vol. 55, No. 3 (2009)

Heftberger, Adelheid, ”Ask not what your web can do for you – ask what you can do for your web! Some speculations about film studies in the age of the digital humanities”:, Frames Cinema Journal, 1:1 (2012)

Heftberger, Adelheid. Kollision der Kader. Dziga Vertovs Filme, die Visualisierung ihrer Strukturen und die Digital Humanities. München: edition text + kritik, 2016.

Klenotic, Jeffrey. ”Putting Cinema History on the Map – Using GIS to Explore the Spatiality of Cinema”, in Richard Maltby, Daniel Biltereyst and Philippe Meers (eds.), New Explorations in Cinema History. Approaches and Case Studies. Chichester: Blackwell Publishing, 2011.

Kropf, Vera, Matthias Zeppelzauer, Stefan Hahn and Dalibor Mitrovic, ”First Steps Towards Digital Formalism: The Vienna Vertov Collection”, in Michael Ross, Manfred Grauer and Bernd Freisleben (eds.), Digital Tools in Media Studies. Analysis and Research – An Overview. Bielefeld: transcript Verlag, 2009.

Ladurie, Emmanuel Le Roy, trans. Ben and Siân Reynolds, The Territory of the Historian. Chicago: University of Chicago Press, 1979 [1973].

Lundemo, Trond, ”Towards a Technological History of Historiography?”, in Alberto Beltrame, Giuseppe Fidotta and Andrea Mariani (eds.), At the Borders of (Film) History. Temporality, Archaeology, Theories. Udine: Forum Editrice Universitaria Udinese SRL, 2015.

Petric, Vladimir. ”A Visual/Analytic History of the Silent Cinema (1895-1930)”, paper presented to the 30th Congress of the International Federation of Film Archives, May 25-27, 1974. The paper can be found online at:

Petric, Vladimir, Constructivism in Film: The Man With the Movie Camera – A Cinematic Analysis. Cambridge: Cambridge University Press, 1987.

Rosen, Philip. Change Mummified. Cinema, Historicity, Theory. Minneapolis: University of Minnesota Press, 2001.

Tsivian, Yuri, ”Cinemetrics, Part of the Humanities’ Cyberinfrastructure” in Michael Ross & Manfred Grauer and Bernd Freisleben (eds.), Digital Tools in Media Studies. Analysis and Research. An Overview. Bielefeld: Transcript Verlag, 2009.

Venturini, Simone. ”Il restauro cinemaografico, storia moderna”, in Il restauro cinematografico. Principi, teorie, metodi. Pasian di Prato: Campanotto Editore, 2006.

Zaharieva, Maia and Christian Breiteneder, ”Archive Film Comparison”, in International Journal of Multimedia Data Engineering and Management, 3:1 (2010)

Zeppelzauer, Matthias, Dalibor Mitrovic and Christian Breiteneder, ”Archive Film Material – A Novel Challenge for Automated Film Analysis”, in Frames Cinema Journal, 1:1 (2012)


1 See Yuri Tsivian, ”Cinemetrics, Part of the Humanities’ Cyberinfrastructure” in Michael Ross & Manfred Grauer and Bernd Freisleben (eds.), Digital Tools in Media Studies. Analysis and Research. An Overview. (Bielefeld: Transcript Verlag, 2009).

2 See Jeffrey Klenotic, ”Putting Cinema History on the Map – Using GIS to Explore the Spatiality of Cinema”, in Richard Maltby, Daniel Biltereyst and Philippe Meers (eds.), New Explorations in Cinema History. Approaches and Case Studies. (Chichester: Blackwell Publishing, 2011).

3 Consider for instance how prominent historian Emmanuel Le Roy Ladurie proposed to ‘learn to live with computers’ in the early 1970s at the height of structuralist historiography, foreseeing that historians were becoming ‘historio-metricians’ and that “tomorrow’s historian will have to be able to programme a computer in order to survive. See Emmanuel Le Roy Ladurie, trans. Ben and Siân Reynolds, The Territory of the Historian. (Chicago: University of Chicago Press, 1979 [1973]) 5-6.

4 Philip Rosen, Change Mummified. Cinema, Historicity, Theory. (Minneapolis: University of Minnesota Press, 2001) XVIII.

5 Steve F. Anderson. Technologies of History. Visual Media and the Eccentricity of the Past. (Hanover: Dartmouth College Press) 9.

6 François Albera and Maria Tortjada, ”The 1900 Episteme”, in François Albera and Maria Tortajada, Cinema Beyond Film. Media Epistemology in the Modern Era. (Amsterdam: Amsterdam University Press, 2010) 37-39.

7 Michel de Certeau, ”History: Science and Fiction” in Michel de Certeau, transl. Brian Massumi, Heterologies. Discourse on the Other. (Minneapolis: Minnesota University Press, 1986) 214.

8 For a more in-depth discussion of the latter aspect see Trond Lundemo, ”Towards a Technological History of Historiography?”, in Alberto Beltrame, Giuseppe Fidotta and Andrea Mariani (eds.), At the Borders of (Film) History. Temporality, Archaeology, Theories. (Udine: Forum Editirice Universitaria Udinese SRL, 2015) 149-156.

10 Ibid.

11 ″’Visualizing Vertov’ – new article by Lev Manovich with 33 visualizations for download”, see: Last accessed March 17, 2020.

13 See Klemens Gruber (ed.), Maske und Kothurn, special issue ”Dziga Vertov zum 100. Geburtstag”, Vol 42, No. 1. Wien, Köln, Weimar: Böhlau Verlag, 1996.

14 See Maia Zaharieva and Christian Breiteneder, ”Archive Film Comparison”, in International Journal of Multimedia Data Engineering and Management, 3/1 (2010) 41-56.

15 Michael Loebenstein, Adelheid Heftberger and Georg Wasner (DVD-Supervision), Sestaja cast’ mira / Odinnadcatyj (Vienna: Edition Filmmuseum, 2009).

16 See: Matthias Zeppelzauer, Dalibor Mitrovic and Christian Breiteneder, ”Archive Film Material – A Novel Challenge for Automated Film Analysis”, in Frames Cinema Journal, 1:1 (2012)

17 Simone Venturini, ”Il restauro cinemaografico, storia moderna”, in Il restauro cinematografico. Principi, teorie, metodi. (Pasian di Prato: Campanotto Editore, 2006) 24.

18 Stefan Hahn, ”Filmprotokoll Revisited. Ground Truth in Digital Formalism” in Maske und Kothurn, vol. 55, no. 3 (2009) 132.

19 ”My. Variant manifesta” [Мы, Вариант манифеста], Kino-fot [Кино-Фот] 1 (1922), pp 11-12.

20 For background information and introduction to Cinemetrics see:, last accessed July 4, 2016.

21 Adelheid Heftberger, Michael Loebenstein and Georg Wasner, ”Auf Spurensuche im Archiv. Ein Arbeitsbericht”, in Maske und Kothurn, vol. 55, no. 3 (2009) 146. See also:, and last accessed July 4, 2016.

22 See: Adelheid Heftberger, Yuri Tsivian and Matteo Lepore, ”Man with a Movie Camera (SU 1929) under the Lens of Cinemetrics”, in Maske und Kothurn, Vol. 55, No. 3 (2009) 61-80.

23 Ibid., 60.

24 Vladimir Petric, ”A Visual/Analytic History of the Silent Cinema (1895-1930)”, paper presented to the 30th Congress of the International Federation of Film Archives, May 25-27, 1974. The paper can be found online at:

25 Michael Loebenstein, Adelheid Heftberger and Georg Wasner, op.cit.

26 Seth Feldman, ”Vertov after Manovich”, in Canadian Journal of Film Studies – Revue canadienne d’études cinématographiques, vol. 16, no. 1 (2007) 40.

27 Vera Kropf, Matthias Zeppelzauer, Stefan Hahn and Dalibor Mitrovic, ”First Steps Towards Digital Formalism: The Vienna Vertov Collection”, in Michael Ross, Manfred Grauer and Bernd Freisleben (eds.), Digital Tools in Media Studies. Analysis and Research – An Overview. (Bielefeld: transcript Verlag, 2009).

28 For a more thorough discussion and historical context on this chart see Adelheid Heftberger, Kollision der Kader. Dziga Vertovs Filme, die Visualisierung ihrer Strukturen und die Digital Humanities. (München: edition text + kritik, 2016) 346.

29 K. Amann and K. Knorr Cetina, ”The fixation of (visual) evidence”, in Michael Lynch and Steve Woolgar (eds.), Representation in Scientific Practice. (Cambridge, MA: The MIT Press, 1990) 90.

30 ”Cultural Analytics”. See: Last accessed 27 September, 2015.

31 Adelheid Heftberger, ”Ask not what your web can do for you – ask what you can do for your web! Some speculations about film studies in the age of the digital humanities”:, Frames Cinema Journal, 1:1 (2012), see:

Introducing MIMEHIST: Annotating EYE’s Jean Desmet Collection

After successfully defending my dissertation in May, I have been busy working on and preparing a few new projects which I will be involved in, in the next couple of years and which are all very exciting. Now that a new academic year is beginning I thought it was about time I began telling a bit about them. The first project I will discuss here is the MIMEHIST project, which aims at developing a scholarly annotation environment for EYE Filmmuseum’s Jean Desmet Collection.

There is a very brief presentation of MIMEHIST on the website of CLARIAH (Common Lab Research Infrastructure for the Arts and Humanities). However, the description is very general and not very detailed which was also a reason for me to write a blog post. In this post I will describe how our project builds on earlier multimedia annotation projects in media studies, discuss some of its theoretical underpinnings and how these may nurture new ways of using annotation in film historical research. The focus in this post is mostly on theory, methodology and functionalities. The historical case studies of the project will be described in greater detail in another post.

CLARIAH and MIMEHIST: Annotating EYE’s Jean Desmet Collection

Towards the end of February I received a grant for a video annotation project which will last a bit more than a year – from April 2017 to June 2018. The project is called MIMEHIST: Annotating EYE’s Jean Desmet Collection and is a pilot project within the larger research program CLARIAH (Common Lab Research Infrastructure for the Arts and Humanities) which aims at developing a national research infrastructure for  humanistic disciplines and the arts in the Netherlands. Within this research program there was a call for pilot projects last year, which invited applications focusing on digitized collections and different historical research questions and case studies. The cases and collections will be embedded in the Media Suite developed by CLARIAH in order to advance digital research methods in media studies. It will be available for scholars in the Netherlands with a Dutch university login.

With help from Liliana Melgar, postdoctoral researcher at the University of Amsterdam, I wrote an application in the fall of 2016 focusing on developing an annotation environment for the Jean Desmet Collection of EYE Filmmuseum in Amsterdam. The project officially started in April and I am working on it together with Liliana Melgar who is in charge of defining the user requirements with me, as well as Willem Melder from the Netherlands Institute for Sound and Vision who works on data interoperability and Roeland Ordelman and Jaap Blom who are taking care of the software development and engineering within the project. After the project’s beginning we also had the great luck of being joined by Ivan Kisjes, who works as an archaeologist and programmer at the University of Amsterdam’s CREATE project, and who will be doing OCR work and text mining in our project.



Jean Conrad Ferdinand Theódore Desmet (1875-1956)


The Jean Desmet Collection, preserved at Amsterdam’s EYE Filmmuseum, contains the archives of film distributor and cinema owner Jean Desmet (1875-1956) who was active mostly in the early period of silent cinema and its transitional years. The collection consists of approximately 950 films produced between 1907 and 1916, a business archive containing more than 120.000 documents (these will be OCR’ed in our project), some 1050 posters and around 1500 photos. Parts of the collection were acquired by the Filmmuseum shortly after Desmet’s death in 1957 and then gradually expanded throughout the years with additional acquisitions. The Desmet Collection is unique because of its large amount of rare films from the transitional years of silent cinema, and because of the richness of its business archive holds extensive documentation of early film exhibition and distribution practices in the 1910s. These features contribute to its immense historical value which was one of the main reasons why it was inscribed on UNESCO’s Memory of the World Register in 2011.

Most of the material in the Collection has been digitized in different projects throughout the years. In the past few months I have worked on a report to outline exactly how the digitization work for each of the Collection’s subcollections was carried out which will provide a fundament for embedding it in the CLARIAH Media Suite. For more information on the Desmet Collection, EYE Filmmuseum has made a dossier on its website. And, for a more elaborate history of the Collection, film historian Ivo Blom has written a beautiful monograph on Desmet’s business activities and life which is available as an open access book from the Amsterdam University Press in the OAPEN Library.

Scholarly background: Audiovisualcy, multimedia editions and film segmentation as interpretation

Where do the methods and assumptions of MIMEHIST come from and why are they important for media historians? This is what I will try to answer in this section.

Readers of this blog will know that I have a great deal of enthusiasm for multimedia editions of archival film which facilitate analytical interaction with film – or video versions of films. These formats work with audiovisual media in media res by insist on a certain kind of audiovisualcy – as film scholar Catherine Grant has aptly called it – as a basis for scholarly interpretative processes – be they critical or historiographical. A fundamental key tenet in this kind of scholarship is that scholars, instead of only writing about film, cite, segment and re-edit films in different ways and to different ends as a foundation for their reflection and interpretation. There are several projects and theoretical writings which MIMEHIST finds inspiration in, in this regard. I would like to highlight some of them here.

In particular, I have been inspired by the Digital Formalism research project carried out in Vienna. This was a research project on Dziga Vertov’s work and theory which ran from 2007-2010 involving media scholars at the University of Vienna, archivists from the Austrian Filmmuseum and computer scientists from the Vienna University of Technology. Through extensive manual segmentation and data visualization the project explored Vertov’s montage style in relation to his theories on film montage and rhythm. Moreover, I have previously written enthusiastically about the Hyperkino DVD series which aimed at developing a historical-critical presentation format for annotated versions of Soviet classics or rarities.

Before these more recent projects, there have been several advanced efforts to conceive scholarly multimedia projects for film historiography in the past decades. In the 1990s, for instance, there was a rich experimentation with what was then referred to as hypermedia in CD-Rom formats in film studies, in the US in particular. Several groundbreaking projects were developed such as Lauren Rabinovitz’ The Rebecca Project (1995), the multimedia textbook The Virtual Screening Room developed at MIT by Henry Jenkins, Ben Singer, Ellen Draper and Janet Murray between 1992-1999, as well as Yuri Tsivian’s CD-Rom on pre-Soviet silent cinema Immaterial Bodies: Cultural Anatomy of Early Russian Cinema (2000). The latter was released by the University of Southern California’s Labyrinth project in its groundbreaking series of Cine-Discs edited by Marsha Kinder.


immaterial bodies

An original order form for Yuri Tsivian’s Cine-Disc Immaterial Bodies: A Cultural Anatomy of Early Russian Film (2000) which Marsha Kinder has uploaded to her great website.


Conceptually however, one may trace the urge to work directly on films as a historicizing working method even further back, for instance to the 1920s when Parisian cinephiles would be making compilation films of moments of particular cinematic beauty – or photogénie – to mark the end of a year spent at the cinema with a critical hit list. Or for instance – as film scholar Michael Witt has discussed with great insight – to Jean Mitry’s Film sur le montage (France, 1965), as well as to a great deal of compilation and found footage films.

There are also several interesting texts written in the 1970s, which tend to be considered hermeneutical antecedents in today’s digital humanities and discussions on audiovisualcy. In particular, Raymond Bellour’s classic article Le texte introuvable (1975) – in English The Unattainable Text – is often cited as almost prophetic in its heartfelt plea for a format or situation which allows film scholars to cite audiovisual fragments and to manipulate playback speed and direction as a way to interpret films through segmentation and scrutiny. Elsewhere in his writings on film analysis – to be precise in the article “To Segment/To Analyze (on Gigi)” – Bellour has also beautifully highlighted film segmentation as a process of interpretation in which one constantly finds new meanings and layers:

…segmentation is a mise-en-abîme, a ‘plumbing of depths’, a process that has no end theoretically – which does not mean that it has no meaning, in fact, that is its whole meaning.

In recent years, Bellour’s ideas to a large extent provided the fundament for the brilliant scholarly video annotation software Lignes de Temps, developed by the Pompidou Centre’s Institut de recherche et d’innovation. Unfortunately, this software is no longer being updated.



Screen cap from Lignes de Temps (here I was busy analyzing the fantastic concert scene with the Yardbirds in Antonioni’s Blow-Up (UK/I/US, 1966))


Another text which is less frequently cited today, but which I find immensely interesting, is an article written by film scholar and co-founder of the Harvard Film Archive Vlada Petric in 1974-1975, “From a Written Film History to a Visual Film History”, published in Cinema Journal in 1975. Different from Bellour’s hermeneutical approach to film analysis, Petric suggested a more scientistic, quantitative approach to especially the study of montage and style, as a way to create a more accurate empirical basis for film historiography. While the canon of films and historiographical assumptions underpinning Petric’s article may seem dated today, I think the article is hugely interesting in the way it articulates a special role for film archives and insists on an analytical situation in which scholars scrutinize film prints in archives to develop a “visual/analytical” approach to film analysis. As Petric wrote in his article:

…the appropriate methodology of film history cannot be attained in our time without the full cooperation of the film archives, which possess the prints and have access to technical facilities, without which it is impossible to grasp the cinematic structure of a film.

In many respects, these projects and theoretical texts underpin the MIMEHIST project’s development of annotation functionalities for media scholars, working with the example of EYE’s Jean Desmet Collection. It would undoubtedly sound a bit grand to claim that MIMEHIST will fulfill all the ambitions articulated by them. However, it is certainly a theoretical lineage which we feel indebted to and find inspiration in, in developing video annotation for historical analysis in new directions. Working from these theoretical coordinates we try to nurture a historical methodology relying on video annotation which we may qualify as more open-ended and dynamic, something we will also try to achieve by drawing on annotation practices in other disciplines.

Open-ended and dynamic historical representations in annotated film editions

How are we planning to achieve this and how will this differ from previous multimedia editions of (archival) films or annotation software/projects?

There are several aspects in which we could say that our project wishes to create a research situation which allows for doing audiovisual analysis reminiscent of the projects discussed above but which also builds on them by drawing on annotation software and practices in other disciplines.

First of all, we will facilitate and encourage segmentation and labelling of films as a basis for interpretation of, in particular, stylistic and formal aspects of films. The annotation environment resulting from MIMEHIST should ideally allow scholars to analyze digitized films from the Jean Desmet Collection – and audiovisual material more broadly – through a process of segmentation and labelling. It will allow scholars to engage in a process of coding through labelling, in a manner which we consider congruent with the “plumbing of depths” of film segmentation as suggested by Raymond Bellour; a subjective interpretation process which is not finite, but which may have different end points for different scholars because they observe and annotate different aspects of a film. By creating different layers and timelines in the CLARIAH Media Suite’s Segmenting Player scholars may categorize shot types, scenes and sequences following a personal coding scheme. Furthermore, scholars may also use such a coding procedure to get a better grasp on the structure of films, as suggested by Petric, in order to establish firm empirical evidence for their interpretation. MIMEHIST will not facilitate statistical functionalities however – for instance shot boundary detection – but may allow scholars to annotate and count film features as they like.



A first glimpse of the CLARIAH Media Suite Segmenting Player in its early development phase.


In order to develop this, MIMEHIST also looks beyond film theory, and finds inspiration in qualitative methodology and Grounded Theory approaches used for coding video or audio interviews in the social sciences, anthropology and ethnography. What scholars in these fields often do is to use video/audio editing software to tag and label interview bits and organize them into different categories and color-coded layers on a time-line, while linking them to related materials. This work lasts until the researcher feels the material is exhausted in a subjective – or collaborative – coding process. The coding procedures of qualitative methodology may be considered somewhat similar to the mise-en-abîme-feeling of film segmentation as described by Bellour because it is an analytical process without a logical end point. For MIMEHIST this is interesting because it allows us to draw productively on a lot of expertise and practical experiences on interpretation of audiovisual material from qualitative methodology in order to develop film and media annotation further.

Beyond labelling and tagging, it is also the idea that scholars should be able to present an annotated version of a film with MIMEHIST and synthesize the main findings of their work in an edited and staged version for peers as a last step in their historiographical operation. This may be reminiscent of the kind of presentation format which Hyperkino developed. In particular, it will rely on links made between films and related material in the Desmet Collection which researchers can organize in a personal user space. Furthermore, the presentation format of MIMEHIST should ideally offer a kind of video playback mode in which annotations – or footnotes as they were called in Hyperkino – may be accessed by clicking on an icon. It will also differ in several fundamental aspects from a project such as Hyperkino. First of all, MIMEHIST will allow scholars to make different versions of the same film. Scholars may thus produce different – perhaps competing – interpretations of the same title. This should allow for a greater multiplicity of viewpoints to be expressed about the films in the Desmet Collection from various theoretical angles and will also, hopefully, nurture a more open-ended and dynamic historical interpretation of the films as well as representations of them. Whereas Hyperkino relied on the closed DVD-format to present one interpretation, MIMEHIST will not have the limitation of such a format. Furthermore, we also aim to make the scholarly annotation work more transparent. For instance, in addition to making an edited, annotated version available to other scholars, a researcher may also offer access to the work folder and coding underlying a version so as to allow other scholars to critically scrutinize exactly what they did or to engage in collaborative annotation.

Hopefully this will all work out. These functionalities are now in the development phase and what I have discussed above is the goal we are working towards. I will write more about the case studies which the project will focus on later. But as it looks now, I have lined up two types of cases: one case, which will focus on non-fiction films in the Desmet Collection, with particular attention to the representation of war during WWI and the films’ distribution in the Netherlands in this period. A second case study will focus on an incomplete film, to analyse how its form and distribution life may possibly be better understood by linking it to film-related material in the Desmet Collection. More about that later.



Bellour, Raymond, ed. Constance Penley. The analysis of film. Bloomington: Indiana University Press, 2000.

Gauthier, Christophe. La passion du Cinéma. Cinéphiles, ciné-clubs et salles spécialisées à Paris de 1920 à 1929. Paris: AFRHC and École des Chartes, 1999.

Grant, Catherine, ”Film and Moving Image Studies: Re-Born Digital? Some Participant Observations”, in Frames Cinema Journal, vol. 1, no. 1 (2012)

Olesen, Christian Gosvig, Eef Masson, Jasmijn van Gorp, Giovanna Fossati & Julia Noordegraaf “Data-Driven Research for Film History: Exploring The Jean Desmet Collection” in The Moving Image, Vol. 16, No. 1 (2016)

Petric, Vladimir, ”From a Written Film History to a Visual Film History”, in Cinema Journal, Vol. 14, No. 2, Symposium on the Methodology of Film History (Winter, 1974-1975) 21.

Singer, Ben, “Hypermedia as a Scholarly Tool”, in Cinema Journal, vol. 34, no. 3 (1995)

Witt, Michael. Jean-Luc Godard. Cinema Historian. Bloomington: Indiana University Press, 2013.

PhD Defense 10 May, 2pm

I am very happy to be able to announce the upcoming defense of my PhD dissertation. March 15, the PhD Committee approved my dissertation and with that also the defense date. In the Netherlands, the defense usually takes place approximately two months after the approval of the dissertation and is held as a public ceremony.

The defense will take place on Wednesday 10 May, 2pm at the University of Amsterdam’s Agnietenkapel. On the committee will be  – apart from my supervisor and promotor Professor Julia Noordegraaf (UvA) – the following distinguished members:

Prof. dr. Vinzenz Hediger, Goethe-Universität Frankfurt

Prof. dr. Frank Kessler, Universiteit Utrecht

Prof. dr. Barbara Flückiger, Universität Zurich

Dr. Eef Masson, Universiteit van Amsterdam

Prof. dr. Patricia Pisters, Universiteit van Amsterdam

Prof. dr. Charles Jeurgens, Universiteit van Amsterdam

Prof. dr. Giovanna Fossati, Universiteit van Amsterdam

Below is a glimpse of the cover of the printed version prepared for the defense. The cover image is a so-called summary visualisation of the film L’obsession du souvenir – a Gaumont production from 1913 starring Suzanne Grandais – which I created with the scientific visual analytics software ImageJ. There is more info about how and why this visualisation was created here.

More info on the event can be found at the University of Amsterdam’s and the Research School for Media Studies’ websites here and here.


Update: Digital Film Historiography – A Bibliography

Since 2014 I have been putting together a bibliography of scholarly literature which in one way or another addresses aspects of using and analyzing digitised archival film or film-related sources in research. In particular I have been interested in how scholars currently recast old traditions of film historical research or imagine new ones with digital techniques and tools of analysis. As the bibliography kept growing bigger I began to feel it was necessary to turn it into a thematic bibliography which in a clearer way shows which publications belong to different research traditions. To achieve this, I have recently grouped the publications into a few categories – eight in total. The categories are:

  1. Audiovisal Essays, Found Footage and Remix Culture
  2. CD-Roms, Historical-Critical DVD Editions and Annotation
  3. Stylometry and Cinemetrics
  4. New Cinema History, Databases and GIS
  5. Online Collections, Presentation and Curation
  6. Digital Film Restoration and Historiography
  7. Scientific Visualization, Visual Studies and Epistemology
  8. Digital Exhibition Design and Museology

While I find these categories fairly accurate and productive they are of course debatable, but this is how I felt the publications should be grouped together at this point. I think most of them speak for themselves but my choices of which publications to include in which sections do perhaps require some explanation. On the most basic level, it is a condition for me that a publication offers – in some way – a reflection on film historiography by discussing either a theory, model, method or representational practice which is computer-based or which uses digitised sources or digital means of analytical intervention. As a consequence many publications which discuss these categories/themes but which do not address digitisation or computerised methods have been left out. For instance for section one on “Audiovisual Essays, Found Footage and Remix Culture” there are many essays and monographs which in the past decades have dealt with the interrelation between filmic appropriation practices and film historiography but which fall out of my bibliography’s scope because they do not discuss for instance digital video editing techniques or the use of digitised film collections. The same goes for a section such as “Stylometry and Cinemetrics” which does leave out some of the fundamental reference literature for statistical style analysis from BC (Before Computers) in favour of more recent publications. Of course, it goes without saying that the bibliography is not comprehensive (suggestions and comments are more than welcome on !).

When reorganising the bibliography I also updated all the links associated with the publications, when applicable, so they should – at least for the moment – be working fine and forward you to additional, useful information. In addition, this update provided an opportunity for me to add more publications which have recently come to my attention. One title which I am particularly excited about is the recent monograph by film scholar and archivist Adelheid Heftberger (Austrian Filmmuseum), Kollision der Kader – Dziga Vertovs Filme, die Visualisierung ihrer Strukturen und die Digital Humanities, on the computer-based visualization of structures within Dziga Vertov’s films. This is the outcome of Heftberger’s fascinating doctoral research and the Digital Formalism project in which she meticulously annotated the shots in a group of Vertov films using the open source software Anvil. By doing this it became possible to visualize structures in Vertov’s work using different kinds of scientific, visual analytics (such as MatLab and ImageJ). In her book she discusses the broader implications of these methods both for Vertov research and for the digital humanities. The book is published in a new series on Film Heritage (Filmerbe) directed by Professor in Audiovisual Heritage Chris Wahl at the Filmuniversität Babelsberg Konrad Wolf in Potsdam. Somewhat related – but fundamentally different in its approach and scope – I have also added and recently acquired film scholar André Habib’s book La Main gauche de Jean-Pierre Léaud (Les Éditions du Boréal, 2015). Habib’s book aspires to combine a more anarchic, cinephile tradition of film appreciation with contemporary, representational practices opening with a reflection on the evocative potential of the beautiful visualizations of among other films Alfred Hitchcock’s Psycho (USA, 1960) created by San Francisco-based artist Jim Campbell. Made with ImageJ techniques (or, at least entirely reminiscent of scientific visualizations created with ImageJ techniques) Habib, I think, may with his book possibly be opening a path for film scholars who draw on cinephile theory to historicize films and who wish to define and appropriate such visualizations to their own ends within this tradition. It undoubtedly promises an exciting read in the near future!


Data-Driven Film History at the FIAT World Conference 2014, Amsterdam

I am currently busy working on the project ‘Data-Driven Film History: a Demonstrator of EYE’s Jean Desmet Collection’ which is a small pilot project funded by the Netherlands Organization for Scientific Research (NWO) running from September 2014 till May next year. The aim of the project is to develop a demonstrator tool which proposes new ways for scholars to research and visualize the Jean Desmet Collection held at the EYE Film Institute Netherlands, a collection which has been almost completely digitized.

The Jean Desmet collection is an immensely important collection, recognized as UNESCO World Heritage in 2011, which contains approximately 950 films from cinema’s early years from between 1907-1916, 2000 posters, 700 posters as well as some 120.000 business documents. It is a collection which especially in the 1980s and 1990s through screenings at the silent film festival Le Giornate del Cinema Muto in Pordenone in  Italy made film scholars aware of the great variety and richness of for example film colours in the silent era, and has been crucial in understanding the cinema distribution networks in the Netherlands as well as in Northern Europe. The Dutch film historian Ivo Blom who is one of the most knowledgeable scholars on the collection has written in more detail on this in blog posts and in book form (which can be downloaded for free!) here and here.

It is exactly the aspects of film distribution and colour that we wish our tool to address and which we are currently figuring out how could be done best. I will write about this later in more depth, when we have gone beyond the phase of deciding on the research design and determining exactly how the programming will be done. Yet, for those who cannot wait and are eager to know more about our project, I will be co-presenting a paper on the research questions and methods we have lined up so far, tomorrow at the world conference of the International Federation of Television Archives (FIAT) at the Pakhuis de Zwijger in Amsterdam with my colleague Jasmijn van Gorp from Utrecht University. The paper is co-written with Eef Masson (University of Amsterdam), Giovanna Fossati (EYE Film Institute Netherlands/University of Amsterdam) and Julia Noordegraaf (University of Amsterdam) and a livestream can be followed here, so there is no excuse for not tuning in at 3.30 pm CET!


PhD Defence Sonia Campanini/Guest lecture Barbara Flueckiger

The upcoming week will be eventful for students and scholars with an interest in issues of film preservation and presentation in Amsterdam.

Thursday 17 April Sonia Campanini, joint PhD Candidate of the Università degli studi di Udine and the University of Amsterdam, will be defending her dissertation on the preservation and presentation of film sound with particular attention to sound practices of the silent era. Info on the defence can be found here.

Sonia gave a very fascinating guest lecture on her research on among other things Gaumont’s Chronophone in a seminar series on research methodology which I co-taught in early 2012 for the students of the Preservation and Preservation of the Moving Image MA, so I am particularly excited about seeing the final outcome of her research very soon.

The dissertation committee will include among others Barbara Flueckiger, Professor of Film Studies at the University of Zürich in Switzerland. At the Media Studies department of the University of Amsterdam we saw this as a nice occasion to invite Flueckiger to give a guest lecture on her extensive research on the historiography of film color, her Timeline of Historical Film Colors  and the broader activities of her Diastor research team. As I have had  the responsibility of setting up that lecture and can see that there are still a few places available, I post the info on it below for people who might be interested in joining.



Description of Hermann Wilhelm Vogel’s color system in the Timeline of Historical Film Colors developed by Professor Barbara Flueckiger at the University of Zürich.


Tuesday 15 April between 5.15pm – 7pm, the Media Studies department of the University of Amsterdam will be welcoming Barbara Flueckiger, Professor at the University of Zürich, for a guest lecture on digital film restoration and film color historiography.

The lecture will focus on Professor Flueckiger’s current research in the framework of the project DIASTOR Bridging the Gap Between Analog Film History and Digital Technology (, and its recent involvement in the restoration of Robert Wiene’s “DAS CABINET DES DR. CALIGARI” (1920). After the lecture there will be a Q&A and an open discussion.

The lecture will take place at the following location:

Potgieter zaal, Universiteitsbibliotheek
Singel 425
1012 WP Amsterdam

A map of the University Library’s ground floor where the Potgieter zaal is located can be found here:

Attendance is open to interested students and researchers but registration is necessary as the number of seats available is limited. You can register by contacting Christian Olesen on the following address:

SCMS 2014

I am currently attending the annual conference of the Society for Cinema and Media Studies. The 55th of its kind which is taking place in Seattle this year. Yesterday I gave a talk to a small but dedicated crowd on video essays as a (possible) research practice in film historical scholarship and was overall quite happy with how it went. You can read the abstract for my presentation below.

So, now it is time to enjoy a wealth of great talks and projections until Sunday. I am especially looking forward to Saturday’s screening of Harry Smith’s Heaven and Earth Magic (1957/1962) together with a program of classic experimental films by among others Jud Yalkut. The remaining days will also be a time for meeting people with whom I share research interests and who I do not get a chance to see often, and of course to drink coffee; Seattle is full of great cafés!

scms seattle

From Film Historiography to Videography: Film Historical Video Essays as Scolarly Research Practice 

Recently established online academic journals and video communities such as Frames Cinema Journal and Audiovisualcy testify to an increased tendency to research film history in the form of video essays. While films on film history have existed in the forms of compilation films since the 1920s as a means of discerning aesthetically significant films, and in filmic appropriation art and documentaries since the late 1960s at a nexus with academic film history, the proliferation of scholarly video essays indicates that audiovisual film history is gaining momentum as a conventional scholarly practice. With attention to this development, this paper adresses the need for developing standards for assessing scholarly video essays and critically evaluate the perspectives they establish on film history. The aim of raising such a discussion the paper stresses, is to facilitate the further integration of the scholarly video essay into academia as a research practice. To answer how this could be done, the paper proposes a conceptual avenue which combines meta-historical perspectives from scholarship on filmic appropration art and current debates in digital humanities on the evaluation of digital scholarship.

To bridge these two perspectives the paper takes its cue from the recent introduction to digital scholarship, Digital_Humanities (MIT Press, 2012), co-authored by Anne Burdick, Johanna Drucker, Peter Lunenfeld, Todd Presner and Jeffrey Schnapp which invokes the subjective filmmaking of Chris Marker and Errol Morris as a conceptual model for evaluating time-based digital scholarship1. Departing from their proposition, the paper reflects upon the intersection between scholarly film historiography and independent filmmaking, thinking along the lines of film scholar Bart Testa’s conception of filmic appropriations as ”pedagogical interventions” applicable for teaching in film studies curricula (Testa, 1992) and the meta-historical perspective developed by film scholar Christa Blümlinger on the appropration works of Jean-Luc Godard, Ken Jacobs and Alexander Kluge (Blümlinger, 2009). Mobilizing key concepts from these scholars’ works such as moment, materiality and re-enactment in relation to examples of video essays from Frames and Audiovisualcy, the paper expounds on these concepts’ applicability as scholarly standards for evaluating film historical video essays, to conclusively propose a new direction for their further integration into scholarly practice.

Digital Film Historiography – A Bibliography

With this post I introduce a new page which I have added to my blog called ‘Digital Film Historiography – A Bibliography’. Moving deeper into my research I have become increasingly aware of how little literature exists specifically on the theory and practice of digital methods in film historical research. For that reason I thought it was particularly urgent to try to create an overview of the existing literature, and to create an awareness about this still somewhat limited discussion, first of all for my own sake but potentially also for scholars with similar interests. Hopefully, it may engender additional suggestions from readers, which I may have overlooked.

The page will be updated each time I stumble upon something relevant, so make sure to check it regularly if you, like I am, are particularly interested in this topic. Should the page grow significantly I might consider restructuring it into a thematically structured bibliography, but as for now I did not see a good reason for doing that.

For the sake of simplification I have avoided including general literature on film historiography and film archiving history as it would mean listing a very large number of film historical reference works. In other words, the aim here is not to surpass the effort of Jean Mitry’s Bibliographie internationale du cinéma et de la télévision (1966-1968) but rather to create a growing list of writings on the digital research practices that are currently emerging.


A Numerate Film History

Detail from the flyer accompanying the symposium “A Numerate Film History? Cinemetrics Looks at Griffith, Sennett and Chaplin (1909-1917)” at the University of Chicago


I have written a little introduction to the bibliography which reads as follows:

With digitisation of film heritage occurring at an increasing pace, the past decade has seen an array of digital formats of access and reuse emerge, in scholarly as well as in museum contexts, to become central to the production, contemplation and validation of film historical knowledge: video essays, data visualizations, DVDs, online platforms and museum installations are formats that increasingly permeate sites of film historical knowledge.

As pointed out by film and media scholars Vincenz Hediger (2008), Malte Hagener (2011) and Katherine Groo (2012) with regard to this development, it becomes increasingly urgent to understand how and if the appropriations of digitised films in these formats confirm, challenge or reformulate understandings of film history.

Addressing this debate and the concerns it expresses, I have established the bibliography below with the aim of enhancing the overview of emerging uses of digital methods in film historical research amongst film and media scholars. I have called it “Digital Film Historiography – A Bibliography” to align it to the sub-field of “Digital Historiography” which has existed in the discipline of history for well over a decade and has already seen the publication of several pioneering monographs. As historian David J. Staley puts it in his 2003-monograph Computers, Visualization and History – How New Technologies Will Transform Our Understanding of the Past:

Without our recognizing them as such, visual secondary sources do exist in our profession in the form of diagrams, maps, films, dramatic recreations, and museum displays. While these visual secondary sources surround us daily, historians accord them supplementary status to the ‘real history’ we believe is written (p. 59-60)

My hope in making this bibliography is that the aforementioned formats can indeed be recognized as secondary sources of film history and contribute to the scholarly discussion about their theoretical implications. The bibliography is updated regularly.