Remembering in the metaverse

Electronic memory - computing hardware and software that provides services to extend the capacity of our biological memories - can be seen as the fulfillment of the long-established vision of the MEMEX by Vannevar Bush. In a world of ubiquitous computing, our digital shadows - the proportion of our lives that has some digital representation - is no longer limited to individual documents, but reflects the continuous activities in many parts of our lives. Especially, our digital shadows are no longer isolated, but are connected to other people's digital shadows in the space of social data and software. Based on three specific case studies, this thesis tries to develop a concept for a future metaverse archive: an electronic memory infrastructure that enables the long-term preservation, evaluation and dissemination of the information we acquire throughout our lives. The first case study focuses on preservation and introduces the Permanent Visual Archive (PEVIAR) as a solution to digital preservation. Although electronic storage has become abundant and quite cheap, the long-term preservation of information in the digital realm still poses great challenges. While it is not yet clear whether electronic memory ought to be perfect (in contrast to the benign imperfection of our biological memories), the possibility of safely preserving information in the long term must be given. PEVIAR offers a very specific kind of electronic memory, one that is long-term stable, easily accessible, and authentic, but also very static. The second case study focuses on the evaluation of data. It shows how social data can be used to extract the history of collectives. The email communication of 151 individuals working at the former Enron corporation (amounting to a total of around a quarter of a million of messages) is processed in order to reconstruct, visualize and analyze the social network between these individuals. It will be shown how a physical simulation is suitable for visualizing a very complex network while avoiding information overload and how this simulation not only produces the basis for a suitable visualization, but can further be used to analyze the data in combination with established graph metrics. The third study focuses on perception and shows how context-aware display technologies (more specifically, mixed reality) are an indispensable tool in the capture, evaluation and dissemination of our digital corpora. Since much of the information we acquire is directly related to a real-world context, the recalling and consumption of this information should be able to consider this relation. We focus on spatial context to demonstrate two crucial aspects of context-aware information, namely (spatial) context detection and (spatial) context integration. The concept of hybrid images - images that contain real and virtual parts - is introduced as an example of a context-aware information system applied to the field of architecture visualization. The three case studies are connected through their role as building blocks for a future electronic memory infrastructure, the metaverse archive. In the conclusion, we summarize the possibilities and limitations of such an archive and highlight some of the societal implications that will need to be addressed.