Skip to main content

Metadata

The sub-pages under this section (on the left) embed the metadata model with explanations. There are two separate, but related schemas for the LAUDATIO metadata: the first concerns the TEI-based metadata that are used as input from data providers. The second are JSON-based metadata that result from the TEI metadata being ingested into Elasticsearch. This set of metadata provides the attributes that are directly displayed in LAUDATIO.

If you are a data provider, you might want to check out the section on how to structure your corpus below first.

In addition, we also provide a summarization of the underlying metamodel for corpus metadata that provides the abstraction layer above the concrete metadata model.

How to structure your corpus​

Let's assume you have a concrete corpus at hand, be it a collection of .txt, Excel, or TEI files or some kind of compressed corpus structure, e.g. ANNIS or EXMARaLDA. In any case, we recommend you to create a .zip archive of your corpus for the actual upload.

You now need at least two and up to three different kinds of TEI files that contain the metadata:

Note that there is only one file for describing the metadata of the corpus and it must be named corpusHeader.xml.1 The pages on document metadata and annotation metadata contain best practices regarding the naming of these files.

tip

If you want to make the document plaintext accessible in LAUDATIO as well, you MUST use the same file name (without extension) for both the underlying document and the metadata file. For example, let's say your actual corpus contains text files (extension: .txt). For a document, we recommend naming the text file and the document header file containing the metadata like this: document1.txt and document1.xml.

MKM – Metamodel for corpora and metadata describing them​

(excerpts from Odebrecht, C. (2018). MKM – ein Metamodell für Korpusmetadaten)

MKM is a modeling, documentation, specification, and visualization of complex metadata from document-based (historical) corpora, regardless of the format or subject area in which they were created. The metamodel, similar to Abney and Bird (2011: 125), understands the corpus as a sum of the steps used to create it. The corpora entail a comprehensive picture developed from the properties that corpora frequently share.

The following assumption applies: All corpora share common, technically abstract properties, regardless of their research question implemented through annotations and categories, or their subject-specific perspective on concepts such as primary or secondary text.

Abstract corpus properties relating to, for example, tokenization, transcription, and annotations should be uniformly represented in the metamodel. Specific forms of tokenization, such as sentence or word-by-sentence tokenization, can thus be described by the same metamodel.

The following question is central to the development of the MKM:

  • Which properties of historical corpora are relevant for their indexing by third parties for the purpose of reuse? More specifically, the following questions arise with regard to the historical corpora to be described:
  • What characteristics do corpora from different disciplines have in common?
  • How can or should discipline-specific interpretations be taken into account?

The foundation of the MKM is formed by the three classes Annotation, Document, and Corpus. From a technical-abstract perspective, the metamodel describes a corpus as the set of its documents and annotations. The following figure shows the structure of the MKM with all its classes, excluding their attributes.

MKM model The metamodel for corpus metadata (MKM). Representation with all classes and relations between classes, without attributes.

Three-level modeling for corpus metadata​

Three-level corpus structure Three level corpus structure.

The three-level modeling in the above figure represents object-oriented corpus metadata at varying levels of abstraction. For example, the RIDGES corpus in the real world contains documents such as the New Kreüterbuch and a specific word class annotation. Their diverse properties can be described with metadata. In the next level, these objects are then partially modeled with the property of having a title.

Additionally, the association (line with a filled diamond) indicates a composition between an object of the Corpus class (e.g., RIDGES) and an object of the Document class. As a next abstraction step, all equivalent objects are grouped into classes, and their shared properties and relationships are described in a metamodel.

Thus, each object of the Annotation class is described with a title, just like each object of the Document and Corpus classes. Furthermore, this level also abstracts the fact that an object of the Corpus class consists of one or more objects of the Document class. An object of the Document class, in turn, consists of one or more objects of the Annotation class.

The Annotation class​

This class represents all metadata for all annotations in a document, and uniformly describes annotation categories and concepts, including primary or secondary text interpretations such as transcriptions or normalizations.

All different transcription levels in the various corpora, as well as all linguistic annotations for i.e. word classes, pos, and non-linguistic annotations such as author , are uniformly modeled as objects of the Annotation class. This also explains why a document is modeled as a composition of one or more annotations. One or more annotations can, for example, represent the historical text (the document) as a transcription or normalization.

Annotation class model The Annotation class and its attributes, as well as further relationships to other classes and their attributes. Values, processing steps, and the formats used are assigned to an object of this class. In addition, each object of this class is assigned people who act as annotators or editors of the annotation, as well as one or more languages.

The attributes of the Preparation class, namely step, ID, mode, description, and correction, specify technical and descriptive metadata that specify each step with a reference, type of processing, and correction method, as well as a description.

Each annotation is implemented in some way in one or more formats, which are modeled with the Format class. Technical metadata is modeled as the name, version, and ID attributes and specifies the name and version of the format and a reference. The toolName and description attributes specify technical metadata about the tool used and the specific processing procedure. This allows all tools and their formats used in a processing step to be documented.

The Document class​

The second central class, Document, does not exist independently of the Annotation class. This class represents all descriptive and structural metadata that describe the historical documents (templates) represented in the corpus. The following figure shows the Document class with its attributes and further associations to other classes.

Document class model The Document class with its attributes, as well as further relationships to other classes with their attributes. An object of the Document class is assigned possible additional sources attributed to the same work, one or more languages, and a publication. In addition, each object of this class is assigned persons acting as authors or publishers of the document, as well as one or more languages.

The Corpus class​

The Corpus class does not exist independently of the Document class. This class represents all descriptive and structural metadata for corpora. The following figure shows the Corpus class with its attributes and further associations to other classes and their attributes.

Corpus class model The Corpus class with its attributes and further relationships to other classes with their attributes. An object of the Corpus class is assigned a project, a person acting as the editor of the corpus, and at least one publication.

The Corpus class thus primarily structurally summarizes the properties of the contained objects of the Document class and indirectly of the Annotation class.

Footnotes​

  1. If you want to upload an updated version of a published corpus, you still need the singular corpusHeader.xml file, but you can simple declare the increased version within the file. See details on how to define a corpus version in the revisionDesc tag of your TEI file. ↩