Aincient and Partners: Smarter Archive Search through AI and Crowdsourcing

Published on: 22 November 2020

Thanks to the use of Artificial Intelligence (AI), crowdsourcing 2.0 and Handwritten Text Recognition (HTR), it is possible to search more than a million scans of historical texts word for word. The goal of the project is smarter searching in archives from our living labs: Stadsarchief Amsterdam, Nationaal Archief and Noord-Hollands Archief.

However, searching on location is problematic for several reasons, while the need for this is significant. Together with the archives and a crowd of volunteers, Aincient and partners create AI datasets for locations. They develop site-specific searches in a prototype search environment where scans, transcriptions, historical maps and imagery can be explored. Aincient and partners is a consortium of heritage and AI specialists, consisting of Aincient, Picturae, Sioux Technologies and Islands of Meaning.

Deployment of Artificial Intelligence

The AI application offers a solution to the searchability problem of the archives and the users of the archives, especially in the area of locations. Every day, archives are searched online by thousands of users, partly as an important source for scientific research. In addition, this solution can be applied in many heritage institutions in the public sector, in the Netherlands and abroad.

In recent years there has been an enormous development within AI in the field of Natural Language Processing (NLP). Aincient and partners focus within NLP on Named Entity Recognition (NER) for the automatic recognition of locations in historical texts.  For this they use open source tool BERT by Google, a deep learning AI solution. In the previous SBIR phase, the initial results achieved with BERT were good.

What challenge does it solve?

This project solves several challenges. The first challenge is the fact that locations are hard to find in the extensive online archives, while there is a great need for them. Even just for researching the history of your own house or city. This applies to searching through text as well as through maps. The other challenge is the scarcity of datasets to train AI in the heritage sector, also when it comes to locations. Together with the archives and a crowd of volunteers, Aincient and partners create AI datasets for locations, among others. For this purpose they launched the project Tag the text on crowdsourcing platform VeleHanden. Over 10,000 historical texts have been tagged in a month.

Small Business Innovation Research (SBIR)

Aincient and partners participate in the SBIR ‘Artificial intelligence for public services’. Phase 1 has been successfully completed and they are currently working on phase 2, the development of a prototype. They also participate in the SBIR for the National Archives, of which phase 2 has been completed and the implementation phase has started. In addition, Aincient is a member of the working group Culture of the Netherlands AI Coalition.

SBIR makes use of an exception in the procurement legislation for applied research and development. A societal question posed by the government is central.  SBIR works in a tiered innovation competition towards the innovative solution(s). The best proposals execute a feasibility study and the best feasibility studies are commissioned to develop an innovation. This innovation is ideally tested in practice by the potential government customer.

In collaboration with:

Aincient is carrying out the SBIR AI project in collaboration with the Stadsarchief Amsterdam (living lab), the Nationaal Archief (living lab), Noord-Hollands Archief (living lab), Picturae, Sioux Technologies, and Islands of Meaning.

Share via: