[Extended abstract for a paper presented at DH2026]
Introduction
Large Language Models (LLMs) offer exciting opportunities for automating a range of tasks relevant to digital libraries which would not have been possible at a comparable scale prior to their development (Parandjuk and Ress, 2025). This paper presents results of concrete integration of Large Language Models into an existing widely-used digital library and scholarly database of premodern Chinese writing, the Chinese Text Project (Sturgeon, 2019). The main contributions of this paper are: 1) a practical workflow for leveraging generative AI for historical languages in crowdsourced contexts and scholarly digital libraries; 2) quantitative evaluation of using fine-tuning to resolve commonly observed types of LLM-induced error in the domain of translation of premodern primary sources; 3) workflows and evaluations of novel approaches using existing textual annotations to improve the accuracy of LLM outputs.
Background
The Chinese Text Project is an open access digital library that has been online since 2005, primarily containing early published and manuscript editions of Chinese written works. These are collected firstly as page images, currently totaling over 36 million pages, and transcribed content created through a combination of automated alignment (Sturgeon, 2017), Optical Character Recognition (Sturgeon, 2018), and crowdsourcing (Sturgeon, 2022), and currently amounting to over 7 billion characters. These include over 46,000 texts and editions, by authors spanning much of China’s recorded history from around 1000 BC through to the early 20th century. The vast majority of this collection is written in Classical or Literary Chinese, and only a very small proportion of it has previously been translated into any other language. Modern readers – even if skilled in the language itself – may frequently lack important context about entities mentioned in a text, such as a dynastic history, that would have been obvious to a reader at the time the work was composed, but would now be something only a specialist in that particular subject area or time period would be likely to have heard of. While a majority of users of the system are native Chinese speakers, a substantial minority are native speakers of English or other non-Chinese languages, for whom reading aids and translations are of particular benefit.
For this reason, reading aids and translations have been part of this library since its creation.[1] Nevertheless, of the very small fraction of the corpus that has ever been translated into any language, only an even smaller subset is in the public domain or available for reuse through publication under an open license.
Reading aids provided consist of linked dictionary data, as well as named entity annotations. The latter are also used to provide disambiguation for same-named entities (e.g. to distinguish two Qing dynasty officials both named “Yu Chenglong”); this disambiguation can be challenging to perform even for human annotators, as disambiguation decisions rely upon further data on each of the candidate entities (e.g. known facts about each of two same-named persons, together with the context in which the name to be annotated appears). Relevant historical data is extracted from annotated texts and stored in a knowledge graph to record known information about entities; in the case of well-known historical entities, Wikipedia articles are linked to the entity records to aid in disambiguation, and provide human-readable summaries and contextualization. At present, 30% of entities (out of a total of around 90,000) have a corresponding article in Chinese Wikipedia, and less than 6% in English Wikipedia.
Entity summarization
Although only a small part of the collection as a whole, the Chinese Text Project includes a large number of key historical sources, such as the official dynastic histories, private histories, diaries, and local gazetteers. Many of these sources include passages in Classical Chinese that provide detailed “definitional” information about an entity, as occurs in the “biographies” section of many Chinese histories, where the work describes what the author considered to be the important facts about an individual that should be recorded for posterity. Notably, for less well-known entities in particular, it is common for an encyclopedia article to cite and rely on precisely these passages – either because these are the only known source, or the most conveniently accessible one.[2] In this section, we automatically retrieve relevant passages, and create multilingual human-readable summaries of annotated entities for which no corresponding Wikipedia article exists.
While current LLMs perform remarkably well on many styles of Classical Chinese, in creating these summaries we identify and address two persistent types of error:
- Errors of transliteration. In creating English entity summaries, transliteration from Chinese characters into Latin characters is frequently required to complete the task. While Hanyu Pinyin is most commonly used today, unless specifically trained to generate this type of Romanization only, LLMs in fact produce a mixture of non-standard and/or incorrect Romanizations for proper names.
- Incorrect contextualization of dates. Mapping from dates expressed in the traditional Chinese calendar system to the Julian or Gregorian calendars is a complex task. LLMs will have processed during training many documents that include a specification of a Chinese date, followed by its corresponding date in another calendar – e.g. “6th year of Qianlong (1741)” or “The Bingshen day of the first month of the first year of Qianlong (12 February 1736)”. In practice, we observe that while LLMs typically generate somewhat plausible-looking contextualized dates, with the exception of dates at the granularity of a year, the generated dates are almost always incorrect.
We address the first of these by supervised fine-tuning based on pairs of Romanized Chinese proper names. To do this, we sample manually curated data from the China Biographical Database (Chen and Wang, 2022), which contains names in both Chinese and Pinyin Romanization for over 600,000 historically attested Chinese individuals, and use this to generate synthetic Chinese and English sentence pairs that include at least one proper name requiring Romanization (Figure 1). We apply supervised fine-tuning to Qwen3-32B (Yang et al., 2025), and demonstrate that this significantly improves transliteration accuracy, reducing the error rate by more than half from 25% to 12%.
| Translate the following description of the person “孫長卿” into English. Passage: 孫長卿，字鳳池，臨猗人。 |
| Sun Changqing, courtesy name Fengchi, was a native of Linyi. |
Figure 1. Example generated prompt and target completion used for Romanization fine-tuning.
For contextualization of dates, we use a combination of prompt engineering and explicit contextualization. Where present, annotations are used to rewrite the source text to include the desired converted date before feeding this to the LLM (Figure 2).
| Answering in English, based on the information in the following passage, write an encyclopedia article for the person “馬鳴蕭”. Passage: 馬鳴蕭，字和鑾，號子乾，青縣人。順治丁亥進士，歷官工部員外郎。有《惕齋詩草》。 |
| Answering in English, based on the information in the following passage, write an English encyclopedia article for the person “馬鳴蕭”. Passage: 馬鳴蕭，字和鑾，號子乾，青縣人。順治丁亥（公元1647）進士，歷官工部員外郎。有《惕齋詩草》。 |
Figure 2. Example transformation of source text. The prompt has been abbreviated for clarity.
Aligned translations of primary sources
Contemporary LLMs typically have been trained with substantial volumes of data for multilingual translation tasks, and so can be expected to give strong performance on similar tasks. However, requirements for translation of historical primary source material in scholarly contexts often introduce additional constraints not present in the typical general-purpose translation task. In particular, for practically any scholarly use of AI generated translations of historical Chinese primary sources, a key requirement is likely to be verifiability – in particular, providing a reader of the translation with the ability to precisely identify the source of each part (e.g. sentence or phrase) of the translation, in order for the researcher to evaluate based on their own knowledge the adequacy of the translation. In this section, we evaluate two approaches to the creation of aligned translations – i.e. translations where units (e.g. sentences) of the translated output text are explicitly mapped to units of the source text – and use these to implement corresponding functionality in the digital library (Figure 3). We compare two approaches:
- Providing a complete text as a single prompt, inserting explicit markers between units we wish to align, and stating in the prompt that these markers should be maintained in the output. In this approach, the model has the full context when translating.
- Segmenting the input text into individual sentences, and treating the translation task as a turn-based sequence of inputs and generated outputs. In this approach, the model has access to its prior generated context (but not future context) when generating.
We observe that while initially promising, the first approach is prone to frequent errors, due to missed or corrupted markers. Modifications attempted included numbering the markers (in order to detect and recover from a missing marker in the output); this actually leads to worse results if the markers are sequentially numbered, but becomes viable if randomized identifiers are used for each marker. The second approach by contrast is incapable of producing misaligned translations, with the obvious disadvantage being the inability of the model to make use of future context – for example, producing less appropriate translations of a title due to being required to produce it without having seen the content of the text. This can be mitigated by providing some or all of the text as context prior to requesting translation of the first sentence.
Figure 3: A passage of text with a sentence-aligned LLM-generated translation, displayed as aligned passages (top), and aligned sentences (bottom). Two layers of the original text – a historical text (black) and a later interlinear commentary on that text (green) are visible, together with a search hit (red/blue), which is highlighted in translation in both views at sentence granularity.
[1] A concrete example is the Confucian Analects, with aligned English and Modern Chinese translations. https://ctext.org/analects
[2] For an example, see the Chinese Wikipedia page for 馬鳴蕭, which currently (15 December 2025) relies upon a single historical source text.
Bibliography
Chen, S., Wang, H., 2022. China Biographical Database (CBDB): A Relational Database for Prosopographical Research of Pre-Modern China. Journal of Open Humanities Data 8. https://doi.org/10.5334/johd.68
Parandjuk, J., Ress, S., 2025. The Use of AI Tools for Enhancing Digital Library Services With Information Architects as Responsible Partners, in: Digital Libraries Across Continents. Routledge.
Sturgeon, D., 2022. Crowdsourcing the Historical Record: Creating Linked Open Data for Chinese History at Scale. IJHAC 16, 50–63. https://doi.org/10.3366/ijhac.2022.0276
Sturgeon, D., 2019. Chinese Text Project: A Dynamic Digital Library of Premodern Chinese, Digital Scholarship in the Humanities.
Sturgeon, D., 2018. Large-scale Optical Character Recognition of pre-modern Chinese texts. International Journal of Buddhist Thought and Culture 28, 11–44.
Sturgeon, D., 2017. Unsupervised Extraction of Training Data for Pre-Modern Chinese OCR., in: FLAIRS. pp. 613–618.
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, Jian, Tu, J., Zhang, J., Yang, Jianxin, Yang, Jiaxi, Zhou, Jing, Zhou, Jingren, Lin, J., Dang, K., Bao, K., Yang, K., Yu, L., Deng, L., Li, Mei, Xue, M., Li, Mingze, Zhang, P., Wang, P., Zhu, Q., Men, R., Gao, R., Liu, S., Luo, S., Li, T., Tang, T., Yin, W., Ren, Xingzhang, Wang, X., Zhang, X., Ren, Xuancheng, Fan, Y., Su, Y., Zhang, Yichang, Zhang, Yinger, Wan, Y., Liu, Y., Wang, Z., Cui, Z., Zhang, Z., Zhou, Z., Qiu, Z., 2025. Qwen3 Technical Report. https://doi.org/10.48550/arXiv.2505.09388