TEXT - Center for Contemporary Cultures of Text

TEXT – Center for Contemporary Cultures of Text is dedicated to understanding how Generative Artificial Intelligence (GenAI) and Large Language Models (LLMs) are transforming writing cultures at a defining moment in human history. After more than 6,000 years of text being created by hand and human intention, we are now entering a new technological era where all aspects of text creation, circulation, and interpretation are being reshaped by AI.

The center investigates how these changes affect the role of text in society and explores how new practices and technologies may strengthen or challenge existing text cultures. The core belief of TEXT is that a research-based understanding of writing and reading in an AI-driven world is essential to ensure human-centered control over text production and use.

Research within TEXT is conducted across four thematic work packages, each concerned with a different aspect of “LLM-hard” problems – that is, tasks and applications that LLMs are not designed for, and fundamentally struggle to perform at a human level. 

The four work packages are:

1. Creativity and Co-Creation
2. LLM-Hard Assessment
3. Addressing LLM-hard problems in practice
4. LLMs in Language and Cognition Research

Role of Center for Humanities Computing  

CHC is heading the second work package named LLM-Hard Assessment.

This work package focuses on creating evaluation techniques that challenge LLMs with complex, real-world tasks - so-called LLM-Hard problems - that go beyond simple text generation or keyword search. The aim is to develop models that can engage with text in ways that reflect human values, creativity, and contextual understanding.

Current AI evaluation practices often test models on tasks that are too simple to capture the complexity of human writing cultures. This work package challenges that approach by introducing new benchmarks for tasks that require deep reasoning, long-form text generation, cultural knowledge, and value alignment.

By combining insights from creative writing, rhetoric, linguistics, and AI research, CHC will develop evaluation protocols and alignment frameworks that ensure AI models are better suited to assist in complex writing practices — whether in education, literature, journalism, or public debate.

The research is structured around four key tasks:

• Designing new benchmarks for evaluating AI performance on LLM-Hard problems.
• Testing and evaluating existing language models using these benchmarks.
• Developing alignment strategies to ensure that models meet human-centered criteria for text creation.
• Fine-tuning and retraining models to improve their ability to handle complex, real-world writing tasks.

This work package is housed at Center for Humanities Computing at Aarhus University and collaborates closely with Danish Foundation Models - another CHC-project. The team consists of AI researchers, linguists, engineers, and project managers working together to advance responsible and robust AI for text-based applications.