What is the purpose of data collection?

In this lesson, we analyze the importance of data collection in evaluating the behavior of language models. These evaluations are conducted using benchmarks, test sets designed to measure performance and, in particular, detect bias. Bias benchmarks function as controlled experiments containing stereotyped sentences and scenarios to observe how the model responds. However, much of the current research focuses on English, which means metrics, datasets, and mitigation strategies are rooted in specific cultural contexts and do not always represent realities like those in Latin America.

Within this framework, we present the HESEIA 2024 Project, an educational initiative involving teachers and students in Argentina that combined critical AI literacy with collaborative data production. Using the EDIA tool, participants explored biases related to social class, gender, and nationality, building a dataset of nearly 50,000 sentences grounded in local and intersectional contexts. Results showed that many regional stereotypes go undetected by current models. HESEIA thus serves as both a pedagogical and technological intervention: fostering critical thinking while contributing to the creation of fairer, more diverse, and more representative benchmarks.

At the end of the lesson, you are invited to answer a few questions. 

👨‍🏫 Instructor: Guido Ivetta

Looking at Other Experiences

In this lesson, we propose viewing artificial intelligence not as an end point, but as a starting point. The closed answers offered by a model can serve to spark questions, dismantle assumptions, and demonstrate that every statement is situated within a cultural and historical context. As we saw in the lesson "Teaching in Times of Artificial Intelligence," two complementary approaches are distinguished: learning about AI, analyzing it as an object of study—its scope, limits, and biases—and learning with AI, using it as a tool to build knowledge without replacing independent thought. Within this framework, the EDIA tool is revisited as a resource to explore stereotypes without requiring advanced technical expertise.

We also present classroom experiences that integrate AI from a reflective perspective: debates on career choice and gender, analysis of myths in sex education, and historical fact-checking activities using conversational assistants. In all cases, the focus is on the critical re-appropriation of information: not accepting the model's answer as a finished product, but transforming it into an original, contextualized, and well-reasoned production. The goal is to move toward a computational literacy that strengthens autonomy, critical thinking, and the collective construction of knowledge.

At the end of the lesson, you are invited to answer a few questions. 

👩‍🏫 Instructor: Emilia Echeveste