How Do Large Language Models Learn?

In this lesson, we delve into the inner workings of language models. How do they learn the meaning of words? Through massive volumes of data and the constant prediction of the next word in a sequence, LLMs recognize patterns and semantic proximity.However, generating plausible sentences is not the same as understanding or guaranteeing their truth. What does it mean for a response to be statistically probable, but not necessarily correct?

We also analyze the risks: implicit assumptions, reproduced stereotypes, and confusion between entities. Finally, we distinguish between a base model (trained to complete text) and a model adjusted via fine-tuning, aligned to follow specific instructions. This lesson invites us to understand that behind every answer lie training, classification, and alignment processes, and that knowing these mechanisms is essential for using these tools with pedagogical judgment.

At the end of the lesson, you will be asked to answer a few questions. 

👨‍🏫 Instructor: Guido Ivetta

Where Do Large Language Models Learn From?

What data are models trained on, and what do we know (or not know) about it? In this lesson, we analyze the composition of the data used to train language models. Where do those texts come from? The internet, digitized books, academic articles, forums, code repositories, and specialized databases form part of the large corpora used for their training. We review concrete cases (such as BERT, GPT-1, or GPT-3) to understand how data sources have evolved and what happens when companies stop disclosing this information in more recent versions.

We also explore how models are categorized based on their level of openness and transparency: open-source, open-weights, or closed-source. What does it mean to be able to audit a model? What differences exist in terms of costs, access, and community participation? Through current examples, this lesson invites us to reflect on the relationship between data, transparency, and technological power, and to consider what types of models we want to promote in the educational sphere.

At the end of the lesson, you will be asked to answer a few questions. The answers will be graded, and completing this activity is required to finish and earn credit for the course.

👨‍🏫 Instructor: Guido Ivetta

Biases and Ambiguity

In this lesson, we analyze the different types of biases that can emerge in artificial intelligence systems (social, emergent, and automation bias) and how they influence the responses generated. Through concrete examples, we observe how models can reinforce stereotypes, such as associating certain professions with income levels or linking "intelligence" exclusively to developed nations. These cases demonstrate that model responses are not neutral, but rather reflect cultural stereotpyes and hierarchies present in their training data.

We also examine the concept of uncertainty, distinguishing between ambiguous questions and scenarios where the model lacks sufficient knowledge to answer with certainty. When AI lacks awareness of local cultural contexts or regional knowledge, it tends to fill those gaps with generic or stereotyped responses. Therefore, recognizing the limitations of these systems and the importance of being able to state "I don't know" is essential for critical and responsible application in educational settings.

At the end of the lesson, you will be asked to answer a few questions. The answers will be graded, and completing this activity is required to finish and earn credit for the course.

👨‍🏫 Instructor: Guido Ivetta

What are stereotypes in AI models?

In this lesson, we address stereotypes, also known as social biases, in language models. What is a stereotype? They are generalizations about groups of people often built from limited information or prejudices. If AI models learn from data produced by our societies, isn't it expected that they also learn our prejudices? We analyze how biases originate in training data (which is often non-transparent) and how the strong dominance of English and European languages excludes a large portion of the world's linguistic and cultural diversity.

We also distinguish between in-group and out-group stereotypes, observing concrete examples of how these can appear in automated responses. The goal is to recognize that AI is not neutral and that its outputs can reinforce existing inequalities. Through a practical activity, we propose identifying in-group stereotypes and reflecting on out-group stereotypes to understand how technology that relies on stereotypes can impact the educational practices we build.

At the end of the lesson, you will be asked to answer a few questions. 

👩‍🏫 Instructor: Luciana Benotti

Stereotype Validator

In this hands-on activity, you are invited to work directly with biases in generative language models using a stereotype validation tool. The core premise is to formulate pairs of sentences or questions that differ by only one element (such as gender, nationality, profession, or any other social attribute) and observe how the AI responds in each case. By comparing the responses, you can detect whether the model assigns better results to one variant over another, revealing implicit associations or prejudices in its outputs.

The purpose is not to "prove that AI is good or bad," but to develop a critical view that allows you to identify stereotyped patterns in model outputs and reflect on their potential effects in educational and social contexts. The activity encourages recording the observed differences, discussing them with peers, and reflecting on how these biases can influence the way we use and teach with AI.

👨‍🏫 Instructor: Pietro Palombini