IMFD researcher to present at major conference on logic and automata – Millennium Institute Foundational Research on Data

The 2024 edition of Highlights will take place between September 16 and 20 at the University of Bordeaux (France). Pablo Barceló's talk will cover three case studies that highlight the key role of logic in improving our understanding of modern machine learning architectures.

The Laboratoire Bordelais de Recherche en Informatique (LaBRI), which operates at theUniversity of Bordeaux(France) and brings together more than 100 researchers, will this year hostHighlights, one of the world's most important conferences in the fields of model theory, automata theory, databases, and games for logic and verification. The 2024 edition, to be held from September 16 to 20, will feature four guest speakers, including Pablo Barceló, director of the Institute of Mathematical and Computational Engineering (IMC UC).

"Highlights brings together the best work from the most relevant conferences in different areas. It is an informal conference where there are no proceedings, which are collections of papers published in the context of an academic meeting. Instead, people submit the best papers that have already been published, making it a major gathering for those working in areas such as computer science logic and automata," says Barceló, who holds a PhD in computer science and is a researcher at the National Center for Artificial Intelligence (CENIA) and the Millennium Institute Foundational Research on Data IMFD).

The academic adds that, with the aim of offering a broad overview of the latest advances in the different areas covered by the conference, each year the organizers include speakers who are renowned researchers in their respective fields. During this edition, Barceló will be joined by Albert Atserias (Polytechnic University of Catalonia), Mikołaj Bojańczyk (University of Warsaw, Poland), and Laure Daviaud (University of East Anglia, United Kingdom). "In addition to the people who present papers in just 10 minutes, there are the main speakers who give longer talks lasting an hour. In my case, this is the first time I have been invited," he says.

Pablo Barceló, director of the IMC.

The talk given by the director of the IMC in France will be entitled "The Role of Logic in Advancing Machine Learning: Three Case Studies" and will cover case studies that highlight the essential role of logic in improving the understanding of modern machine learning architectures. The first two papers are "Logical Languages Accepted by Transformer Encoders with Hard Attention"and "The Logical Expressiveness of Graph Neural Networks,"which also feature the participation of researchers Alexander Kozachinskiy (postdoc IMC UC/CENIA/IMFD) and Juan (academic IMC/Department of Computer Science UCand director of IMFD), respectively. The third is called "A Uniform Language to Explain Decision Trees"and has Marcelo Arenas ( IMC/UC Department of Computer Science and IMFD researcher) as one of its co-authors.

What I want to show is that the logical aspects of computer science can be used to understand the capabilities of current artificial intelligence models. We work in three areas: one relates to graph neural networks, another is the expressive power of transformers, which are the architectures behind language models such asChatGPT. Finally, we address the interpretability of the models. That is, how I explain the decisions they make, showing in each of them that logic can be an important tool for understanding what can be done and what the computational cost is," says Barceló.

The Transformer Revolution

Transformer models originated from an article published by Google researchers in 2017 entitled "Attention Is All You Need,"quickly becoming one of the most influential developments in the field of machine learning. As the director of the IMC points out, its architecture is fundamental to OpenAI's cutting-edge language models such as ChatGPT, as well as being key to the creation of software such as DeepMind'sAlphaStar.

Transformers are language models that recognize sequences of symbols and words. So, from a mathematical point of view, it is interesting to understand which languages can be recognized by transformers and which cannot. To do this, we formalize a computational model behind transformers and study that model based on which languages it can accept, which generates links with logical language theory and automata theory, which are also mathematical objects that process sequences," says Barceló.

Transformers have become essential for applications such as ChatGPT.

This work is essential for the current development of transformers, whose application can be seen in chatbots, virtual assistants, and other interactive artificial intelligence systems where dynamic conversation with users is vital. "Today we are seeing that they can generate and improve texts. But with the refinement of these models, they will also be used to address multimodal data, which is not only text, but also includes images and sounds. These transformers will be able to extract information from all these sources as if they were a single type of data," explains the researcher.

The potential of graphs

A graph is a mathematical data structure consisting of a set of vertices—called nodes—and a set of edges connecting these vertices. In the 1980s, researchers began exploring ways to apply neural networks to data organized as graphs, but it wasn't until the early 2010s that graph neural networks began to become more widely known, as a result of the development of new architectures and learning algorithms.

"These neural networks are models that process information structured on the basis of graphs. We analyze which properties of graphs can be recognized by a neural network of this type, and to do so we use logical methods. We demonstrate that any expression that can be expressed with a certain logic can be expressed in a graph neural network, and vice versa. If there is a logical expression that can be recognized by a graph neural network, then it can be expressed in this logic. It is like a one-to-one relationship between the two," explains Barceló.

Graph neural networks have seen significant advances in recent years.

Today, these networks have applications in a variety of fields. "For example, graph neural networks are used to detect fraud in banks. If you have a graph of transactions, with a neural network you can start to detect areas where there are operations that look strange. They are also used to optimize transportation networks. They are also very useful in the development of new drugs, as they can show that drugs that tend to attack certain diseases more effectively adhere to a certain molecular structure. With that information, you can go on to create another new drug," says the director of the IMC.

Explainable artificial intelligence

The third study that Barceló will present in France addresses the search for explainable artificial intelligence and demonstrates how first-order logic can be used to design languages that declare, evaluate, and calculate explanations for decisions made by machine learning models.

"Ultimately, models function like black boxes, as we understand very little about how they make decisions. That is why there is a growing trend toward designing models that are more explainable, that somehow provide reasons for why they made certain determinations," says the academic.

An example of this is when a bank uses one of these models to grant or deny a loan to a customer: "It is important to know what criteria were used to reach that decision, rather than simply having the model provide an answer. Ultimately, these criteria are mathematical concepts that can also be formalized through logic." This is because today, people want the explanations they receive to provide mathematical guarantees and to be the best, most concise, or most efficient.

Source: IMC UC