IMFD: confirming the international quality of the institute’s work at NeurIPS
**October, 2023.-** “With the AI boom in recent years, **NeurIPS has become the place where much of the most relevant research and developments in machine learning worldwide are presented**,” says Pablo Barceló, director of the Institute for Mathematical and Computational Engineering (IMC UC) and associate researcher at the **Millennium Institute Foundational Research on Data (IMFD)** and Cenia, referring to the relevance of a conference that has been held since 1987 and has today become the most prestigious in the world in its field. This year, the 37th edition of the Neural Information Processing Systems (NeurIPS) event will take place in New Orleans (USA) and includes **two papers by researchers from the Millennium Institute Foundational Research on Data**.
This achievement is highly significant, especially considering that the conference, which each year brings together more than 10,000 academics and industry representatives, is extremely competitive. In each call for submissions, around 10,000 scientific papers are collected from researchers specialized in various fields related to machine learning and artificial intelligence, such as neuroscience and natural language processing. **Out of this enormous volume of papers, barely 25% are selected for presentation at the event, which this year will take place between December 10 and 16 at the Ernest N. Morial Convention Center.**
Barceló, co-author of the accepted papers, comments that **having a paper accepted at NeurIPS confirms that the work researchers are doing is of international standing**: “It gives you visibility. When the list of accepted papers is released, academics look for everything that was accepted in their areas to stay up to date, so a lot of people start reviewing them,” he says. The IMC director, who also holds a PhD in Computer Science, adds that **inclusion in NeurIPS also boosts the global prestige of the institutions the authors belong to.**
“In the papers that will be presented at NeurIPS, not only do the authors’ names appear, but also their affiliations. That helps people abroad say, ‘In Chile there’s a National Center for Artificial Intelligence, there’s a Millennium Institute Foundational Research on Data, there’s an Institute for Mathematical and Computational Engineering.’ That starts to put local researchers on the world’s radar, because it tells them, ‘Look, **today there is an important development hub in Chile. It may not have the same output as other places in the world, but what is produced in Chile does have the same level of quality**,” adds Barceló.
One of the papers presented at the conference is titled “Three iterations of (d − 1)-WL test distinguish non isometric clouds of d-dimensional points“, and its authors are **Pablo Barceló**, Mircea Petrache (IMC UC – CENIA), Cristóbal Rojas (IMC UC – CENIA); **postdoctoral researcher at the Millennium Institute Foundational Research on Data** **Alexander Kozachinskyi**, and Valentino Delle Rose, postdoctoral researcher (IMC UC – CENIA).
Cristóbal Rojas, PhD in Mathematics and Computer Science, comments that this work arises from a question present in the scientific community that **uses geometric graph networks to classify, predict, and work with representations of molecules and their physical properties, for example, to develop medications.** To achieve this, they use different versions of this tool, but they don’t know which one is best for each case, because there is no clear guide to choosing the appropriate version, nor the correct number of parameters.
“Representations of molecules are point clouds in a three-dimensional space. In this context, **what really matters is the shape of the molecule**, not the order in which the atoms are described. However, if the order of the atoms is rearranged, the model will consider that different information is being provided. **To deal with this problem, unordered lists of distances between points are used as a way to represent molecules**,” says researcher Mircea Petrache.
In this sense, the goal of the paper is to identify how many of these unordered distances are necessary to guarantee an accurate and distinctive representation of point clouds in the context of molecular modeling. With this, **it would be possible to determine how much essential information can be extracted from the distances between points and whether they can be simplified without losing precision in the representation.**
Rojas adds that this paper represents a major achievement, as it closes a problem that has intrigued the community for quite some time. The study establishes a theorem that guides people programming these tools on which version to use and how to configure it to get the best results for their specific data. This theorem also provides theoretical guarantees about the expected efficiency and performance. **This is important since artificial intelligence is often considered a “black box” in which one doesn’t know when it will work well. In this case, work has been done to open up that “box” and show how programming decisions affect results.**
“This is part of an area in which this type of result has been constantly sought. In the field of geometric graph neural networks, which is quite broad, we’ve seen a large number of papers trying to capture this problem, attempting to solve it through partial contributions, comparisons, and counterexamples. **Our theorem basically closes this problem and now the answer is complete**. In fact, this was a specific point that one of the paper’s reviewers highlighted,” concludes Cristóbal Rojas.
The second paper Barceló participates in is titled “A theory of link prediction via relational Weisfeiler-Leman” and its co-authors are Xingyue Huang and Ismail Ilkan Ceylan, from the Department of Computer Science at the University of Oxford, United Kingdom; and Miguel Romero Orth, from the Department of Computer Science at UC and researcher at CENIA. In this work, **the authors address several particularities of so-called knowledge graphs, which are used today in numerous studies in the field of machine learning and artificial intelligence**.
“**Knowledge graphs are ways of representing semantic information**. By that I mean information that has an interpretation that can be understood both by humans and by machines. **They can be very large, and what they do is correlate entities, indicating for example, ‘this person is friends with this person’**, ‘this person works with this other person’, ‘from this city to this other city there is a route to get there’, or ‘this molecule is linked to this other molecule’,” says Barceló.
The researcher adds that the goal of “large machine learning architectures, known as transformers, or very deep neural networks, is that **they somehow use these graphs to acquire semantic information that allows them to learn with less data, in order to perform new tasks and also have a greater degree of explainability in the decisions they make**.” However, the problem with knowledge graphs is that, in general, “**data quality is very low and they are highly incomplete**. There are many relationships that are present, for example, ‘this person is friends with this other one,’ but there are many others that aren’t, and the idea is **how to extract that missing information**, how to learn from the graph’s structure in order to say, ‘look, with high probability these two people know each other anyway.’ **So if I’m going to offer a product to this person, I should also offer it to this other one**, because even though the graph doesn’t explicitly tell me they’re connected, I infer from their properties and structure that they should be friends.”
Today there are many ways to complete these knowledge graphs or generate the information that isn’t available, but one of the most effective involves applying graph neural networks to them. “**These neural networks are a bit different from those that have normally been studied, and what we did was try to understand them in relation to the more well-known ones.** We build a complete theory of what these graph neural networks do, which predict connections but at the same time act as an extension of the usual graph neural networks,” says Barceló.
By better understanding what these graph neural networks do, the IMC director notes, it is also possible to improve their forms and understand which functionalities are necessary and which are added extras. “The fewer functionalities a network has, the better it learns, since it generalizes better. We do a kind of cleanup and can find its fundamental core, which allows them to have better performance when doing connection prediction,” says Barceló.
The paper’s results were validated both theoretically and empirically, but there is still progress to be made. “**One thing is being able to predict whether two people or molecules are connected, but if two people are connected and are friends and also know someone through work, it’s no longer just a link between two individuals, but a triangle-like structure is generated**. We would like to be able to find these structures, which are much more complex and at the same time show a richer picture of the graph’s structure,” says Barceló.
What uses could this information have? The study of interactions between different molecules for the development of new drugs is just one of them. “In economics, one could detect what kind of correlations exist between different stock market shares. There could also be a transaction graph, and from its structure, one could begin to detect possible fraud or instances of money laundering. If I know there’s a fraud and, when analyzing the graph, I see a similar structure, I could say ‘With high probability there’s fraud here too’ and investigate further,” says the IMC director. In addition, one could “think of climate studies, geophysics, or areas such as social networks.”
**Broad Chilean presence**
Mircea Petrache is a faculty member at the Institute for Mathematical and Computational Engineering, in a joint position with the UC Faculty of Mathematics, and is also a researcher at CENIA. He is also an expert in geometric deep learning, generalization errors, equivariant neural networks, topological data analysis, and data geometry.
The researcher is co-author of two papers that will be presented at the NeurIPS conference. One of them is titled “Approximation Generalization Trade-offs under (Approximate) Group Equivariance”, in collaboration with Shubhendu Trivedi, from the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT). This study explores **how symmetries applied in learning models can improve accuracy and efficiency, and what role allowing approximate symmetries can play in achieving those benefits**.
Source: IMC UC