NeurIPS: confirming the international quality of IMFD's work
October, 2023.- "With the boom in artificial intelligence in recent years, NeurIPS has become the place where many of the most relevant global research and developments in machine learning are presented, "says Pablo Barceló, director of the Institute of Mathematical and Computational Engineering (IMC UC) and associate researcher at the Millennium Institute Foundational Research on Data (IMFD) and Cenia, referring to the importance of a conference that has been held since 1987 and has now become the most prestigious in the world in its field. This year, the 37th edition of the event Neural Information Processing Systems (NeurIPS) will take place in New Orleans (USA) and includes two papers by researchers from Millennium Institute Foundational Research on Data.
This achievement is highly significant, especially considering that the conference, which brings together more than 10,000 academics and industry representatives each year, is extremely competitive. Each call for papers attracts around 10,000 scientific publications from researchers specializing in various fields related to machine learning and artificial intelligence, such as neuroscience and natural language processing. Of this enormous volume of articles, only 25% are selected to be presented during the event, which this year will take place between December 10 and 16 at the Ernest N. Morial Convention Center.
Barceló, co-author of the accepted papers, comments that the inclusion of a paper in NeurIPS serves to confirm that the work being carried out by researchers is of international standard: "It gives you visibility. When the papers to be presented are announced, academics look for everything that has been accepted in their areas to catch up, so many people start reviewing them," he says. The director of the IMC, who also holds a PhD in computer science, adds that inclusion in NeurIPS also enhances the global prestige of the institutions to which the authors belong.
"In the papers that will be presented at NeurIPS, not only do the authors' names appear, but also their affiliations. This helps people abroad say, 'In Chile, there is a National Artificial Intelligence Center, there is a Millennium Institute Foundational Research on Data, and there is a Mathematical and Computational Engineering Institute.' This puts local researchers on the world's radar, because it tells them, 'Look, there is now an important development hub in Chile. It may not have the same productivity as other places in the world, but what is produced in Chile is of the same high quality, 'adds Barceló.
One of the papers presented at the conference is entitled "Three iterations of (d − 1)-WL test distinguish non isometric clouds of d-dimensional points," and was authored by Pablo Barceló, Mircea Petrache (IMC UC – CENIA), Cristóbal Rojas (IMC UC – CENIA); postdoctoral researcher at Millennium Institute Foundational Research on Data Alexander Kozachinskyi, and Valentino Delle Rose, postdoctoral researcher (IMC UC – CENIA).
Cristóbal Rojas, PhD in Mathematics and Computer Science, says that this work stems from a question raised by the scientific community that uses geometric graph networks to classify, predict, and work with representations of molecules and their physical properties, for example, to create medicines. To achieve this, they use different versions of this tool, but they do not know which one is best for each case, because there is no clear guide for choosing the right version or the correct number of parameters.
“Molecular representations are point clouds in three-dimensional space. In this context, what really matters is the shape of the molecule, not the order in which the atoms are described. However, if the order of the atoms is rearranged, the model will consider that different information is being provided. To deal with this problem, unordered lists of distances between points are used as a way to represent molecules, " says researcher Mircea Petrache.
In this regard, the objective of this paper is to identify how many of these unordered distances are necessary to ensure accurate and distinctive representation of point clouds in the context of molecular modeling. With this information, it would be possible to determine how much essential information can be extracted from the distances between points and whether they can be simplified without losing accuracy in the representation.
Rojas adds that this paper represents a major achievement, as it resolves an issue that has puzzled the community for quite some time. The study establishes a theorem that guides the People program these tools on which version to use and how to configure it to obtain the best results for their specific data. This theorem also provides theoretical guarantees about efficiency and expected performance. This is important because artificial intelligence is often considered a "black box" where it is not known when it will work well. In this case, work has been done to open that "box" and show how programming decisions affect results.
"This is part of an area in which this type of result has been constantly sought. In the field of geometric graph neural networks, which is quite broad, we saw a large number of papers attempting to capture this problem and, through partial contributions, comparisons, and counterexamples, trying to solve it. Our theorem basically closes this problem, and now the answer is complete. In fact, this was a specific aspect that one of the article's reviewers highlighted," concludes Cristóbal Rojas.
The second paper in which Barceló participates is entitled "A theory of link prediction via relational Weisfeiler-Leman" and was co-authored by Xingyue Huang and Ismail Ilkan Ceylan, from the Department of Computer Science at the University of Oxford, UK; and Miguel Romero Orth, from the Department of Computer Science at UC and a researcher at CENIA. In this work, the authors address several particularities of the so-called knowledge graphs that are currently used in multiple research projects in the field of machine learning and artificial intelligence.
Knowledge graphs are ways of representing semantic information. By that I mean information that has an interpretation that can be understood by both humans and machines. They can be very large and what they do is correlate entities, indicating, for example, 'this person is friends with this person,''this person works with this other person,' 'there is a route from this city to this other city,' or 'this molecule is linked to this other molecule,'" says Barceló.
The researcher adds that the goal of "large machine learning architectures, known as transformers, or very deep neural networks, is to somehow use these graphs to acquire semantic information that allows them to learn with less data, to be able to perform new tasks, and also to have a greater degree of explainability in the decisions they make." However, the problem with knowledge graphs is that, in general, "thequality of the data is very low and it is very incomplete. There are many relationships that are there, for example, 'this person is friends with this other person,' but there are many others that are not, and the idea is how to extract that missing information, how to learn from the structure of the graph to be able to say, 'look, with a high probability, these two People know each other. So if I'm going to offer a product to this person, I should also offer it to this other person, because even though the graph doesn't explicitly tell me that they are connected, I infer from its properties and structure that they should be friends."
Today, there are many ways to complete these knowledge graphs or generate information that is not available, but one of the most effective involves applying graph neural networks to them."These neural networks are a little different from those that have been studied normally, and what we did was try to understand them in terms of the better-known ones. We have developed a complete theory of what these graph neural networks do, which predict connections but at the same time act as an extension of the usual graph neural networks," says Barceló.
By better understanding what these graph neural networks do, explains the director of the IMC, it is also possible to improve their forms and understand which functionalities are necessary and which are added. "The fewer functionalities a network has, the better it learns because it generalizes better. We do a kind of cleaning and can find its fundamental core, which makes it perform better when predicting connections," says Barceló.
The results of the paper were validated theoretically and empirically, but there is still work to be done. "It's one thing to be able to predict whether two People molecules are connected, but if two People connected and are friends and also know someone through their work, it's no longer just a link between two individuals, but a triangle-like structure is generated. We would like to be able to find these structures, which are much more complex and at the same time show greater richness in the structure of the graph," says Barceló.
What uses could this information have? Studying interactions between different molecules for the development of new drugs is just one of them. "In economics, it could be used to detect correlations between different stock market actions. It could also be used to create a graph of transactions and, based on its structure, begin to detect possible fraud or money laundering. If I know there is fraud and, when analyzing the graph, I see that there is a similar structure, I could say, 'There is a high probability that there is fraud here too,' and thus investigate further," says the director of the IMC. In addition, one can "think of climate studies, geophysics, or areas such as social networks."
Chile's extensive presence
Mircea Petrache is an academic at the Institute of Mathematical and Computational Engineering, sharing his position with the UC Faculty of Mathematics, and is also a researcher at CENIA. He is also an expert in geometric deep learning, generalization errors, equivariant neural networks, topological data analysis, and data geometry.
The researcher is co-author of two papers that will be presented at the NeurIPS conference. One of them is entitled "Approximation Generalization Trade-offs under (Approximate) Group Equivariance," in collaboration with Shubhendu Trivedi of the Massachusetts Institute of Technology (MIT) Computer Science and Artificial Intelligence Laboratory. This study explores how symmetries applied in learning models can improve accuracy and efficiency, and what role allowing approximate symmetries can play in achieving those benefits.
Source: IMC UC