NeurIPS: confirming the international quality of IMFD work
With the boom in artificial intelligence in recent years, NeurIPS has become the place where many of the most relevant global research and developments in computational learning are presented," says Pablo Barceló, director of the Institute for Mathematical and Computational Engineering (IMC UC) and research associate at the Millennium Institute for Computational Learning( MCI). Millennium Institute Foundational Research on Data (IMFD) and Cenia, referring to the relevance of a conference that has been held since 1987 and that today has become the most prestigious in the world in its field. This year, the 37th version of the Neural Information Processing Systems (NeurIPS) will be held in New Orleans (USA) and includes two papers by researchers from the Millennium Institute Foundational Research on Data.
The achievement is highly relevant, especially considering that the conference, which each year brings together more than 10,000 academics and industry representatives, is highly competitive. In each call for papers, around 10,000 scientific publications are collected from researchers specialized in various fields related to machine learning and artificial intelligence, such as neuroscience and natural language processing. Of this enormous volume of articles, only 25% are selected to be presented during the event, which this year will be held between December 10 and 16 at the Ernest N. Morial Convention Center.
Barceló, co-author of the accepted papers, comments that the incorporation of a paper in NeurIPS serves to confirm that the work being developed by the researchers has an international level: "It gives you visibility. When the papers that will be presented are made known, the academics look for everything that was accepted in their areas to get up to date, so many people start to review them," he says. The IMC director, who also holds a PhD in computer science, adds that inclusion in NeurIPS also boosts the global prestige of the institutions to which the authors belong.
"In the papers that will be in NeurIPS, not only the name of the authors appears, but also their affiliations. That helps people abroad to say 'In Chile there is a National Center for Artificial Intelligence, there is a Millennium Institute Foundational Research on Data, there is an Institute of Mathematical and Computational Engineering'. This starts to put local researchers on the world's radar, because it tells them 'Look, today there is an important development pole in Chile. Maybe it doesn't have the productivity that occurs in other parts of the world, but what is produced in Chile does have the same level of quality," adds Barceló.
One of the papers presented at the conference was entitled "Three iterations of (d - 1)-WL test distinguish non isometric clouds of d-dimensional points"and its authors are Pablo Barceló, Mircea Petrache (IMC UC - CENIA), Cristóbal Rojas (IMC UC - CENIA); postdoctoral researcher at the Millennium Institute Foundational Research on Data Alexander Kozachinskyi, and Valentino Delle Rose. Valentino Delle Rose, postdoctoral researcher (IMC UC - CENIA).
Cristóbal Rojas, PhD in Mathematics and Computer Science, comments that this work arises from a question present in the scientific community that uses geometric graph networks to classify, predict and work with representations of molecules and their physical properties, for example, to create drugs. To achieve this, they use different versions of this tool, but they do not know which one is the best for each case, because there is no clear guide to choose the right version, nor the right number of parameters.
"Representations of molecules are clouds of points in three-dimensional space. In this context, what really matters is the shape of the molecule, not the order in which the atoms are described. However, if the order of the atoms is rearranged, the model will consider that different information is being provided. To deal with this problem, unordered lists of distances between the points are used as a way of representing the molecules," says researcher Mircea Petrache.
In this sense, the aim of the paper is to identify how many of these unordered distances are necessary to guarantee an accurate and distinctive representation of point clouds in the context of molecular modeling. With the above, it would be possible to determine how much essential information can be extracted from the distances between points and whether they can be simplified without losing accuracy in the representation.
Rojas adds that this paper represents a great achievement, as it closes a problem that has puzzled the community for quite some time. The study establishes a theorem that guides the People who program these tools on which version to use and how to configure it to obtain the best results on their specific data. This theorem also provides theoretical guarantees on efficiency and expected performance. This is important as artificial intelligence is often considered to be a "black box" where it is not known when it will perform well. In this case, work has been done to open that "box" and show how programming decisions affect the results.
"This is part of an area in which this type of result has been constantly sought. In the area of geometric graph neural networks, which is quite broad, we saw a lot of papers going by where they try to capture this problem, and in partial contributions, comparisons and counterexamples, they try to solve it. Our theorem basically closes this problem and now the answer is complete. In fact, this was a specific aspect highlighted by one of the reviewers of the article", concludes Cristóbal Rojas.
The second paper in which Barceló participates is entitled "A theory of link prediction via relational Weisfeiler-Leman"and is co-authored by Xingyue Huang and Ismail Ilkan Ceylan, from the Department of Computer Science at the University of Oxford, UK; and Miguel Romero Orth, from the Department of Computer Science at UC and researcher at CENIA. In this work, the authors address several particularities of the so-called knowledge graphs that are nowadays used in multiple researches in the area of machine learning and artificial intelligence.
"Knowledge graphs are ways of representing semantic information. By that I mean information that has an interpretation that can be understood by both humans and machines. They can be very large and what they do is correlate entities, indicating for example 'this person is a friend of this person','this person works with this other person', 'from this city to this other city there is a route to get there' or 'this molecule is linked to this other molecule'," comments Barceló.
The researcher adds that the objective of "large machine learning architectures, known as transformers, or very deep neural networks is to somehow use these graphs to acquire semantic information that allows them to learn with less data, to be able to perform new tasks and also to have a higher degree of explainability in the decisions they make". However, the problem with knowledge graphs is that in general the "quality of the data is very low and the data is very incomplete. There are many relationships that are there, for example, 'this person is a friend of this person', but there are many others that are not there, and the idea is how to extract that missing information, how to learn from the structure of the graph to be able to say 'look, with high probability these two People know each other anyway'. So if I'm going to offer a product to this person I should also offer it to this person, because even though the graph doesn't explicitly tell me that they are connected, I infer from their properties and structure that they should be friends".
Today there are many ways to complete these knowledge graphs or to generate information that is not available, but one of the most effective has to do with applying graph neural networks to them. "These neural networks are a little different from those that have been normally studied, and what we did was to try to understand them in terms of those that are better known. We do a complete theory of what these graph neural networks do, which predict connections but at the same time act as an extension of the usual graph neural networks," says Barceló.
By better understanding what these graph neural networks do, says the IMC director, it is also possible to improve their forms and understand which functionalities are necessary and which are added. "The fewer functionalities a network has, the better it learns because it generalizes better. We do a kind of cleaning and we can find its fundamental core, which makes them perform better when predicting connections," says Barceló.
The results of the paper were theoretically and empirically validated, but there is still some way to go. "It is one thing to be able to predict whether two People or molecules are connected, but if two People are connected and are friends and also know someone through their work, it is no longer just a link between two individuals, but a triangle-like structure is generated. We would like to be able to find those structures that are much more complex and at the same time show a greater richness in the structure of the graph," says Barceló.
What uses could such information have? The study of interactions between different molecules for the development of new drugs is just one of them. "In economics, you could detect what kind of correlations there are between different stocks. You could also have a graph of transactions and start to detect, from their structure, possible fraud or instances of money laundering. If I know that there is a fraud and when analyzing the graph I see that there is a similar structure, I could say 'With high probability there is also a fraud here' and thus investigate further," says the IMC director. In addition, one can "think about climate studies, geophysics or areas such as social networks".
Large Chilean presence
Mircea Petrache is an academic at the Institute of Mathematical and Computational Engineering in a position shared with the Faculty of Mathematics UC and is also a researcher at CENIA. He is also an expert in geometric deep learning, generalization errors, equivariant neural networks, topological data analysis and data geometry.
The researcher is co-author of two papers to be presented at the NeurIPS conference. One of them is entitled "Approximation Generalization Trade-offs under (Approximate) Group Equivariance"in collaboration with Shubhendu Trivedi of the Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT). This study explores how symmetries applied in learning models can improve accuracy and efficiency, and what role enabling approximate symmetries can play in those benefits.
Source: IMC UC