The Battleship Approach to the Low Resource Entity Matching Problem
Abstract
Entity matching, a core data integration problem, is the task of deciding whether two data tuples refer to the same real-world entity. Recent advances in deep learning methods, using pre-trained language models, were proposed for resolving entity matching. Although demonstrating unprecedented results, these solutions suffer from a major drawback as they require large amounts of labeled data for training, and, as such, are inadequate to be applied to low resource entity matching problems. To overcome the challenge of obtaining sufficient labeled data we offer a new active learning approach, focusing on a selection mechanism that exploits unique properties of entity matching. We argue that a distributed representation of a tuple pair indicates its informativeness when considered among other pairs. This is used consequently in our approach that iteratively utilizes space-aware considerations. Bringing it all together, we treat the low resource entity matching problem as a Battleship game, hunting indicative samples, focusing on positive ones, through awareness of the latent space along with careful planning of next sampling iterations. An extensive experimental analysis shows that the proposed algorithm outperforms state-of-the-art active learning solutions to low resource entity matching, and although using less samples, can be as successful as state-of-the-art fully trained known algorithms.
Biography
Avigdor Gal is the Benjamin and Florence Free Chaired Professor of Data Science and the Co-chair of the Center for Humanities & AI at the Technion - Israel Institute of Technology. He is with the Faculty of Data & Decision Sciences, where he led the design of the first engineering program in data science in Israel (and possibly the world). Gal’s research focuses on elements of data integration and process management and mining under uncertainty, making use of state-of-the-art machine learning and deep learning techniques to offer an improved data quality with about 150 publications in leading journals, books, and conference proceedings (including multiple best paper and test-of-time awards). His research is implemented, through his ties as a consultant, in multiple industries including FinTech (e.g., Pagaya). In recent years, with the increasing penetration of AI to all aspects of life, Gal has been involved in developing methods for embedding responsible AI in companies and government authorities through an education process that increases dialogue abilities between data scientists and other stake-holders (e.g., lawyers and regulators).