From known knowns to unknown unknowns in AI: Historical and Technical Issues
12 Jun 2024, 15:30 — DIBRIS/DIMA, Via Dodecaneso 35, Room 506
![Fabio.Roli - [Fronte Camicia elegante Cravatta]](/media/images/U0tGWFhp_rkuanz_rveXvWB.width-800.jpg)
Speaker:
Fabio Roli — University of Genoa
Fabio Roli — University of Genoa
Abstract:
AI has been originally developed for closed-world, and noise-free, problems where the possible states of natures and actions that a rationale agent could implement were perfectly known. One could argue that, at that time, AI dealt with known knowns. Since the 1980s, when machine learning became an experimental science, AI researchers started to tackle pattern recognition problems with noisy data, using probability theory to model uncertainty and decision theory to minimize the risk of wrong actions. This was the era of known unknowns, characterized by the rise of benchmark data sets, larger and larger year after year, and the belief that real world problems can be solved by collecting enough training data. However, prior work has shown that available data sets often have a limited utility when used to train pattern recognition algorithms that will be deployed in the real world. The reason is that modern machine learning has often to face with unknown unknowns. When learning systems are deployed in adversarial environments in the open world, they can misclassify (with high-confidence) never-before-seen inputs that are largely different from known training data. Unknown unknowns are the real threat in many security problems ( e.g., zero-day attacks in computer security). In this talk, I give a historical and technical overview of the evolution of AI and machine learning for pattern recognition and discuss how this evolution can be regarded as a transition from known knowns to unknown unknowns, and the key role that adversarial machine learning can play to make AI safer.
AI has been originally developed for closed-world, and noise-free, problems where the possible states of natures and actions that a rationale agent could implement were perfectly known. One could argue that, at that time, AI dealt with known knowns. Since the 1980s, when machine learning became an experimental science, AI researchers started to tackle pattern recognition problems with noisy data, using probability theory to model uncertainty and decision theory to minimize the risk of wrong actions. This was the era of known unknowns, characterized by the rise of benchmark data sets, larger and larger year after year, and the belief that real world problems can be solved by collecting enough training data. However, prior work has shown that available data sets often have a limited utility when used to train pattern recognition algorithms that will be deployed in the real world. The reason is that modern machine learning has often to face with unknown unknowns. When learning systems are deployed in adversarial environments in the open world, they can misclassify (with high-confidence) never-before-seen inputs that are largely different from known training data. Unknown unknowns are the real threat in many security problems ( e.g., zero-day attacks in computer security). In this talk, I give a historical and technical overview of the evolution of AI and machine learning for pattern recognition and discuss how this evolution can be regarded as a transition from known knowns to unknown unknowns, and the key role that adversarial machine learning can play to make AI safer.