How Safe Is The Data You Use For Training Your Machine Learning Model?

On Apr 28, 2022

This Article Is Based On The Research Paper 'Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets'. All Credit For This Research Goes To The Researchers Of This Paper 👏👏👏

Please Don't Forget To Join Our ML Subreddit

In today’s world, when machine learning and artificial intelligence are being used in every domain importance of data increases tremendously. A new risk to the data utilized to train these machine learning algorithms has emerged. According to recent studies, the data used to train a machine learning model is no longer secure. A person given only the model’s algorithm may reconstruct and deduce the sensitive information used to train the model in a variety of ways.

According to these studies, machine learning models may be poisoned by a user in order to reconstruct the data used to train that specific model. The worrisome nature of these attacks is highlighted by researchers from Google, the National University of Singapore, Yale-NUS College, and Oregon State University. Previously, it was understood that after a machine learning model was developed, the original training data was discarded, making it difficult for hackers to harvest important information. However, new research indicates that it is quite viable for an attacker to exploit the model to make predictions and then use those predictions to derive a pattern. Once a pattern is identified, it may be used to reconstruct the whole original training dataset.

This new sort of attack involves just observing the model’s predictions without influencing the training process. Researchers are attempting to determine how effective data poisoning might be in disclosing sensitive information. To do this, they want to evaluate the effectiveness and threat level of different forms of inference attacks, as well as the ‘poisoning’ of training data. To begin with, they examined membership inference attacks, which allow attackers to identify whether a certain data record was part of the training set or not. They also looked at reconstruction attacks, which allow them to partially recreate the training data. These attacks, for example, can generate phrases that significantly overlap with words used for training a language model or concluding a statement. Researchers discovered that these attacks are frighteningly successful, implying that cryptographic privacy technologies may not be adequate to protect users’ data privacy.

This study identified severe flaws in the data privacy of machine learning algorithms. Furthermore, it was revealed that data leakage is significantly larger when a user is permitted to poison data when compared to regular inference attacks. While researchers concentrate on developing sophisticated inference attacks to reveal new system vulnerabilities, a team of NUS researchers has created an open-source tool to help analyze data leaks of AI models. The tool simulates membership inference attacks and in the process is able to quantify the level of risk. This tool helps to identify weak points in the dataset and show possible techniques that can be used to mitigate the leakages, The NUS team named this tool Machine Learning Privacy Meter. Until proper integration of data privacy-preserving tools occurs no AI model is safe from inference attacks.

Paper: https://arxiv.org/pdf/2204.00032.pdf

References: