Researchers from Meta AI released ‘balance,’ a Python Package for Balancing Biased Data Samples

Artificial intelligence and machine learning are now essential components in various tasks that contribute to a company’s growth, such as marketing, thanks largely to recent technological advancements in these fields. However, AI has its own set of challenges. There are several scenarios where results generated by machine learning algorithms can be viewed as sexist or discriminatory. For instance, a facial recognition system could be racially discriminatory, or an employee selection process may start to favor one gender over another. These results can be traced to the same root cause, i.e., data bias.

Data bias can occur when a machine learning algorithm is trained with a dataset that is not accurately reflective of its intended usage. Biased data is defined as data that has an over- or under-indexing toward the population of interest and has not been sampled fully randomly. One example of biased data is survey data, which is used to learn about user experience, such as sentiment and opinions, which cannot be quantified by other means. However, because survey information is gathered from a self-selected set of participants, there is a high likelihood that the data collected is biased.

Direct inference of insights or training machine learning models on biased data might produce underperforming algorithms and result in inaccurate predictions. Therefore, it is crucial for practitioners to comprehend whether and how data is biased and to employ statistical techniques to reduce such biases wherever appropriate. At Meta, decision-making about fundamental research and products is heavily influenced by survey data. This was one of the main causes cited by the researchers as to why they believed there was an increasing demand for software tools that would make statistical survey techniques available to researchers and engineers. Working on this problem statement, researchers at Meta introduced ‘balance,’ an open-source Python package for adjusting skewed data samples. Balance provides a straightforward, easy-to-use framework and methodology to deal with biased data samples and assess their biases both with and without adjustments.

Even researchers with little experience with Python or programming can benefit from using the package to its fullest potential. Anyone wishing to balance skewed samples, such as those from surveys, can easily use the package. This includes demographers, UX researchers, market researchers, data scientists, and statisticians. Balance offers a full-fledged workflow, from identifying data biases and creating weights to balance data to producing weighted estimates and assessing the quality of weights. One of its key differentiators is that balance is one of the few open-source survey statistics software developed in Python and uses the language’s flexible environment and well-supported open-source community.

Balance’s main workflow API consists of three stages. Understanding the data’s initial bias toward an objective is the first step. The next step is to create weights for each unit in the sample based on propensity score to adjust the bias in the data. After applying the calculated weights, the final stage entails assessing the bias and variance inflation. The researcher has several options to pick from in the second step, also often known as the “adjustment step.” The primary goal that the researchers kept in mind while designing the package was not to limit practitioners working in any field. This motivated the team to offer a simple API built on the Pandas DataFrame structure that researchers can easily utilize.

Balance has been made released as part of the Meta Open Source initiative, and by making it open-source, Meta aims to foster a community of practitioners with regard to “balance” so that researchers can easily collaborate and discuss techniques and develop tools that will enhance the course of survey-based research in the future. Researchers, data scientists, engineers, and other professionals working in Python who wish to deal with biased data are strongly encouraged to explore the ‘balance’ package to fulfill their use case.

Check out the Tool, Github, and Blog. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our Reddit Page, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.