How Instagram is Using Causal Inference and Machine Learning to Improve Instagram Notification Management

Instagram is one of the most happening places on social media, and it is quite important to provide a personalized experience by sending most important notifications to the users. Instagram notifications bridge the communication gap between its users. Avoiding less important content in notifications is very much important to enhance user experience. By prioritizing the quality over quantity approach, engineers at Instagram found ways to send fewer notifications that are more important and interesting to them.

A daily digest push notification about stories is a type of notification that lists down stories that are shared and ready to view, and the content may be viewed directly upon clicking the notification. Usually, older Machine Learning models are used to predict the likelihood of clicking on a notification and decide upon the quality of the content in the notification. The lower the quality, the lesser the chances of the notification being pushed to the users. Engineers at Meta are applying Statistics and Machine Learning to improve user experience and manage notifications on a daily basis. Also, using Causal inference and Machine Learning, highly active users have been identified, and the number of notifications sent to them has been reduced, improving the overall user experience.

A traditional Machine Learning model called Click Through Rate (CTR) Model is used to predict the likelihood of the user clicking on the notification. CTR models have worked well in many applications across the industry. The predictions obtained from these models have been used as a proxy to indicate the notification quality to the user. If the predicted click probability is low, then the notification will not be sent to the user and will be dropped in the middle because it is deemed low quality. CTR – model-based filtering proved its worth as the average click rate for a CTR-based model was significantly higher than without the model. 

But with the usage of CTR models, a large number of notifications were also sent to users who are highly active in terms of using Instagram, and for these active users, there isn’t a necessity to send these notifications as they are already active and they tend to see the stories naturally. This has put up a task in front of the Engineers at Meta to send fewer notifications to active users as they tend to see the stories even without receiving the notifications, and the pain point is to identify these users carefully. Because if the users who are active because of the notifications sent to them are put on this list, they might go inactive again, leading to a decrease in User Experience and Engagement.

This boils down to a user selection problem, and we would like to maximize the efficiency by sending notifications to proper cohorts. The solution adapted to tackle the problem is by using Machine Learning and Causal Inference. For problem formulation, we assume that there is a fixed computational cost to send each daily digest notification and a total budget for the notifications to spend, and so this becomes a budget allocation problem now. We will have to figure out the increment in sending the notification compared to not sending it. The user activeness increment is defined as follows ui = Pri(active|do(send notification)) – Pri(active|do(drop notification)). For active user cohorts, the increment value is small, so sending notifications to these cohorts is inefficient and may lead to spamming the users. User cohorts with high incremental values can be selected by arranging them in Descending order, and notifications can be sent out to them for better user experience and management of notifications. This will also help in optimizing the incremental value and budget. But then again, how to decide the incremental value even before sending or dropping the notification? This is where causal inference problems and uplifting modeling techniques emerge. Well, to apply uplift models, we design a randomized experiment in which a notification can be randomly sent or can be dropped, and data is collected from this randomized experiment. Based on the data collected, a neural-network-based uplift model is built to predict the incremental impact between sending and not sending the daily digest notification, and then the solution becomes a trivial budget allocation problem. But in reality, the notifications are generated and scored online. So we cannot have the estimated incremental impact ready for all the notifications in advance. So we need an online approach to determine which notification is to be sent and which is to be dropped. One easy solution to this is to have a fixed threshold and compare it with the generated score, and if the score is above the threshold value, the notification will be sent, leading to a constant notification sending rate r (0<r<1). On the contrary, the sending rate has some ups and downs because the uplifts estimated from the Machine Learning models may change due to various reasons over time. To stabilize these sending rates, online quantile computation services are used to transform raw uplift estimates to a standard uniform distribution while preserving the order.

Using this model, users and notifications with high incremental impact are targeted, leading to a decrease in the send volume compared to the CTR model without any decrease in user engagement and the added benefit of reduced usage of resources. 

Reference Article