Black Swan analyzes social media to predict which products will be successful – TechCrunch

Consumer packaged goods companies — think PepsiCo or Nabisco — face steep challenges from the rising cost of living and distribution. As inflation continues unabated, consumers’ disposable income isn’t going as far as it used to while products are becoming more expensive to ship. The pressure is on businesses to place their bets on the right innovations, then. That’s true during less chaotic times, but the stakes are higher at the moment.

While founded long before the pandemic, Steve King says that Black Swan Data, the data science and tech company that he helped co-launch in 2011, is well-suited to the current environment. Black Swan taps into data from conversations on social media and analyzes the data to map “growth opportunities” for companies, attempting to identify trend signals more accurately than traditional market research approaches.

Prior to co-founding Black Swan, King was a technical director at creative agency Digital Jigsaw. Black Swan’s other co-founder, Hugo Amos, was a digital marketing strategy manager at PepsiCo.

“One night in a Toronto bar, Amos and I had our ‘eureka’ moment to connect seemingly disparate data sets to predict consumer behavior,” King told TechCrunch in an email interview. “Having scribbled the idea down on the back of a beer mat (which is now hanging in the London HQ office reception area), Hugo and I returned to the U.K. to start Black Swan. We felt there had to be a better way for businesses and brands to make use of the mass of data available to them; data is largely irrelevant unless you can harness its power effectively.”

Black Swan claims to leverage published research out of Stanford, University College London, Meta and others to try to predict social trends and sales data months into the future. To conduct market research, the platform looks at billions of tweets, posts, discussion forum threads, reviews and blog posts over a two-year period and then filters for roughly 400,000 distinct concepts (e.g. “Themes,” “Ingredients,” “Brands,” “Product types,” “Benefits & needs”) in the data, for example when people discuss food that’s healthy for children to eat after school. From this, Black Swan finds the relationships between concepts to extract insights that — hopefully — help guide a company’s product development.

“Embracing AI gives users the ability to glimpse the future — using predictive algorithms to skate to where the puck is going rather than where it is today,” King said. “Black Swan is akin to the world’s largest focus group. It continuously analyses this data to map growth opportunities and identify emerging trend signals earlier, and more accurately, than traditional market research approaches. This capability is bringing a more scientific and comprehensive approach to the new product innovation process, helping brands to de-risk decision making in uncertain times when consumer behaviour is rapidly shifting.”

It’s true that product development is risky. According to one source, 95% of the more than 30,000 new products introduced every year fail. The failure rate of new grocery store products alone is estimated at between 70% to 80%.

But can AI predict success? The answer isn’t clear. Black Swan claims it can, as do startups like the similarly named Black Crow AI, which sells a service that projects which products e-commerce customers will buy, and Turing Labs, which uses AI to formulate CPG products for mass-market appeal. Just because an algorithm is accurate today, however, doesn’t guarantee that it’ll be accurate tomorrow. As the data shifts, the predictions can shift off course, in the worst case giving a false sense of security.

That’s perhaps why King is careful to note Black Swan doesn’t replace human judgement. Rather, it’s meant to help companies see product categories through the eyes of a consumer, he said, while accounting for individual tastes and preferences.

In any case, Black Swan has done quite well for itself as of late, growing its customer base to 50 companies, including PepsiCo, J&J, Kraft Heinz, SC Johnson and P&G. (PepsiCo has been open about the partnership, crediting Black Swan’s platform with its new line of Propel sports drinks infused with immunity ingredients.) Annual recurring revenue stands at $10 million, and Black Swan — which today announced that it raised $17 million — plans to expand its 170-person workforce to more than 200 by the end of the year. Among other areas of focus in the near term will be growing the startup’s U.S. market share and supporting product development, according to King.

Oxx led Black Swan’s latest funding round, with participation from AlbionVC. It brings the company’s total raised to $18.5 million.

“Black Swan was founded on a belief that brands can make better use of what people are talking about publicly online to help understand their behaviour, anticipate moments, and mold them to their advantage,” King said. “The adoption of tech-driven market-research solutions, and in particular AI-driven observational research and predictive analytics, has accelerated dramatically, and this is reflected in the growth of Black Swan … The benefit of this entire paradigm shift is that Black Swan sees the market from the consumers’ perspective and finds new and emerging trends earlier — enabling users to be more consumer-centric and stay ahead of the curve in their innovation strategies.”