Understanding The Cerebras High-End Compute Power And Role in AI and Healthcare: Exclusive Talk with Natalia Vassilieva Director of Product, Machine Learning at Cerebras Systems

Natalia Vassilieva is Director of Product, Machine Learning at Cerebras Systems, a computer systems company dedicated to accelerating deep learning. Her focus is machine learning and artificial intelligence, analytics, and application-driven software-hardware optimization and co-design. Prior to joining Cerebras, Natalia was a Sr. Research Manager at Hewlett Packard Labs, where she led the Software and AI group and served as the head of HP Labs Russia from 2011 until 2015. Prior to HPE, she was an Associate Professor at St. Petersburg State University in Russia and worked as a software engineer for several IT companies. Natalia holds a Ph.D. in computer science from St. Petersburg State University. 

Q1. Tell us about Cerebras Systems.

Natalia: Cerebras is an AI systems company. We’ve built a new type of computer system to greatly accelerate the training of deep neural networks (DNN) and open up new areas of research so that scientists and practitioners can do previously impossible work.

Q2. Can you tell us a little about your role as Director of Product, Machine Learning, at Cerebras?

Natalia: My role is rigorous. As a Director of Product, I sit between our engineers, customers, and the market overall. My role is to understand what kind of product we should be building, how it’s useful for our customers, and how it can potentially open doors in other markets. In practice, what that means is looking into trends in the industry overall. In terms of AI, we need to keep an eye on the latest research to understand where the field is going. Machine learning is a very fast-evolving field and many new research papers are published daily. 

Once new research is published, typically with some delay, enterprises adopt these new methods to make it easier for them to use in hardware and other applications. We are looking at state-of-the-art research, what customers want to do with that research, and what kind of applications our customers are seeking to solve. We ask, what kind of methods can be applied to help them with their task? Collecting data on the engineering organization’s requirements enables us to develop the next version of our product or software release.

Q3. Can you tell us a bit about your products, specifically those with applications in healthcare, pharma, and drug discovery?

Natalia: At Cerebras we built the world’s largest and fastest AI computer – the CS-2 system. It is a very powerful computer that enables you to train deep neural networks in hours or days vs the weeks or months it takes with legacy hardware. What we’re hearing from our customers is that when you’re working on cutting-edge research, time matters. Being able to train a model in hours or days means that researchers can test many more hypotheses that can lead to major scientific breakthroughs.

For example, we are working with pharmaceutical leader GlaxoSmithKline to use AI for drug discovery. They had a hypothesis that adding epigenomic data to their AI models would lead to more accurate and useful models. But they were previously unable to test this hypothesis because it would take too long to run on legacy hardware. They called us and we got them onto our CS-1 system. They were able to prove their hypothesis that by adding epigenomic data they could improve their models.

In regards to the pharma industry, there has been a rise in the quality of models when applied to modeling sequence data. You can think about the natural language of text as a sequence of characters or a sequence of words. People find out how to train efficient and representative models on that data in a self-supervised manner, where you don’t need any labels. You just feed all the text that you have, and it learns representations and can do some useful tasks for them. Many models have been designed to represent natural language and sequence data. The models created for language are directly applicable to modeling for biological tasks.

There is growing interest in working in domain-specific text. Being able to get insights from medical literature and to understand what kind of information can be derived from clinical reports or from any written text is important. In biotech, there are many examples of sequence data. Some examples of biological sequences include proteins, the sequence of amino acids, and DNA. If you want to model what happens in the genome, it’s a lot of modeling those sequences. 

These models are typically quite compute-intensive. High compute-intensive tasks require a heavy infrastructure footprint to be trained in a reasonable time. It is challenging to train at high scale on existing, traditional hardware. Our hardware is capable of accelerating the training of those types of models significantly. We are relevant to pharma because of our ability to process data faster with the CS-2 system.

Q4. What is the Cerebras CS-2 system? How does Cerebras use AI to drive faster drug discovery? How does the CS-2 differ from your competitors? 

Natalia: The Cerebras CS-2 is our second generation system. While our competitors are trying to connect together many weak processors, we have one giant processor that can train very large models. One of the main innovations in the CS-2 is the Wafer-Scale Engine which has 850,000 cores. This is significantly more than you can find on any CPU or GPU. It gives us the ability to significantly accelerate tasks that will require a lot of computing power.

With traditional hardware for high compute-intensive tasks, researchers are forced to cluster or connect together multiple traditional processors to be able to complete work in a reasonable time. It’s not very efficient. Instead of connecting multiple, small core count processors, we can use our single, large chip. It is easier to leverage the computing power of many cores, all packaged in a single device. The CS-2 system accelerates different compute tasks, such as the training of deep neural networks.

Q5. What are some of the biggest challenges that Cerebras is looking to address in healthcare and other verticals?

Natalia: Across all verticals, the main value proposition we offer is a powerful tool that allows domain specialists to complete their experiments much faster. We want to enable researchers to learn more quickly from the results of their experiments. 

The field of machine learning is, by and large, a field of trial and error. There is no golden book of rules on how to compose a specific model. Typically, you need to try many different things before converging on something that works for your problem. The speed of experimentation and the speed at which you can run those different trials is extremely important. 

With our hardware, we give researchers a way to make those trials take less time. We let them test many more hypotheses than they would be able to do otherwise. What we often find in practice is that researchers start with the one or two ideas they want to test. It often takes months to test a single idea in traditional environments. With Cerebras, you can test more ideas, and test them faster. 

The reality is if you don’t have the result of those experiments, it kind of slows down your imagination. If a tool can get some results in a matter of hours or days, the number of ideas that the researchers generate just explodes. Once a researcher can see what works and what doesn’t work, they can come up with 2-5 new ideas they want to test out. It fuels creativity and accelerates research significantly.

Q6. Can you tell us about Cerebras's latest partnerships in healthcare and AI? 

Natalia: We have several projects. Our partnerships with pharma companies provide a tool that enables them to create and develop new AI-driven methods. In the case of GlaxoSmithKline (GSK), we are helping them on the path to new therapeutics and new vaccines, while getting insights with the help from artificial intelligence along the way.

Another example is the collaboration with AstraZeneca. AstraZeneca has been interested in developing an internal search engine that will enable a question and answering engine. This Q&A engine will allow their researchers to find where to quickly access answers to questions about past research and past clinical trials. Another task has been building a domain-specific language model, which can help them build the question answering and machine translation engines.

Q7. How does the Cerebras platform give value to its customers?

Natalia: Typically in healthcare, we work with computational chemists, experts in biology and bioinformatics. Many of them are experts in machine learning, but almost none are experts in distributed programming. It really should be easy for them to test their ideas without knowing how the hardware works underneath, and without spending too much time thinking about they should optimize certain tasks. There is great value in running experiments much faster and making it easier for the researchers. Ease of use and fast experimentation is critical. And that is what our system brings to the table. 

I am from Russia, so let me share one more analogy from my university days. My first programming classes were taken when we were allowed just one hour on a computer. You needed to complete all your programs on a piece of paper first. You got one chance to test if your program runs right. You had to think really carefully about how you design that program, how you write that down, and then you either get it right or not and you don’t have any other chances. In many cases right now, researchers are in the same situation with these deep neural networks. When it takes you months to test your hypotheses, you know that you have only one shot, and it limits what you can do. 

Our system has essentially reduced the cost of curiosity. It enables you to not have to spend so many resources on checking whether your idea is worth pursuing. I can go ahead and test it and get more insight faster.

This interview was originally published in our AI in Healthcare Magazine (March 2022)