Bytedance Announces A New Plugin That Utilizes Machine Learning For Audio Synthesis

On Apr 27, 2022

This Article Is Based On Mawf Insights and Information. All Credit For This Research Goes To The Researchers Of This Project 👏👏👏

Please Don't Forget To Join Our ML Subreddit

Video-sharing platform TikTok has promoted music by both new and established artists alike. TikTok has carved out an increasingly influential role in the music industry. Recently, Bytedance, the company backing the social media platform, has confirmed the development of an intriguing new plugin, a first step toward the music production realm.

Mawf, the new plugin, utilizes machine learning to ‘morph’ incoming audio signals into emulations of real instruments in the DAW. Its ML synthesis engine can also run on only MIDI input. As a result, Mawf can be used as an instrument to color an existent sound or as an independent virtual instrument. Mawf’s beta version includes models for three instruments: saxophone, trumpet, and khlui, a Thai instrument.

The developers behind Mawf utilize machine learning to analyze recordings of professional musicians. The ML models extracted detailed changes in the instrument’s sound associated with variations in pitch and amplitude. Mawf then utilizes these trained models to approximate the sound of these instruments based on input provided by the user. Mawf requires only a solo recording of the target instrument to imitate it.

Though the range of modeled instruments is small, Mawf does feature some exciting additions. There’s an in-built compressor, chorus, reverb effects, and several Control Modes that can adjust how Mawf’s synth engine is triggered, thus allowing the user to control the pitch of the processed audio through MIDI.

According to the developers, ML for audio synthesis is relatively a new technology, and no one has perfected it yet.

Users outside the United States can download Mawf for free; however, beta testing is limited to the first 500 sign-ups.

Source: https://mawf.io/

References: