Microsoft Present AI Controller Interface: Generative AI with a Lightweight, LLM-Integrated Virtual Machine (VM)

The rise of Large Language Models (LLMs) has transformed text creation and computing interactions. These models’ lack of ensuring content accuracy and adherence to specific formats like JSON remains challenging. LLMs handling data from diverse sources encounter difficulties maintaining confidentiality and security, which is crucial in sectors like healthcare and finance. Strategies like constrained decoding and agent-based methods, such as performance costs or intricate model integration requirements, present practical hurdles.

LLMs demonstrate remarkable textual comprehension and reasoning skills, supported by multiple studies. Fine-tuning these models through instruction tuning enhances their performance across diverse tasks, even for unseen ones. However, issues like toxicity and hallucination persist. Conventional sampling methods, including the nucleus, top-k, temperature sampling, and search-based methods like greedy or beam search, often need to pay more attention to future costs. 

Researchers from Microsoft present AI Controller Interface (AICI). AICI enhances feasibility by offering a “prompt-as-program” interface, surpassing traditional text-based APIs for cloud tools. It seamlessly integrates user-level code with LLMs for output generation in the cloud. AICI supports security frameworks, application-specific functionalities, and diverse strategies for accuracy, privacy, and format adherence. It grants granular access to generative AI infrastructure, locally or in the cloud, enabling customized control over LLM processing.

AICI with a lightweight virtual machine (VM), enabling agile and efficient interaction with LLMs. The AI Controller, implemented as a WebAssembly VM, runs alongside LLM processing, facilitating granular control over text generation. The process involves user request initiation specifying AI Controller and JSON program, token generation with pre, mid, and post-process stages, and response assembly. Developers utilize customizable interfaces to deploy AI Controller programs, ensuring LLM output conforms to specific requirements. The architecture supports parallel execution, efficient memory usage, and multi-stage processing for optimal performance.

The researchers have also discussed different use cases. The Rust-based AI Controllers utilize efficient methods to enforce formatting rules during text creation, ensuring compliance through trie-based searches and pattern checks. These controllers support mandatory formatting requirements and are expected to offer more flexible guidance in future versions. Users can control the flow of information, timing, and manner of prompts and background data, enabling selective influence over structured thought processes and preprocessing data for LLM analysis, streamlining control over multiple LLM calls.

To conclude, the researchers from Microsoft have proposed AICI  to address the issues of content accuracy and privacy. AICI surpasses traditional text-based APIs. It integrates user-level code with LLM output generation in the cloud, supporting security frameworks, application-specific functionalities, and diverse strategies for accuracy and privacy. It offers granular access for customized control over LLM processing, locally or in the cloud. AICI can be used for different purposes like efficient constrained decoding, enabling rapid compliance-checking during text creation, information flow control, facilitating selective influence over structured thought processes, and preprocessing background data for LLM analysis.