Together Releases The First Open-Source ChatGPT Alternative Called OpenChatKit

Together is creating the first distributed cloud designed specifically for handling huge foundation models. The company offers an intuitive platform combining data, models, and computing to help AI researchers, developers, and businesses better harness and advance AI.

Together team believes that open-source models for philanthropies have the potential to be more democratic, open, strong, and adaptive. They recently released OpenChatKit 0.15 under the Apache-2.0 license, making the code, model weights, and training datasets freely available to the public. The robust, open-source foundation offered by OpenChatKit enables the development of domain-specific and general-purpose chatbots. Users can submit feedback, and community members can add new datasets using the OpenChatKit tools, all of which add to the increasing corpus of open training data, eventually leading to better LLMs.

The Together team collaborated with LAION and Ontocord to build the dataset used for training. Reasoning, multi-turn discussion, knowledge, and generating answers are all supported by OpenChatKit’s chat model, which has 20 billion parameters and was trained on 43 million instructions.

A useful chatbot must be able to regulate responses, obey directions given in normal language, and keep the conversation in context. The OpenChatKit framework includes a generic chatbot and the components necessary to create specialized bots.

There are four main parts to the set:

1. From EleutherAI’s GPT-NeoX-20B, a large language model tuned for a chat with over 43 million instructions on 100% carbon negative compute
2. A set of customization recipes to fine-tune the model to achieve high accuracy on user’s tasks is documented and available open-source under the Apache-2.0 license on GitHub.
3. A retrieval system that can be expanded so that information from a document repository, API, or another live-updating information source can be added to a bot’s responses at inference time; includes publicly available examples for using Wikipedia and web search APIs.
4. A GPT-JT-6B-derived moderation model is accessible on HuggingFace under the Apache-2.0 license; it selects which queries the bot answers.

Potential fields of study and related assignments include:

• The protected rollout of models that can produce bad data without risking user privacy.
• Exploring and comprehending the flaws and biases of models of conversation and language.
• Create works of art and apply them to design and other creative tasks.
• Tools for learning.
• Study of models of conversation or language.

Just like any other language model-based chatbot, GPT-NeoXT-Chat-Base-20B has some restrictions. For instance, the model might not return an accurate or relevant answer when asked something novel, unclear, or outside of its training data. The team invites participation from many groups and individuals to build a more robust and inclusive chatbot.

Check out the Demo, Model and Reference Article. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 15k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.