Rocket.Chat AI: Empowering you with data sovereignty and accuracy

We live in an era of artificial intelligence, and its benefits are undeniable. However, it is critical to ensure organizations can trust what AI has to offer and reduce risks, especially in government, defense, and critical infrastructure. The European Union AI Act and the Biden Administration’s AI Bill of Rights are a testament to this. 

At Rocket.Chat, security isn’t an afterthought; it’s an integral part of our original blueprint. As we broaden our services to include AI, our commitment to supporting our customers’ unique security requirements only grows stronger. 

We’re thrilled to announce the beta launch of Rocket.Chat AI — a secure, on-premise AI, designed for organizations that prioritize data sovereignty and accuracy. With Rocket.Chat AI, you can deploy a self-hosted Large Language Model (LLM) of your choice and securely augment it with in-house intelligence to generate accurate, contextually appropriate, and up-to-date responses. 

‍

Regulated industries are rich repositories of classified information. In such high-stake environments, the implementation of AI requires thoughtful governance and approach. We recognize the risks associated with integrating AI into their operational processes. As a result, we channeled our efforts into delivering a tool that’s suitable for use in secure, regulated organizations where data sovereignty is paramount. At Rocket.Chat, we believe that security and innovation can go hand in hand when done correctly. 

Our architecture ensures all your proprietary data is handled and processed on-premise within your chosen infrastructure. This data is hosted securely on your servers and doesn’t require interactions with external services. You have full control over the data that is used to enrich your LLM with in-house intelligence. 

Why use Rocket.Chat AI? 

1️. Have complete control and ownership over sensitive data

A vast array of AI solutions available today are cloud-based and you can’t control how these third-party providers use or manage your sensitive data. Unfortunately, in order to avail the AI services, you have to make your data available to these providers.

‍

On the contrary, with a self-hosted LLM, you retain full ownership of your data and can deploy AI services securely, without the stress of potentially exposing your data to public providers. With Rocket.Chat AI, you are in control. 

For our beta launch, we’re using Llama 3, which is one of the most versatile LLMs currently available. Shortly, we will be introducing support for more LLMs, along with the corresponding deployment assets that will enable you to deploy and scale the models. 

2. Implement a reliable tool that is enriched with proprietary data to generate contextually appropriate responses 

We have centered our efforts on delivering a tool that is capable of providing high-quality responses that are accurate, current and pertinent to your business — all while preserving the security and privacy of your intellectual assets. 

On the other hand, an AI solution supporting a standalone LLM without in-house intelligence serves more as a novelty than a necessity — akin to a toy. 

Several collaboration software providers have introduced AI solutions that either rely on external services or do not support the secure augmentation of LLM with internal proprietary data. These standalone LLMs are trained solely on limited, publicly available data. As a result, such a model will only be capable of generating responses that aren’t specific to your business. These limitations render their use unsuitable in secure and regulated organizations.

For instance, if you are a major airline; you wouldn't want your ground staff receiving inappropriate or inaccurate information about flight details. The ramifications could be significant. 

This is precisely where our expertise comes into play. We enhance our LLMs by integrating them with a Retrieval Augmented Generation (RAG) pipeline.  Its primary function is to augment your LLM with company proprietary data to generate accurate, contextually appropriate and up-to-date responses — making it suitable for implementing AI in high-stakes environments. 

Should you be concerned about risking exposure of your classified data to this model? There's no need. Each model is self-hosted, ensuring sensitive information remains within your premises, and you will have full control over the usage and management of your proprietary information. 

Currently, we’re using Rubra.ai for this purpose — however, we are provider-agnostic, and you’re free to use any ingestion pipeline. In the future, we plan to provide support for various data pipelines and vector databases to improve model performance and address your organization’s challenges. Data governance is paramount  — we will maintain and manage the data classification and lineage metadata to ensure controlled and regulated access. 

3. AI-powered app that boosts efficiency and productivity, transforming government and defense operations 

Technology, and AI in particular, is designed to make our lives easier by boosting productivity and efficiency. Our all-new Rocket.Chat AI app embodies this principle.

Facilitate general inquiries — use as internal knowledge engine

The app answers queries using specific, in-house data — so the responses you get are always relevant and accurate because they’re drawn from company-specific data, instead of publicly available information. This is particularly important during government and defense operations. For instance, defense personnel can quickly get information on protocols, making the decision-making process both faster and reliable. Similarly, for government officials, the app can fetch information on interdepartmental policies or regulations, which is an easier way of managing and accessing complex information. 

The app will ensure no important information falls through the cracks. This will accelerate the decision-making process and empower stakeholders to make informed decisions. This is a premium app available at no additional cost for Starter, Pro & Enterprise workspaces. 

Summarize long threads 

Suppose there's an extensive discussion thread on a new security protocol within the homeland security department. In that case, our app can easily navigate the vast exchanges and summarize the critical information. It assists in quickly gleaning actionable insights and maximizing productivity. Also, during defense operations, multiple dialogues happen across different spectrums — HQ, ground forces, and so on — where the app can assist in compiling and summarizing strategic conversations into crisp, digestible, and actionable summaries. 

Summarize customer or citizen conversations

Regardless of lengthy or detailed conversations, the app pinpoints and highlights the key elements and helps customer service teams cut through the clutter by delivering concise summaries. 

This is handy in areas like citizen feedback portals or public grievance systems. For example, the local government can use it to summarize and understand key issues from resident inputs regarding a new initiative. In a defense landscape, it can be used to capture crucial feedback from the ground personnel, providing a quick summary of need-to-know information. 

How can I get started? 

Rocket.Chat AI is available at no cost during the beta phase for all customers on Starter, Pro or Enterprise plans. 

Explore our product documentation to understand the requirements and get started. 

If you're looking to test the waters, you can access deployment assets for the LLM and RAG pipeline from our public repository. Please note, these assets currently support processing up to 4 user interactions at a time. 

For scalable deployment assets, kindly send us an email to customersuccess@rocket.chat or reach out to your Customer Success Manager. We will help you enable it within your instance.

With this beta launch, your feedback and insights become our golden compass, helping us refine and improve the AI offering. Your valuable suggestions and inputs would be shaping the way AI technology serves you.