Making a Chatbot with FalconAI, LangChain, and Chainlit

Introduction

Generative AI, particularly the Generative Large Language Models, have taken over the world since their beginning. This was solely potential as a result of they might combine with totally different purposes, from producing working programmable codes to creating totally GenerativeAI-managed Chat Help Programs. However many of the Massive Language Fashions within the Generative AI area have been closed to the general public; most weren’t open-sourced. Whereas there do exist a couple of Open Supply fashions, however are nowhere close to the closed-source Massive Language Fashions. However just lately, FalconAI, an LLM, was launched, which topped the OpenLLM leaderboard and was made Open Sourced. With this mannequin on this information, we’ll create a chat utility with Falcon AI, LangChain, and Chainlit.

Studying Targets

• To leverage Falcon Mannequin in Generative AI Purposes
• To construct UI for Massive Language Fashions with Chainlit
• To work with Inference API to entry pre-trained fashions in Hugging Face
• To chain Massive Language Fashions and Immediate Templates with LangChain
• To combine LangChain Chains with Chainlit for constructing UI Purposes

This text was printed as part of the Data Science Blogathon.

What’s Falcon AI?

Within the Generative AI subject, Falcon AI is likely one of the just lately launched Massive Language Fashions recognized for taking first place within the OpenLLM Leaderboard. Falcon AI was launched by UAE’s Expertise Innovation Institute (TII). Falcon AI’s structure is designed in a manner that’s optimized for Inference. When it was first launched, Falcon AI topped the OpenLLM Leaderboard by shifting forward of state-of-the-art fashions like Llama, Anthropic, DeepMind, and so on. The mannequin was educated on AWS Cloud with 384 GPUs connected repeatedly for 2 months.

Presently, it consists of two fashions, Falcon 40B(40 Billion Parameters) and Falcon 7B(7 Billion Parameters). The primary half is that the Falcon AI makers have talked about that the mannequin will probably be Open Sourced, thus permitting builders to work with it for industrial use with out restrictions. Falcon AI even supplies the Instruct fashions, the Falcon-7B-Instruct and Falcon-40B-Instruct, with which we are able to rapidly get began to construct chat purposes. On this information, we’ll work with the Falcon-7B-Instruct mannequin.

What’s Chainlit?

Chainlit library is just like Python’s Streamlit Library. However the supposed objective of this Chainlit library is to construct chat purposes with Massive Language Fashions rapidly, i.e., to create a UI just like ChatGPT. Growing conversational chat purposes inside minutes with the Chainlit package deal is feasible. This library is seamlessly built-in with LangFlow and LangChain(the library to construct purposes with Massive Language Fashions), which we’ll do later on this information.

Chainlit even permits for visualizing multi-step reasoning; it lets us see the intermediate outcomes to know the way the Massive Language Mannequin reached the output to a query. So you possibly can clearly see the chain of ideas of the mannequin by the UI itself to know how the LLM concluded the given query. Chainlit is restricted to a textual content dialog and permits for sending and receiving Photos to and from the respective Generative AI fashions. It even lets us replace the Immediate Template within the UI as an alternative of returning to the code and altering it.

Producing HuggingFace Inference API

There are two methods to work with the Falcon-7B-Instruct mannequin. One is the normal manner, the place we obtain the mannequin to the native machine after which use it straight. However as a result of this can be a Massive Language Mannequin, it’s going to want excessive GPU reminiscence to make it work. Therefore we go together with the opposite possibility, calling the mannequin straight by the Inference API. Inference API is a HuggingFace API token with which we are able to entry all of the transformer fashions within the HuggingFace.

To entry this token, we have to create an Account in HuggingFace, which we are able to do by going to the official HuggingFace web site. After logging in/signing in along with your particulars, go to your profile and click on on the Settings part. The method from there will probably be

So in Settings, go to Entry Tokens. You’ll create a brand new token, which we should work with the Falcon-7B-Instruct mannequin. Click on on the New Token to create the brand new token. Enter a reputation for the token and set the Function choice to Write. Now click on on Generate to generate our new Token. With this token, we are able to entry the Falcon-7B-Instruct mannequin and construct purposes.

Making ready the Surroundings

Earlier than we dive into our utility, we’ll create a great surroundings for the code to work. For this, we have to set up the required Python libraries wanted. Firstly, we’ll begin by putting in the libraries that help the mannequin. For this, we’ll do a pip set up of the beneath libraries.

$ pip set up huggingface_hub 
$ pip set up transformers

These instructions will set up the HuggingFace Hub and the Transformers libraries. These libraries name the Falcon-7B-Instruct mannequin, which resides within the HuggingFace. Subsequent, we will probably be putting in the LangChain library for Python.

$ pip set up langchain

This can set up the LangChain Package deal for Python, which we’ll work with to create our chat utility with the Falcon Massive Language Mannequin. Lastly, with out the UI, the conversational utility will not be finished. So for this, we will probably be downloading the chainlit library.

$ pip set up chainlit

This can set up the Chainlit library for Python. With the assistance of this library, we will probably be constructing the UI for our conversational chat utility. After putting in chainlit, we have to check the package deal. For this, use the beneath command within the terminal.

chainlit good day

After coming into this command, a brand new window with the handle localhost and PORT 8000 will seem. The UI will then be seen. This tells that the chainlit library is put in correctly and able to work with different libraries in Python.

Creating the Chat Software

On this part, we’ll begin constructing our utility. We now have all the required libraries to go ahead to construct our very personal conversational chat utility. The very first thing we will probably be doing is importing the libraries and storing the HuggingFace Inference API in an environmental object.

import os
import chainlit as cl
from langchain import HuggingFaceHub, PromptTemplate, LLMChain

os.environ['API_KEY'] = 'Your API Key'

• So we begin by importing the os, chainlit and langchain libraries.
• From langchain, now we have imported the HuggingFaceHub. This HuggingFaceHub will allow us to name the Falcon-7B-Instruct mannequin by the Inference API and obtain the responses generated by the mannequin.
• The PromptTemplate is likely one of the components of LangChain, mandatory for constructing purposes based mostly on the Massive Language Mannequin. It defines how the mannequin ought to interpret the person’s questions and in what context it ought to reply them.
• Lastly, we even import the LLMChain from LangChain. LLMChain is the module that chains totally different LangChain parts collectively. Right here we will probably be chaining our Falcon-7B-Instruct Massive Language Mannequin with the PromptTemplate.
• Then we retailer our HuggingFace Inference API in an surroundings variable, that’s,  os.environ[‘API_KEY’]

Instruct the Falcon Mannequin

Now we will probably be inferring the Falcon Instruct mannequin by the HuggingFaceHub module. For this, first, we should present the trail to the mannequin within the Hugging Face. The code for this will probably be

model_id = 'tiiuae/falcon-7b-instruct'

falcon_llm = HuggingFaceHub(huggingfacehub_api_token=os.environ['API_KEY'],
                            repo_id=model_id,
                            model_kwargs={"temperature":0.8,"max_new_tokens":2000})

• First, we should give the id of the mannequin we’ll work with. For us, it is going to be the Falcon-7B-Instruct mannequin. The id of this mannequin could be discovered straight on the HuggingFace web site, which will probably be ‘tiiuae/falcon-7b-instruct’.
• Now we name the HuggingFaceHub module, the place we go the API token, assigned to an surroundings variable, and even the repo_id, i.e., the id of the mannequin we will probably be working with.
• Additionally, we offer the mannequin parameters, just like the temperature and the utmost variety of new tokens. Temperature is how a lot the mannequin ought to be artistic, the place 1 means extra creativity, and 0 tells no creativity.

Now now we have clearly outlined what mannequin we will probably be working with. And the HuggingFace API will allow us to hook up with this mannequin and run our queries to begin constructing our utility.

Immediate Template

After the mannequin choice, the following is defining the Immediate Template. The Immediate Template tells how the mannequin ought to behave. It tells how the mannequin ought to interpret the query offered by the person. It even tells how the mannequin ought to conclude to provide the output to the person’s question. The code for outlining our Immediate Template could be

template = """

You might be an AI assistant that gives useful solutions to person queries.

{query}

"""
immediate = PromptTemplate(template=template, input_variables=['question'])

The above template variable defines and units the context of the Immediate Template for the Falcon mannequin. The context right here is straightforward, the AI wants to offer useful solutions to person queries, adopted by the enter variable {query}. Then this template, together with the variables outlined in it, is given to the PromptTemplate perform, which is then assigned to a variable. This variable is now the Immediate Template, which is able to later be chained along with the mannequin.

Chain Each Fashions

Now now we have each the Falcon LLM and the Immediate Template prepared. The ultimate half will probably be chaining each these fashions collectively. We’ll work with the LLMChain object from the LangChain library for this. The code for this will probably be

falcon_chain = LLMChain(llm=falcon_llm,
                        immediate=immediate,
                        verbose=True)

With the assistance of LLMChain, now we have chained the Falcon-7B-Instruct mannequin with our very personal PromptTemplate that now we have created. We now have even set the verbose = True, which is useful to know what occurs when the code is being run. Now let’s check the mannequin by giving a question to it

print(falcon_chain.run("What are the colours within the Rainbow?"))

Right here, now we have requested the mannequin what the rainbow colours are. The rainbow accommodates VIBGYOR (Violet, Indigo, Blue, Inexperienced, Yellow, Orange, and Pink) colours. The output generated by the Falcon 7B Instruct mannequin is spot on to the query requested. Setting the verbose possibility lets us see the Immediate after formatting and tells us the place the chain begins and ends. Lastly, we’re able to create a UI for our conversational chat utility.

Chainlit – UI for Massive Language Fashions

On this part, we’ll work with Chainlit Package deal to create the UI for our utility. Chainlit is a Python library that lets us construct Chat Interfaces for Massive Language Fashions in minutes. It’s built-in with LangFlow and even LangChain, the library we beforehand labored on. Creating the Chat Interface with Chainlit is straightforward. We now have to write down the next code:

@cl.langchain_factory(use_async=False)

def manufacturing facility():

    immediate = PromptTemplate(template=template, input_variables=['question'])
    falcon_chain = LLMChain(llm=falcon_llm,
                        immediate=immediate,
                        verbose=True)

    return falcon_chain

Steps

• First, we begin with the decorators from Chainlit for LangChain, the @cl.langchain_factory.
• Then we outline a manufacturing facility perform that accommodates the LangChain code. The code right here we’d like is the Immediate Template and the LLMChain module of LangChain, which builds and chains our Falcon LLM.
• Lastly, the return variable should be a LangChain Occasion. Right here, we return the ultimate chain created, i.e., the LLMChain Occasion, the falcon_chain.
• The use_async = False tells the code to not use the async implementation for the LangChain agent.

Let’s Run the Code!

That’s it. Now after we run the code, a Chat Interface will probably be seen. However how is that this potential The factor is, Chainlit takes care of all the things. Behind the scenes, it manages the webhook connections, it’s accountable for making a separate LangChain Occasion(Chain, Agent, and so on) for every person that visits the location. To run our utility, we kind the next within the terminal.

$ chainlit run app.py -w

The -w signifies auto-reload at any time when we make adjustments dwell in our utility code. After coming into this, a brand new tab will get opened with localhost:8000

That is the opening web page, i.e., the welcome display of Chainlit. We see that Chainlit builds a whole Chat Interface for us simply with a single decorator. Let’s strive interacting with the Falcon Mannequin by this UI

We see that the UI and the Falcon Instruct mannequin are working completely high-quality. The mannequin can present swift solutions to the questions requested. It actually tried to elucidate the second query based mostly on the person’s context (clarify to a 5-year-old). That is the start of what we are able to obtain with these Open Sourced Generative AI fashions. With little to few modifications, we are able to have the ability to create far more problem-oriented, actual scenario-based purposes.

Because the Chat Interface is a web site, it’s utterly potential to host it on any of the cloud platforms. We are able to containerize the applying, then attempt to deploy it in any container-based companies in Google Cloud, AWS, Azure, or different cloud companies. With that, we are able to share our purposes with the surface world.

Conclusion

On this walkthrough, now we have seen find out how to construct a easy Chat Software with the brand new Open Supply Falcon Massive Language Mannequin, LangChain, and Chainlit. We now have leveraged these three packages and have interconnected them to create a full-fledged resolution from Code to Working Software. We now have even seen find out how to receive the HuggingFace Inference API Key to entry 1000’s of pre-trained fashions from the HuggingFace library. With the assistance of LangChain, we chained the LLM with customized Immediate Templates. Lastly, with Chainlit, we might create a Chat Software Interface round our LangChain Falcon mannequin inside minutes.

Among the key takeaways from this information embody:

• Falcon is an Open Supply mannequin and is likely one of the highly effective LLm, which is presently on the prime of the OpenLLM Leaderboard
• With Chainlit, it’s potential to create UI for LLM inside minutes
• Inference API lets us hook up with many alternative fashions current within the HuggingFace
• LangChain helps in constructing customized Immediate Templates for the Massive Language Fashions
• Chainlit’s seamless integration with LangChain permits it to construct LLM purposes faster and error-free

Ceaselessly Requested Questions

The media proven on this article will not be owned by Analytics Vidhya and is used on the Creator’s discretion. 

Associated