All the pieces it is advisable to know to make use of CountVectorizer effectively in Sklearn
Essentially the most primary information processing that any Pure Language Processing (NLP) venture requires is to transform the textual content information to the numeric information. So long as the info is in textual content type we can’t do any type of computation motion on it.
There are a number of strategies out there for this text-to-numeric information conversion. This tutorial will clarify one of the vital primary vectorizers, the CountVectorizer methodology within the scikit-learn library.
This methodology could be very easy. It takes the frequency of prevalence of every phrase because the numeric worth. An instance will make it clear.
Within the following code block:
- We are going to import the CountVectorizer methodology.
- Name the tactic.
- Match the textual content information to the CountVectorizer methodology and, convert that to an array.
import pandas as pd
from sklearn.feature_extraction.textual content import CountVectorizer #That is the textual content to be vectorized
textual content = ["Hello Everyone! This is Lilly. My aunt's name is also Lilly. I love my aunt.
I am trying to learn how to use count vectorizer."]
cv= CountVectorizer()
count_matrix = cv.fit_transform(textual content)
cnt_arr = count_matrix.toarray()
cnt_arr
Output:
array([[1, 1, 2, 1, 1, 1, 1, 2, 1, 2, 1, 2, 1, 1, 2, 1, 1, 1]],
dtype=int64)
Right here I’ve the numeric values representing the textual content information above.
How do we all know which values characterize which phrases within the textual content?
To make that clear, it is going to be useful to transform the array right into a DataFrame the place column names would be the phrases themselves.
cnt_df = pd.DataFrame(information = cnt_arr, columns = cv.get_feature_names())
cnt_df
Now, it reveals clearly. The worth of the phrase ‘additionally’ is 1 which suggests ‘additionally’ appeared solely as soon as within the check. The phrase ‘aunt’ got here twice within the textual content. So, the worth of the phrase ‘aunt’ is 2.
Within the final instance, all of the sentences have been in a single string. So, we received just one row of information for 4 sentences. Let’s rearrange the textual content and…
