Updated Unit 6 Natural Language Processing Class 10 Notes

Updated Natural Language Processing Class 10

Introduction

In this Unit 6, Natural Language Processing of class 10, Students are introduced to

1. NLP and its importance.
2. Various applications of NLP.
3. Different stages of NLP.
4. Various text processing techniques used in NLP.
5. Different No-Code NLP tools.
6. The Bag of Words model.
7. The concept of TFIDF.
8. No code Orange Data Mining Tool

What is Natural Language?

A natural language is a human language, such as French, Spanish, English, Japanese, etc.

Features of Natural Languages

• They are governed by fixed syntax, lexicon, and semantics.
• All natural languages are redundant.
• All natural languages change over time.

Choose the right word:

1. I am so tired; I want to take a ____________________ (break/brake)

2. Let’s _________________ her a letter. (write/right)

NOTE: Same-sounding words can have totally different meanings in different context.

In Natural Language, it is important to understand that a word can have multiple meanings and the
meanings fit into the statement according to the context of it.

What is Computer Language?

Computer languages are languages used to interact with a computer, such as Python, C++, Java, HTML, etc.

Can computers understand our language?

Yes, but for that we need Natural Language Processing to help computers understand natural language

What is Natural Language Processing?

Natural Language Processing, or NLP, is the sub-field of AI that is focused on enabling computers to understand and process human languages.

Why is NLP important?

Natural Language Processing is all about how machines try to understand and interpret human
language and operate accordingly. Thus, the whole purpose of NLP is to make communication between computer systems and humans possible.

What are the applications of Natural Language Processing?

Some of the applications of Natural Language Processing which are used in the real-life scenario:

1. Voice Assistants: These days Google Assistant, Cortana, Siri, Alexa, etc have become an integral part of our lives. We can talk to them but they also make our lives easier by keeping notes of our tasks, make calls for us, send messages and a lot more. For example:

Hey Google, set an alarm at 3.30 pm
Hey Alexa, play some music
Hey Siri, what’s the weather today

2. Auto generated captions: Captions are generated by turning natural speech into text in real-time. It is a valuable feature for enhancing the accessibility of video content. For example: Auto-generated captions on YouTube and Google Meet.

3. Language Translation: This involves the conversion of text or speech from one language to another For example: Google Translate

4. Automatic Summarization: It is required when we need to access a specific and important piece of information from a huge knowledge base as well as to understand the emotional meanings within the information, such as in collecting data from social media.

5. Sentiment Analysis: Sentiment Analysis is a tool to express an opinion, whether the underlying sentiment is positive, negative, or neutral.. Companies use sentiment analysis, to identify opinion and sentiment online to help them understand what customers think about their products and services.

6. Text Classification: Text classification help to assign predefined categories to a document and organize it to help you find the information you need or simplify some activities. For example, an application of text categorization is spam filtering in email.

7. Chatbots: One of the most common applications of Natural Language Processing is a chatbot. There are a lot of chatbots available. for example

Mitsuku Bot
https://www.pandorabots.com/mitsuku/

CleverBot
https://www.cleverbot.com/

Jabberwacky
http://www.jabberwacky.com/

There are two types of chat bot available around us

1. Script-bot
2. Smart-bot

Difference between Script-bot and Smart-bot

8. Keyword Extraction: Keyword extraction is a tool that automatically extracts the most used, important words and expressions from a text. Customer Service can be improved by using a Keyword extraction tool.

How to extract keyword?

Keyword extraction in NLP involves automatically identifying and extracting the most important words or phrases from a piece of text. Steps are

1. Go to the given website: https://cloud.google.com/natural-language

2. Click on Demo (left panel) and then click on ANALYZE Button.

3. The keywords from the paragraph in the textbox have been highlighted in different colours e.g., Google, Mountain View, etc.

4. Click on other options to check the output.

Stages of Natural Language Processing (NLP)

The stages of Natural Language Processing (NLP) typically involve the following:

1. Lexical Analysis: It is the process of dividing a large chunk of words into structural paragraphs, sentences, and words. Lexicon stands for a collection of the various words and phrases used in a language.

2. Syntactic Analysis / Parsing: It is the process of checking the grammar of sentences and phrases. It forms a relationship among words and eliminates logically incorrect sentences.

3. Semantic Analysis: In this stage, the input text is now checked for meaning, and every word and phrase is checked for meaningfulness. For example: It will reject a sentence that contains ‘hot ice cream’ in it.

4. Discourse Integration: It is the process of forming the story of the sentence. Every sentence should have a relationship with its preceding and succeeding sentences.

5. Pragmatic Analysis: Pragmatic means practical or logical, i.e., this step requires knowledge of the intent in a sentence. It also means to discard the actual meaning of word taken after semantic analysis and take the intended meaning.

Natural Language Processing Class 10

Human Language VS Computer Language

Humans communicate through language which we process all the time. As a person speaks, the sound travels and enters the listener’s eardrum. This sound then converted into neuron impulse and transported to the brain for processing. After processing, the brain gains understanding around the meaning of sound.

The computer understands the language of numbers. Everything that is sent to the machine has to be converted to numbers. And while typing, if a single mistake is made, the computer throws an error and does not process that part. The communications made by the machines are very basic and simple.

Difficulties faced by machine to understand human language :

1. Arrangement of the words and meaning : There are rules in human language which provide structure to a language. There are nouns, verbs, adverbs, adjectives. A word can be a noun at one time and an adjective some other time.

2. Multiple meanings of a word : In natural language, a word can have multiple meanings and the meanings fit into the statement according to the context of it.

3. Perfect Syntax, no Meaning : Sometimes, a statement can have a perfectly correct syntax but it does not mean anything. For example, take a look at this statement:
Chickens feed extravagantly while the moon drinks tea.
This statement is correct grammatically but does not make any sense.

Quiz Time – Natural Language Processing Class 10

Q1. Natural Language Processing majorly deals with__________ processing.

a. Numeric data

b. Textual data

c. Image data

d. Visual data

Q2. ______________is an NLP tool to express an opinion, whether the underlying sentiment is positive, negative, or neutral.

a. Text Classification

b. Machine Translation

c. Sentiment Analysis

d. Automatic Text Summarization

Q3. What is the first stage of Natural Language Processing (NLP)?

a. Semantic Analysis

b. Pragmatic Analysis

c. Lexical Analysis

d. Syntactic Analysis

Q4. Words that we want to filter out before doing any analysis of the text are called__________.

a. Rare words

b. Stop words

c. Frequent words

d. Filter words

Q5. What does discourse integration involve in the context of sentence formation?

a. Identifying individual words in a sentence

b. Forming a coherent story within a sentence

c. Establishing relationships between preceding and succeeding sentences.

d. Applying punctuation and grammar rules to a sentence

Text Processing

How NLP makes it possible for the machines to understand and speak just like humans?

We all know that the language of computers is Numerical, so the very first step that comes to our mind is to convert our language to numbers. This conversion happen in various steps which are given below.

1. Text Normalisation :

In Text Normalisation, we undergo several steps to normalise the text to a lower level. Text Normalisation helps in cleaning up the textual data in such a way that it comes down to a level where its complexity is lower than the actual data. Steps of Text Normalisation are:

a. Sentence Segmentation: In Sentence segmentation, the whole corpus(the whole textual data from all the documents) is divided into sentences.

b. Tokenization : After segmenting the sentences, each sentence is then further divided into tokens. Tokens is a term used for any word or number or special character occurring in a sentence.

c. Removing Stopwords, Special Characters and Numbers : In this step, the tokens which are not necessary are removed from the token list. Some examples of stop words are: a, an, and, are etc.

d. Converting text to a common case : After the stopwords removal, we convert the whole text into a similar case, preferably lower case.

Here in this example, all the 6 forms of hello would be converted to lowercase and hence would be treated as the same word by the machine

e. Stemming : It is the process in which the affixes of words are removed and the words we get after removing affix are called stem which may or may not be meaningful.

f. Lemmatization : In both stemming and lemmatization we remove the affixes of words but the difference between them is that in lemmatization, the word we get after affix removal (also known as lemma) is a meaningful one. It takes longer time to execute than stemming.

NOTE: With this, we have normalised our text to tokens which are the simplest form of words present in the corpus. Now it is time to convert the tokens into numbers. For this, we would use the Bag of Words algorithm.

2. Bag of Words :

In bag of words, we get the occurrences of each word and construct the vocabulary for the corpus. Bag of words gives us two things:

1. A vocabulary of words for the corpus
2. The frequency of these words (number of times it has occurred in the whole corpus).

Here is the step-by-step approach to implement bag of words algorithm:

1. Text Normalisation: Collect data and pre-process it.
2. Create Dictionary: Make a list of all the unique words occurring in the corpus. (Vocabulary)
3. Create document vectors: For each document in the corpus, find out how many times the word from the unique list of words has occurred.
4. Create document vectors for all the documents.

Let us go through all the steps with an example:

Step 1: Collecting data and pre-processing it.

Document 1: Aman and Avni are stressed

Document 2: Aman went to a therapist

Document 3: Avni went to download a health chatbot

Here are three documents having one sentence each. After text normalisation, the text becomes:

Document 1: [aman, and, avni, are, stressed]
Document 2: [aman, went, to, a, therapist]
Document 3: [avni, went, to, download, a, health, chatbot]

Step 2: Create a Dictionary

Go through all the steps and create a dictionary i.e., list down all the words which occur in all three documents:

Dictionary:

NOTE: while creating the dictionary, we create the list of unique words.

Step 3: Create a document vector

In this step, the vocabulary is written in the top row. Now, for each word in the document, if it matches the vocabulary, put a 1 under it. If the same word appears again, increment the previous value by 1. And if the word does not occur in that document, put a 0 under it.

Since, in the first document, we have words: aman, and, avni, are,stressed. So, all thesewords get a value of 1 and the rest of the words get a 0 value.

Step 4: Create document vectors for all the documents.

The same exercise has to be done for all the documents. Hence, the table becomes:

In this table, the header row contains the vocabulary of the corpus and three rows correspond to three different documents. Finally, this gives us the document vector table for our corpus.

3. TFIDF: Term Frequency & Inverse Document Frequency

TFIDF helps un in identifying the value for each word. Let us understand each term one by one.

a. Term Frequency : Term frequency is the frequency of a word in one document. Term frequency can easily be found from the document vector table as in that table we mention the frequency of each word of the vocabulary in each document.

Here, you can see that the frequency of each word for each document has been recorded in the table. These numbers are nothing but the Term Frequencies!

b. Inverse Document Frequency : Document Frequency is the number of documents in which the word occurs irrespective of how many times it has occurred in those documents. To obtain the inverse document frequency, we need to put the document frequency in the denominator while the total number of documents is the numerator.

Finally, the formula of TFIDF for any word W becomes:
TFIDF(W) = TF(W) * log( IDF(W) )

After applying the above formula, the words have been converted to numbers. These numbers are the values of each for each document.

Summary of the concept :

1. Words that occur in all the documents with high term frequencies have the least values and are considered to be the stopwords.
2. For a word to have high TFIDF value, the word needs to have a high term frequency but less document frequency.
3. These values help the computer understand which words are to be considered while processing the natural language. The higher the value, the more important the word is for a given corpus.

Applications of TFIDF : TFIDF is commonly used in the Natural Language Processing domain. Some of its applications are:

1. Document Classification : Helps in classifying the type and genre of a document.
2. Topic Modelling : It helps in predicting the topic for a corpus.
3. Information Retrieval System : To extract the important information out of a corpus
4. Stop word filtering : Helps in removing the unnecessary words out of a text body.

Examples of Code and No-code NLP Tools

Disclaimer : I tried to give you the correct “Natural Language Processing Class 10 Notes” , but if you feel that there is/are mistakes in “Natural Language Processing Class 10 Notes” given above, you can directly contact me at csiplearninghub@gmail.com. The above “Natural Language Processing Class 10 Notes” are created for practice of students and the entire content is from CBSE Study material. Screenshots used in above article “Natural Language Processing Class 10 Notes” is taken from CBSE study material.

SAMPLE PAPER Class 10 – AI(417)

AI CBSE Sample Question Paper 2025-26

AI CBSE Sample Question Paper 2025-26 Marking Scheme

AI CBSE Sample Question Paper 2024-25

AI CBSE Sample Question Paper 2024-25 Marking Scheme

AI CBSE Sample Question Paper 2023-24

AI CBSE Sample Question Paper 2023-24 Marking Scheme

AI CBSE Sample Question Paper 2022-23

AI CBSE Sample Question Paper 2022-23 Marking Scheme

IMPORTANT LINKS Class 10

Unit -1-Revisiting AI Project Cycle & Ethical Frameworks for AI – NOTES

Unit -1-Revisiting AI Project Cycle & Ethical Frameworks for AI – Question Answers

Unit 2. Advanced concepts of Modeling in AI – NOTES

Unit 2. Advanced concepts of Modeling in AI – Question Answers

Unit 3: Evaluating Models NOTES

Unit 3: Evaluating Models Question Answers

Natural Language Processing Class 10

Natural Language Processing Class 10

v v v v v v v v v v v v v v v v v v v v v v v v v v v v v vv v v v v v v v v v v v v v