Natural language processing and how Power BI uses it
Natural language refers to the way we, humans, communicate with each other either in speech form or text.
In today’s world, only 21% of data is in structure form. Most of the data is in text form like whatever we tweet, message or comment, etc. which is unstructured.
Computers are great at working with structured data like database tables and financial records. They are able to process that data much faster than humans. But we don’t communicate in “structured data” form. We, humans, communicate using words which is an unstructured form.
But there is no standardized technique to work with unstructured data. We program computers using C++, Python or R, we just give computers some set of rules that it should operate by.
NLP is sub-field of AI that is focused on enabling computers to understand and process human language. It is basically an ability of a computer to understand, analyze, manipulate, and potentially generate human language. In general terms, NLP tasks break down language into shorter, elemental pieces, try to understand relationships between the pieces and explore how the pieces work together to create meaning.
Components of NLP:
• Natural Language Understanding
- It uses computer software to understand input language in the form of text/speech
- It basically understands what a speaker actually means.
• Natural Language Generation
- Translate computer’s artificial language into text/audible speech.
- Basically, turning pieces of information into human language.
Major steps involved:
• Entity extraction
Entity extraction involves segmenting a sentence to identify and extract entities, such as a person (real or fictional), organization, date, geographies (country, cities, states, etc.), events, etc.
E.g.: Amazon is an e-commerce company based in Washington.
Here NLP will identify Amazon as an organization and Washington as a country.
• Syntactic analysis
Syntactic analysis involves parsing a sentence to understand the grammar and co-relation between words in that sentence. This step does a ‘context-free grammar check’, which is independent of the overall context of the text. It refers to the proper ordering of words. It forms in a tree format.
e.g.: The dog saw a man in the park.
- At first, the sentence will be divided into two parts i.e. noun phrase and verb phrase.
- Then from noun phrase, it identifies determiners and noun and from verb phrase, it will fine determiners, noun, and prepositions.
- ‘The’ will be classified as determiners and ‘dog’ as noun in the noun phrase and in verb phrase ‘saw’ will be classified as verb, ‘a’ as determiner, ‘man’ as noun, ‘in’ as preposition, ‘the’ as determiners and ‘park’ as noun.
• Semantic analysis
Semantic analysis finds the meaning of the sentence in a context-free form as an independent sentence. The sentence is treated as independent of overall paragraph/text. On the basis of knowledge based on grammar and entities, it finds the meaning of the text.
• Sentiment Analysis
Once the syntactic and semantic analysis has been completed, we need to understand the sentiment behind every sentence. There are two terms that are widely used to understand the sentiment of a statement:
• Polarity: In this part -1 and +1 that is used to understand the negativity or positivity of a sentence respectively.
• Magnitude: The value of the magnitude ranges from 0 to infinity that signifies the weight of the assigned polarity.
• Pragmatic analysis
Pragmatic analysis involves analyzing the statement in relation to the preceding or succeeding statements or even the overall paragraph in order to understand its meaning.
For example, I cooked lunch today. However, I forgot to have it.
In this case, the pronoun ‘it’ is referring to ‘lunch’. For the system to understand what I actually ‘forgot’, it is important for it to understand the earlier statement.
Applications of NLP:
1. Google Assistant
2. Spam filter in mails
3. Reviews of any application
4. Google Translation
5. Text analysis
PowerBI has recently made the ‘Ask a Question’ feature inside the Power BI desktop. This allows to directly place the ask a question box in your reports. This helps in finding useful insights from the data. This feature also auto-generate sample suggestions to help one quickly get started. It provides the result of the data in Power BI in the form of charts and graphs.
Power BI Q&A only supports answering queries asked in English. Q&A feature looks for answers in all of the datasets that are updated in the dashboard.
To get the correct answer to your question one should keep in mind a few essential steps.
1. At first, add all the missing relationships between tables to get the correct answer.
2. Rename tables and column so that questions can be asked in an easy way.
3. Fix incorrect data types. E.g. Date, country, etc. should be in the correct format.
Power BI uses NLP for this question and answers feature to help one to understand data in an easier form.
Nowadays NLP make life easy for us by all these applications through which we can easily understand and do several things.
Business Analyst at NeenOpal Intelligent Solutions Pvt Ltd @Ayushi Sharma
Machine Learning based Forecasting in Supply Chain
March 28, 2018
Predictive Customer Lifetime Value
March 28, 2018