(2 min read)
“Pope’s baby steps on gays*”
What! I can see your jaw drop. Obviously, this sentence is not meant to convey what you thought at first. But, Why did we get two interpretations?
“Pope’s-baby steps on gays” – If we take ‘baby’ as a noun, attach it with ‘pope’ to form a noun compound and take ‘steps’ as a verb, we get that funny meaning!
On the other hand, “Pope’s baby-steps on gays” – If we make a noun compound baby-steps, it gives the intended meaning
Now, in NLP, we need to give this intelligence to the computer…
If this is not complex enough, the language of humans is a hopelessly slow delivery mode of communication. We overcome this & make it efficient by conveying maximum meaning by not saying things…For example, the word ‘tie’ will take three different meanings depending on the context whether it refers to a piece of dress, a result of a sport or joining things.
Thus, the context of a word gives more information than the stand-alone meaning of the word itself. As J.R. Firth says “You shall know a word by the company it keeps“. So it turns out in NLP the paramount problem to solve is accurately predicting the context of a given word.
The other challenge with human languages is, dealing with creative syntaxes. In computer languages like Python, the syntax is precise. It has the Indentations at the right spot. An ‘IF’ is combined with the nearest ‘ELSE’. However, in human language, the ELSE takes whatever the hero feels as appropriate IF.
These two characteristics explain why Human languages are referred to as Natural language. How do we solve these challenges? That, along with how annotation service providers such as NextWealth, with their deep domain capability in NLP are enabling the same, is the subject of my next blog. Let me give you a sneak preview
The basic methods used are
1. Word vectors – converting every word into a numerical vector and
2. Distributional Semantics. It is based on the concept that the context is the set of words that appear nearby (within a fixed-size window). This technology is so exciting & galloping by the day that you wouldn’t want to miss the next blog…would you?
*Source : Time Magazine headline & Stanford