In the last session, we talked about extracting words from bodies of text and how the different parts of
Speech each have their uses for different semantic analysis tasks. As we begin to look at Concept Extraction, we see that a deeper level of understanding of language, beyond merely recognizing and labeling Parts of Speech, is necessary to extract meaningful chunks of information from a corpus of text. We need to look at more complex theories of language that take into account the relationships between words within a language. These linguistic theories include Context-Free Grammars, Construction Grammars, Dependency and Link Grammars, and Frame Semantics. They allow us to treat unstructured text as more than just a "bag of words", recognizing that the way we string words together is what gives them context- a whole layer of additional complexity that we would not have to deal with if we were only interested in analyzing numerical data. For this reason, we must take a lengthy detour into the field of linguistics in this session.