Natural Language Processing with Python

This book offers a highly accessible introduction to natural language processing, the field that supports a variety of language technologies, from predictive text and email filtering to automatic summarization and translation. With it, you'll learn how to write Python programs that work with large collections of unstructured text. You'll access richly annotated datasets using a comprehensive range of linguistic data structures, and you'll understand the main algorithms for analyzing the content and structure of written communication.

Packed with examples and exercises, Natural Language Processing with Python will help you:

Extract information from unstructured text, either to guess the topic or identify "named entities"
Analyze linguistic structure in text, including parsing and semantic analysis
Access popular linguistic databases, including WordNet and treebanks
Integrate techniques drawn from fields as diverse as linguistics and artificial intelligence

This book will help you gain practical skills in natural language processing using the Python programming language and the Natural Language Toolkit (NLTK) open source library. If you're interested in developing web applications, analyzing multilingual news sources, or documenting endangered languages -- or if you're simply curious to have a programmer's perspective on how human language works -- you'll find Natural Language Processing with Python both fascinating and immensely useful.

Preface;
Audience;
Emphasis;
What You Will Learn;
Organization;
Why Python? ;
Software Requirements;
Natural Language Toolkit (NLTK);
For Instructors;
Conventions Used in This Book;
Using Code Examples;
Safari® Books Online;
How to Contact Us;
Acknowledgments;
Royalties;
Chapter 1: Language Processing and Python;
1. 1 Computing with Language: Texts and Words;
1. 2 A Closer Look at Python: Texts as Lists of Words;
1. 3 Computing with Language: Simple Statistics;
1. 4 Back to Python: Making Decisions and Taking Control;
1. 5 Automatic Natural Language Understanding;
1. 6 Summary;
1. 7 Further Reading;
1. 8 Exercises;
Chapter 2: Accessing Text Corpora and Lexical Resources;
2. 1 Accessing Text Corpora;
2. 2 Conditional Frequency Distributions;
2. 3 More Python: Reusing Code;
2. 4 Lexical Resources;
2. 5 WordNet;
2. 6 Summary;
2. 7 Further Reading;
2. 8 Exercises;
Chapter 3: Processing Raw Text;
3. 1 Accessing Text from the Web and from Disk;
3. 2 Strings: Text Processing at the Lowest Level;
3. 3 Text Processing with Unicode;
3. 4 Regular Expressions for Detecting Word Patterns;
3. 5 Useful Applications of Regular Expressions;
3. 6 Normalizing Text;
3. 7 Regular Expressions for Tokenizing Text;
3. 8 Segmentation;
3. 9 Formatting: From Lists to Strings;
3. 10 Summary;
3. 11 Further Reading;
3. 12 Exercises;
Chapter 4: Writing Structured Programs;
4. 1 Back to the Basics;
4. 2 Sequences;
4. 3 Questions of Style;
4. 4 Functions: The Foundation of Structured Programming;
4. 5 Doing More with Functions;
4. 6 Program Development;
4. 7 Algorithm Design;
4. 8 A Sample of Python Libraries;
4. 9 Summary;
4. 10 Further Reading;
4. 11 Exercises;
Chapter 5: Categorizing and Tagging Words;
5. 1 Using a Tagger;
5. 2 Tagged Corpora;
5. 3 Mapping Words to Properties Using Python Dictionaries;
5. 4 Automatic Tagging;
5. 5 N-Gram Tagging;
5. 6 Transformation-Based Tagging;
5. 7 How to Determine the Category of a Word;
5. 8 Summary;
5. 9 Further Reading;
5. 10 Exercises;
Chapter 6: Learning to Classify Text;
6. 1 Supervised Classification;
6. 2 Further Examples of Supervised Classification;
6. 3 Evaluation;
6. 4 Decision Trees;
6. 5 Naive Bayes Classifiers;
6. 6 Maximum Entropy Classifiers;
6. 7 Modeling Linguistic Patterns;
6. 8 Summary;
6. 9 Further Reading;
6. 10 Exercises;
Chapter 7: Extracting Information from Text;
7. 1 Information Extraction;
7. 2 Chunking;
7. 3 Developing and Evaluating Chunkers;
7. 4 Recursion in Linguistic Structure;
7. 5 Named Entity Recognition;
7. 6 Relation Extraction;
7. 7 Summary;
7. 8 Further Reading;
7. 9 Exercises;
Chapter 8: Analyzing Sentence Structure;
8. 1 Some Grammatical Dilemmas;
8. 2 What's the Use of Syntax? ;
8. 3 Context-Free Grammar;
8. 4 Parsing with Context-Free Grammar;
8. 5 Dependencies and Dependency Grammar;
8. 6 Grammar Development;
8. 7 Summary;
8. 8 Further Reading;
8. 9 Exercises;
Chapter 9: Building Feature-Based Grammars;
9. 1 Grammatical Features;
9. 2 Processing Feature Structures;
9. 3 Extending a Feature-Based Grammar;
9. 4 Summary;
9. 5 Further Reading;
9. 6 Exercises;
Chapter 10: Analyzing the Meaning of Sentences;
10. 1 Natural Language Understanding;
10. 2 Propositional Logic;
10. 3 First-Order Logic;
10. 4 The Semantics of English Sentences;
10. 5 Discourse Semantics;
10. 6 Summary;
10. 7 Further Reading;
10. 8 Exercises;
Chapter 11: Managing Linguistic Data;
11. 1 Corpus Structure: A Case Study;
11. 2 The Life Cycle of a Corpus;
11. 3 Acquiring Data;
11. 4 Working with XML;
11. 5 Working with Toolbox Data;
11. 6 Describing Language Resources Using OLAC Metadata;
11. 7 Summary;
11. 8 Further Reading;
11. 9 Exercises;
Afterword: The Language Challenge;
Language Processing Versus Symbol Processing;
Contemporary Philosophical Divides;
NLTK Roadmap;
Envoi. . . ;
Bibliography;
NLTK Index;
General Index;
Colophon;