Stochastic Systems Group  

Inference and Learning in Largescale Relational Conditional Random Fields
Andrew McCallum
University of Massachusetts Amherst
Advances in machine learning have enabled the research community to build fairly accurate models for individual components of a natural language processing system, such as noun phrase segmentation, named entity recognition and entity resolution. However there has been significantly less success stitching such components together into a useful, highaccuracy endtoend system. This is because errors cascade and compound in a pipelinefor example, six components each having 90% accuracy may have only about 50% accuracy when pipelined.
In this talk I will describe work in largescale, relational conditional random fields that perform joint inference across multiple components of an information processing pipeline in order to avoid the brittle accumulation of errors. In a single factor graph we seamlessly integrate multiple task components using our new probabilistic programming language to compactly express complex, mutable variablefactor structure both in firstorder logic as well as in more expressive Turingcomplete imperative procedures. We avoid unrolling this relational graphical model by using Markovchain Monte Carlo for inference, and make inference more efficient with learned proposal distributions. Parameter estimation is performed by a method we call SampleRank, which avoids complete inference as a subroutine by learning simply to correctly rank successive states of the Markovchain.
Joint work with colleagues at UMass: Charles Sutton, Aron Culotta, Khashayar Rohanemanesh, Chris Pal, Greg Druck, Karl Schultz, Sameer Singh, Pallika Kanani, Kedare Bellare, Michael Wick, and Rob Hall.
Problems with this site should be emailed to jonesb@mit.edu