Stochastic Systems Group  

Adaptive Sensing in Uncertain Environments: Maximum Likelihood, Sensor Networks, and Reinforcement Learning
Doron Blatt
University of Michigan, Ann Arbor
The emerging technologies of wireless sensor networks and multimodal sensing systems that collect data in multiple locations and through a variety of sensing modalities have brought about new and exciting applications as well as challenges to the field of signal processing. This talk addresses two of the challenges faced in designing such systems: (1) Performing inference under energy and bandwidth constraints that limit the amount of information that can be shared by the system elements and (2) Finding optimal sensor scheduling policies under a resource allocation constraint that precludes using all data collection modalities at all times.
First, this talk presents the incremental aggregated gradient (IAG) method for performing inference via local information sharing in wireless sensor networks. The gradient aggregation concept relaxes a common requirement for a diminishing step size and a fast convergence rate is established. Like other local search methods, the IAG method may converge to a local optimum. To mitigate this weakness, the following question is addressed: Given a location of a relative maximum of the loglikelihood function, how to assess whether this is the global maximum? This talk analyzes an existing statistical tool, called A Test for Global Maximum, that answers this question by posing it as a hypothesis testing problem. Tests that are insensitive to model mismatch are proposed, thereby overcoming a fundamental weakness of this tool.
Second, the resource allocation problem in multimodal sensing systems is formulated as the Sequential Choice of Experiments problem, and its modelfree instance is formulated as a Reinforcement Learning problem. Optimal policies are then approximated via a reduction to a sequence of supervised learning subproblems. A simulation and an experiment with real data demonstrate the promise of our approach.
Problems with this site should be emailed to jonesb@mit.edu