Work Package*: *WP1

Programme*: *P1

Deliverable*: *Deliverable 1.1: “Development of an adaptive control method”

Deliverable due date: month 18

This document reports on the progress of work on Deliverable 1.1:
“P1 will include insights from active learning, or experimental design, and stochastic optimal control theory together with state of the art techniques from computational neuroscience to develop a theoretical framework and practical algorithms.”

All the planned tasks related to this Deliverable have been accomplished. Obtained results have been submitted to “IEEE Transactions on Signal Processing” peer-reviewed journal in November 2015 and is under peer-review. The publication is a scientific part of the Deliverable 1.1 and this “Project Deliverable Report” document focuses on its relation to the NETT proposal.

In the article we described the implementation of an adaptive importance sampling scheme using a newly developed theoretical framework to learn feedback controllers. We applied this importance sampling method to estimate the posterior probability over processes given a time-series of noisy observations. The estimation of this posterior distribution, called smoothing distribution, is needed for parameter estimation, for instance to learn a time-series model parametrized by neural networks.

We were able to improve the sampling efficiency for the smoothing distribution by a factor of at least 25, for instance from 2% to 50% in one of the examples in the mentioned article. In addition, the estimation of the smoothing distribution using our method has less variability than the state-of-the-art method and are computationally more efficient. The decrease in the computational complexity needed to estimate the smoothing distribution opens new possibilities for learning model of complex high-dimensional processes, for instance the estimation of connectivity in neural networks.

In such a way we accomplish Deliverable 1.1. The second part of P1 “Integration of stochastic optimal control principles in learning neural networks” is work in progress.

Contributors: Hans-Christian Ruiz, Bert Kappen (RU)