A new approach to information processing with filters and pulses

This thesis presents a hierarchical network for the analysis of structured, complex time signals. This network can be viewed as a new kind of artificial neural network working with time signals, without extra synchronization. It consists of multiple layers of feature-detection filters. We focus on pulse-domain processing, where we define an inner-product filter, looking for some pulse pattern in its input signal. The filters work by projecting the input into a one-dimensional subspace, and producing a pulse if the projection exceeds some threshold. They can be implemented efficiently using simple, forward only recursions. Among other models, the filters can be built with biologically plausible neurons. We demonstrate that inner-product filters work well with pulse-domain signals, by implementing two examples of networks for Morse code parsing. We also show how learning can be done in such networks. We first propose algorithms for supervised learning. Their main ingredient is a new type of backpropagation, where starting on the highest layer, “good” pulse positions for the intermediate layers are recursively defined and used in the cost function. We evaluate the performance of the algorithm with examples of 3 and 4-layer networks, which can distinguish 20 piano tunes. We conclude the topic with proposals for unsupervised learning. Finally, we broaden the framework by introducing loops, and emphasize the potential of such a recurrent network.