Piero Scaruffi

These are excerpts and elaborations from my book "The Nature of Consciousness"

Hierarchical Belief Networks This school of thought merged with another one that was coming from a background of statistics and neuroscience. The Swedish statistician Ulf Grenander (who in 1972 had established the Brown University Pattern Theory Group) fostered a conceptual revolution in the way a computer should describe knowledge of the world: not as concepts but as patterns. His "general pattern theory" provided mathematical tools for Identifying the hidden variables of a data set. Grenander's pupil David Mumford studied the visual cortex and came up with a hierarchy of modules in which inference is Bayesian and it is propagated both up and down ("On the computational architecture of the neocortex II", 1992). a feedforward chain of modules in successively higher The assumption was that feedforward/feedback loops in the visual region integrate top-down expectations and bottom-up observations via probabilistic inference. Basically, Mumford applied hierarchical Bayesian inference to model how the brain works. Hinton's Helmholtz machine of 1995 was de facto an implementation of those ideas: an unsupervised learning algorithm to discover the hidden structure of a set of data based on Mumford's and Grenander's ideas. The hierarchical Bayesian framework was later refined with Tai Sing Lee of Carnegie Mellon University ("Hierarchical Bayesian inference in the visual cortex", 2003). These studies were also the basis for the widely-publicized "Hierarchical Temporal Memory" model of the startup Numenta, founded in 2005 in Silicon Valley by Jeff Hawkins, Dileep George and Donna Dubinsky. It was another path to get to the same paradigm: hierarchical Bayesian belief networks.   Back to the beginning of the chapter "Connectionism and Neural Machines" | Back to the index of all chapters