Hierarchical Temporal Memory vs. NEF/SPA

General Discussion
1

Continuing a discussion I was having on Hacker News, since topics there get closed after they are too old and this is a discussion that might still be worth having in public.

What we’ve covered so far.

  • What Nengo has been used for.
  • The relation between the NEF and Deep Learning.
  • What neuromorphic hardware currently exists for Nengo and what it is capable of.

What still remains to be determined.

  • The relation (if any) between the NEF and Hierarchical Temporal Memory (HTM) of Numenta. Was Pete Suma working on this or did I imagine that?
  • The utility of spiking, which I may have briefly covered in this other topic.
  • Implementation of Bayesian Inference in the NEF, which might be relevant to @Su.sharma?
2

Hi! I just looked at “utility of spiking” discussion, and I have to say I hoped for more. I guess matching empirical data is a good one. However, can trying to match the data distract us from understanding it?
Spikes are probably used in a brain for two things: computing something, and signalling something. Why not focus on what that “something” might be?
To use an analogy, if we were reverse engineering a microprocessor, we could try to understand CMOS structures and operations, or we could focus on boolean operations, and higher level stuff (e.g. instruction set, register file, ALU, etc).
Trying to model spikes seems closer to CMOS level, which is not really necessary to understand how a computer works. Of course, understanding those higher level functions might be too hard without figuring out the low-level stuff first, but is it the case? I don’t know much about Nengo or Spaun: in the modeling higher level functions of various brain components, are spikes useful in developing the understanding of those functions?

This brings me to the second topic: HTM theory by Numenta is an attempt to understand what brain does without getting bogged down in low-level details. Currently, they claim to understand what’s going on in layers 2/3 and a bit of layer 4 in a single region of neocortex. How much of that is understood/implemented by people behind Spaun?
Also, it feels a little strange that two fairly high profile groups of people, working on essentially the same thing don’t seem to collaborate at all, and might not even be aware of what exactly the other group is doing. Why is that?

p.s. the title of this topic does not really fit this discussion anymore, should we start a new topic?

1 Like
3

Trying to model spikes seems closer to CMOS level, which is not really necessary to understand how a computer works. Of course, understanding those higher level functions might be too hard without figuring out the low-level stuff first, but is it the case? I don’t know much about Nengo or Spaun: in the modeling higher level functions of various brain components, are spikes useful in developing the understanding of those functions?

If I understand your question, I think you’re asking a somewhat similar question to “what is the right level to model cognition at”? The answer to this is that there is no right level, just like when analyzing a processor. If you want to know how the processor is going to behave when installed in a satellite, you start having to go down to the silicon level and investigate how it will react to the type of radiation found in space. When you start wanting to test out the effect of drugs, the biological details of the brain become more important. Only by moving up and down this ladder of abstraction do you get a true understanding for the system.

Currently, they claim to understand what’s going on in layers 2/3 and a bit of layer 4 in a single region of neocortex. How much of that is understood/implemented by people behind Spaun?

With the Neural Engineering Framework, we don’t really discuss layers much, due to the fact that we don’t really know how to map network functionality onto layer functionality. We describe many different areas (13 I think?) at a layer-agnostic level, as we describe in the Spaun paper.

Also, it feels a little strange that two fairly high profile groups of people, working on essentially the same thing don’t seem to collaborate at all, and might not even be aware of what exactly the other group is doing. Why is that?

I actually don’t know. The Numenta people never send anyone to our Nengo Summer School and I’ve never found a good way to get started with their stuff. Can’t say I’ve looked super hard though… They also seemed more focused on pattern recognition, while our lab is focused on a variety of different tasks? I’m not confident about this claim and will continue to investigate.

p.s. the title of this topic does not really fit this discussion anymore, should we start a new topic?

Yeah, I didn’t know what to title this thing, given our discussion had enjoyably spread out. If you suggest a title, I can edit it.

4

This brings me to the second topic: HTM theory by Numenta is an attempt to understand what brain does without getting bogged down in low-level details. Currently, they claim to understand what’s going on in layers 2/3 and a bit of layer 4 in a single region of neocortex. How much of that is understood/implemented by people behind Spaun?

I looked at implementing HTMs in Nengo last year based on their white paper (http://bit.ly/2oix73a). I remember I ran into a road block right off the bat because they use a learning rule that requires non-local information, or something along those lines. It was an issue such that there wasn’t a straightforward mapping to a biologically constrained network. No Nengo implementation as far as I know!

Now I’ve become interested in what exactly it was that stopped me … if I find the specific issue again I’ll update. Or if anyone else has done work on it please let me know!

1 Like
5

Well, we probably should figure out what an ALU and a register file do, and where they are located on a chip, before trying to install it in a satellite! Also, and more importantly, understanding how radiation affects electrons in silicon might have very little overlap with the understanding of what the chip does.
My point is, we still don’t have a high level understanding of how a brain works. For example, I’d like to know how the input is being fed into a brain, how it is encoded (not necessarily at the spike level), then how does it move through various structures and why; how and where it gets stored, how motor commands are generated, how an attention is formed and managed, how the decisions are being made, how do we learn from a single example, how the abstraction hierarchy is formed, and also, why is brain organized in the way it is (biological constraints, randomness of evolution, or some dynamic chaos theory rules)?
To me, this would be similar to the level of knowledge that C code is broken down into simple instructions, stored as binary strings in memory and decoded before execution into control signals, which load data from main memory, cause ALU to manipulate it in one of several possible ways, and write the result back to memory. Now I can use this knowledge to understand how the results of ALU computations are used by controllers (other programs or processors) to generate physical actions, and overall, how to get the system to do something useful for me.

Alternatively, we can take a bottom-up approach: build a detailed simulation of the entire process, so that in the end our model will be able to calculate and recreate every spike in software (or in our specialized hardware). If this method was applied to reverse-engineering a microprocessor (without understanding first how a microprocessor works on a high level), it would be an extremely difficult task, and it would require a lot of resources. That’s what Henry Markram wants to do, and good luck to him.

A common project in undergrad Computer Architecture course is to implement a simple processor in software. The goal usually is to have it execute some short snippet of machine code, without emulating any of the low level details. If two numbers need to be added or multiplied or copied from memory, there’s no need to emulate the internal organization of an adder or multiplier, or simulate DRAM refresh signals. The point of the project is to understand the architecture of the processor. To me, that’s the right abstraction level when trying to understand the brain.

6

Can you expand on that? What’s the difference? Is 2012 Spaun paper still a good reference?

7

That’s strange, as far as I know, pretty much all information processing there is local. Moreover, Numenta claims that everything they do is guided by neuroscience, meaning that biological plausibility is the most important concern to them. This does not apply to some biological details (such as spikes), but they make sure the actual algorithms represent what they believe is happening in the brain. So it’s neither a brain simulator, nor a machine learning model. I’d describe it as an incomplete functional model of cortical layers in a single region.

By the way, that paper is from 2011, the main algorithms (especially temporal memory) have been revised significantly since, you should check out this page for latest information:
http://numenta.com/papers-videos-and-more/

8

The point of the project is to understand the architecture of the processor. To me, that’s the right abstraction level when trying to understand the brain.

Cool! We have the same skepticism about bottom approach that you do. Although we’re a bit hesitant to use metaphors to describe the brain, that’s where the level where the Neural Engineering Framework and the Semantic Pointer Architecture (the tools we used to build Spaun) operate.

Is 2012 Spaun paper still a good reference?

At this point, I would recommend you install the Nengo Neural Simulator and the Nengo GUI and run through the tutorials. They should give you a good idea about where we’re coming from in terms of simulation. When/if you want to learn more about the theoretical grounding, I would recommend either reading Chris’ course notes or “How to Build a Brain”.

Note that this isn’t me sending you away to close the discussion. I actually believe these resources will be the most helpful to you and would be glad to answer any questions you have while investigating them!

9

It is not always easy to judge what can happen in the brain without trying to implement it with (simulated) neurons.

1 Like
10

Of course! That’s why there have been over 10k commits (stable and experimental branches) trying to do just that:


I started reading the course notes, but immediately in the introduction, there’s “Problems with current approaches” section, which describes large-scale neural models (Blue Brain, etc), and cognitve models (ACT-R, etc), yet completely ignores the elephant in the room: HTM. I also looked through the “How to Build a Brain” book, and couldn’t find any mention of Numenta’s work there either. Both Spaun and HTM fall in between detailed brain simulators and cognitive models. In fact it seems that there is no one else in this category!

Here’s what Jeff Hawkins said about Spaun:

I looked into Spaun a while back. Although they call out brain regions such as the Thalamus and Basal Ganglia what I found was that there was very little about any of this that related to real brains. It felt disingenuous to me to use these labels for what they were doing. Things might have changed since then but that was my conclusion then.
http://lists.numenta.org/pipermail/nupic-theory_lists.numenta.org/2014-December/001817.html

I’d like to know what Chris Eliasmith thinks about HTM.

p.s. Sean, we should probably rename this thread to “Spaun vs HTM” or something like that.

11

Although they call out brain regions such as the Thalamus and Basal Ganglia what I found was that there was very little about any of this that related to real brains.

Assuming he’s read our papers on the Basal Ganglia structure, I would like to know what details he felt were important that we left out. We can always use suggestions about what to consider as we continue our research!

I started reading the course notes, but immediately in the introduction, there’s “Problems with current approaches” section, which describes large-scale neural models (Blue Brain, etc), and cognitve models (ACT-R, etc), yet completely ignores the elephant in the room: HTM.

Right. I was hoping you could draw your own conclusions while I go about figuring out what exactly HTM is, Travis double checks that the learning rule they use is biologically plausible and then we could compare notes.

1 Like
12

Hi @michaelklachko, doing a quick browse from their most recent white paper you linked above, some examples of worrying non-plausible simplification:

  1. All of the inputs are binary vectors. Similarly for connections, if the ‘permanence’ value of a synapse is above a threshold, the connection is assigned a value of 1, otherwise 0. As actual system inputs and neural connections are definitely not binary, this makes it pretty hard to model in a biologically plausible neural network.

  2. In the second step of their spatial pooling (Page 35, Phase 2) they find the k most active columns, to apply learning to only these columns. Dynamically, setting up WTA with lateral inhibitory connections is notoriously very tricky, and isn’t something that can be done in a single time step. On top of that, controlling the learning so that it’s only applied after the network has settled on a set of winners is a whole other issue. It might be the case it works running the whole time as the WTA circuit settles, but the dynamics are complex and can’t just be assumed to work.
    (Side note: robust WTA can be implemented using subcortical circuits, e.g. the Nengo Basal Ganglia model, which is an implementation of a circuit detailed by experimentalist work from Gurney, Prescott, and Redgrave)

  3. Another potential issue is their distinction of activation due to lateral or feedforward connections. They put a neuron in a ‘predictive state’ if it receives activity over a lateral connection, and an ‘active state’ if it receives activity over feedforward connections. I’m not sure what this translates to in an actual neural model. Maybe lateral connections modulate the STDP from feedforward connections. But it’s another thing that gets glossed over by not ‘getting bogged down in details’ and would need to be sorted out in a more biologically constrained model.

So, that’s a few of the issues, but probably the best way to get familiar with them is to do your own HTM implementation. :slight_smile:

I started reading the course notes, but immediately in the introduction, there’s “Problems with current approaches” section, which describes large-scale neural models (Blue Brain, etc), and cognitve models (ACT-R, etc), yet completely ignores the elephant in the room: HTM. I also looked through the “How to Build a Brain” book, and couldn’t find any mention of Numenta’s work there either. Both Spaun and HTM fall in between detailed brain simulators and cognitive models. In fact it seems that there is no one else in this category!

I recommend continuing reading! Insomuch as not being mentioned in How to Build a Brain, I don’t understand how HTMs would be considered ‘an elephant in the room’. Can you elaborate on their relevance to a discussion on cognitive models? I’m only familiar with their application to pattern / anomaly detection.

Given an HTM implementation in a neural network, one of the really fun things about Nengo is that we can then plug that right in. So we if there is a neural net implementation of HTMs we can add pattern detection as a feature of Spaun for sure.

Hopefully that helps!

3 Likes
13

Travis, thank you for posting these comments. This is exactly the sort of communication that is lacking between HTM and Nengo communities, and could be beneficial to both.

I think you’re making good points, but I’m neither a neuroscientist nor HTM expert. Let me post this on HTM forum, and see what more knowledgeable people have to say.

It seems that you’re still referring to the old white paper from 2011. In the link I posted (http://numenta.com/papers-videos-and-more/) you can find the most recent papers, including one on spatial pooling.

The reason I mentioned “an elephant in the room” is that I believe that HTM and Spaun have similar goals (unlike those of lower-level detailed brain simulators, or higher level cognitive models), and there’s no other functional brain model operating at this level of abstraction, and implemented in software. Anomaly detection application of Nupic is due to what is currently understood and implemented, however, HTM’s goal is to understand the entire neocortex to the same degree as layers 2/3 are currently understood.

I think there might some confusion about the potential equivalence of HTM, Nupic, Nengo, and Spaun. HTM is a high level theory of neocortex. It describes some of its fundamental operating principles. It is not high level enough to be called a “cognitive model”, because it is focused on understanding functional blocks of neocortex, not on cognition in general. HTM does not really propose any novel ideas, but attempts to combine several known ideas into a coherent functional model. Nupic is the current implementation of HTM in software. It only represents a fraction of what HTM tries to cover. Currently Nupic implements input encoders, layers 2/3, and some of sensori-motor functionality. As more information from the latest neuroscience research becomes available, the Nupic algorithms are adjusted, extended, or discarded.
Nengo, if I understand correctly, is a framework to implement functional brain models. So it’s more like a programming language. Spaun is a functional brain model implemented in Nengo. HTM ideas (Nupic algorithms) can be implemented in Nengo. Some parts of Spaun could probably be implemented as Nupic extensions. Spaun covers more of brain functionality than HTM, but it is still a similar functional model.
I noticed that recent version of Spaun (BioSpaun) is a lot more detailed, so it’s getting a bit closer to brain simulators like Blue Brain. But the original Spaun and Nupic are the only two projects at this abstraction level (if you know any other similar projects please let me know!)

14

Some answers: https://discourse.numenta.org/t/criticism-of-htm-theory/2250

15

Hi Michael,

it’s interesting to take the chance to look closer at things again! edit: heads up I haven’t read the comments in the Numenta thread yet. I’ll try to read them over asap.

OK, I found Numenta’s BAMI manual (which has updates in the algorithm sections as recent as February 2017, so for sure I’ve got the right one this time :smile:) Are there any particular changes in the most recent papers that I should check out?

Jumping into things, relevant to biological plausibility, the BAMI says that HTMs are comfortable leaving out biological detail / constraints and moving to higher levels of abstraction when the lower levels are understood, and I agree that’s fine. In fact, given computational constraints it’s a requirement for exploring higher functioning of the brain with a grounding in biological constraints.

To that end, though, I would want to be certain that (for example) the previous mentioned questions don’t give rise to any problems in implementation under more stringent biological constraints. I’m not aware of such models that resolve the issues, but also there’s more work out there than anyone can keep up with and it’s very possible it was handled a long time ago and I just haven’t seen it. (Please link if you know them I would be very interested to read!)

I agree we that hashing out the equivalences among HTM, Nupic, Nengo, Spaun, and the NEF is a good idea! I’ll try to recap (please forgive the repetition).

Your assessment of HTM matches my understanding after reading more, in that it’s a set of features models of the cortex should contain (e.g. sparse distributed representations, sensory encoders, on-line learning, etc), and not a specific implementation / cortical model.

NuPIC I’m not familiar with, is it fair to say that it is a platform for hooking up / building systems using implementations of some of the networks and circuits that fall into the HTM theory?

A bit of clarification between Nengo and the NEF. The NEF is a set of methods for implementing differential equations in feedforward and recurrent distributed systems. Essentially a means of optimizing the connection weights in a neural network to perform some specific functionality (we refer to it as a ‘neural compiler’ often).

Nengo is a neural simulation platform that provides an API (and GUI) for modellers to build neural models, and then run these simulations on a bunch of different backends (like CPU, GPU, FPGA, neuromorphic hardware, etc). The API lets users take advantage of the NEF, but also supports other network training methods (such as deep-learning, etc). Often it’s characterized as only supporting the NEF but it’s more! :slight_smile:

Spaun is a functional brain model implemented in Nengo. HTM ideas could be implemented in Nengo. Probably parts of Spaun could be implemented in NuPIC. Yup yup yup.

Hmmm I would be careful though not to call HTM a functional model. I think your previous description of it is more accurate, as a sort of set of checkmarks that cortical models need to meet. The BAMI introduction says that some parts of HTMs can be written out formally, but more these are guiding ideas and a lot of it can’t be captured mathematically (…another statement is that computers can’t be described purely mathematically, which I would be interested to hear more about).
Part of a functional model is wedding to some specific implementation details, which HTM doesn’t do. To that end Spaun and HTM aren’t directly comparable. Maybe a more appropriate comparison would be with Dr. Rick Granger’s theory on cortical-subcortical loops? Even that still isn’t really appropriate because it’s a specific algorithm in addition to a general theory of function.

So, of all of the above, I think that comparisons between Nengo and NuPIC are appropriate, and to compare something to Spaun it should be an explicit implementation that generates predictions about the how the brain operates. Maybe this addresses the original question posed?

But as long as I’m going on with a giant rambling post that just addresses various topics without an overarching thesis: this kind of ties in to a larger discussion about the utility of modeling in general.
I was talking with another research group with more of an engineering bent, and presented a model I had worked on to which they said ‘OK, so…what? What does this brain model gain us? It just matches some data that we’d expect any model to match.’ After drying my tears and regaining composure, we came around to what’s really useful about brain models is when they make specific, testable predictions about how the brain operates. Ideally these predictions should be in direct contention with another implemented model, such that an experiment can be devised that will provide support for one underlying set of functions or the other.

Along those lines, I would be very interested to know about HTM implementations that make these kinds of predictions!

1 Like
16

I’d say the main change is a revision of the temporal memory algorithm. Also, the papers attempt to present HTM concepts in a more rigorous way and provide more detail.

And that’s exactly why HTM and Nengo/Spaun people should communicate and check each other’s work!

Yes, you’re right, it’s not a functional model.

Ok, let me clarify this a bit more. Numenta does not have an equivalent to NEF or Nengo. It has not built a “framework”, or “API”, or a “library” for implementing your own functional models or the brain or its components. Nupic is directly comparable to Spaun, or perhaps more accurately to parts of Spaun. Nupic is an implementation of several ideas from HTM. It’s a simplified illustration (or a guess) of how those ideas have been implemented by nature in the layers of neocortex.
So just like you can’t build something “in Spaun”, you wouldn’t be building something “in Nupic”. You can extend it though, and that’s what Numenta is working on. For example, as they get more information about attention mechanisms in the brain, they will extend Nupic to incorporate those mechanisms, abstracting some details. This means a new Nupic algorithms will be written using existing data structures (SDRs), and relying on existing algorithms.
Moreover, if some new evidence from neuroscience experiments comes in, and it directly contradicts any of Nupic assumptions, they will be adjusted (or even thrown out), and the algorithms will be revised to accommodate the new understanding.

Exactly. Nupic’s goal is to generate prediction about the how brain operates. When sensori-motor algorithms are completed, and all the relevant sensory encoders are in place, you should be able to let Nupic control a robot arm, and it should be similar to how a real brain controls a real arm. Obviously, you wouldn’t be able to compare the exact spiking signals going through the robot arm (because there wouldn’t be any spiking signals), however, Numenta expects that the brain-arm artificial system will behave functionally the same as the real thing. The information flow (sensory input and motor commands) should correspond closely. If you want to know more about this, I recommend watching this QA session with Jeff Hawkins, which provides the latest info on sensori-motor integration: