Dot product calculates degree of similarity between two neural states (two S-pointers). Compared with other basic “neural computations” like e.g., calculation of a binding S-pointer for two states, I have no information, how the dot product is calculated “in reality” on the neural level. The dot product can simply be calculated at the mathematical S-pointer level but is there a basic neural network configuration which would allow to calculate a dot product? This is important for example in case of action selection within the BG-Thal-cortico-cortical control loop.
I know the answer in case of binding: There is a neural configuration of four buffers which easily allows the calculation of a binding S-pointer (or at the neural level: the activation of a new neural activation pattern representing the binding state).
But how does that take place for calculating similarities e.g. in case of the BG-Thal-descision cycle for finding the next action S-pointer?
hey guys
I am still waiting here on an answer since 22 days now 
so: Dot products are calculated only on the mathematical level of semantic pointers and NOT in a neural representation?
That would be an enormous negative point for the NEF-SPA concept because dopt products are an immense important and central concept for action selection.
And action selection is a central concept for each large-scale or brain-scale neural model.
So: If you really never implemented DOT products in a neural realization , please develop a possible neural concept fr DOT product calculation like you have done it for binding and for unbinding etc.
Kind regards
Bernd
I don’t have (all) the proper references at hand, but you start with computing the product of just two numbers. This is realized fairly easily with the NEF. One can do some things, described in this technical report to improve accuracy, but they are not strictly necessary. Once, you can multiply two numbers, just do this as many times as the dimensionality of your vectors requires and you get the dot product. In fact, the Nengo implementation already has a parameter for the dimensionality. Again, further optimizations for Semantic Pointers are possible with methods described in chapter 6 of my thesis or you can look at the implementation in Nengo SPA for the end result.
So computing a dot product can definitely be implemented in neurons. Whether the brain is doing it in exactly that way is another question. Maybe it is using some sort of dendritic computation or similar for that? I think @astoecke did some work on such computations and might be able to say more about that aspect.
Just to repeat what Jan says: the NEF is a method for constructing spiking neural networks that implement some desired function, including, but not limited to, closed-form mathematical expressions such as the dot product. So, in the end, if you build an SPA network with a dot product in Nengo, then it will “compile” all of that into a spiking neural network that only consists of synaptic filters, spiking neurons, and the connection strengths between synapses and neurons.
It is up to you, the modeller, to specify the neural constraints that you think reflect the biological constraints you want to take into account (number of neurons, inter-neuton connectivity, spike rates, excitatory or inhibitory synapses, etc.). You then ask Nengo to optimally realize representations and transformations using those constraints; if the resulting network works, then this is a good litmus test that the network could indeed implement the function you were asking for. You would have to compare the model predictions (i.e., neural activities for certain tasks, or predicted response times, etc.) to actual neurophysiological measurements to test whether the Nengo implementation is actually plausible.
Some of the work that I’ve been doing for my thesis (see Chapter 3; CNRGlab @ UWaterloo | Publications) has been extending the NEF to be able to harness excitatory and inhibitory conductance-based synapses to perform multiplication. That allows you to incorporate more neurobiological constraints into the network and to get away with simpler inter-neuron connectivity, that, in some cases (such as cortical attention mechanisms), should be closer to what neurobiologists would expect.
As far as I am aware, the models that are in Nengo for the Basal Ganglia-Thalamus-Cortex loop are already quite biologically plausible in terms of all of the biological constraints listed above.
thank you