How to normalize the nengo_spa.State?

Semantic Pointer Architecture
1

Hi.
I want to ask you the way to normalize the norm of the vector which nengo_spa.State represents.

import nengo_spa as spa

def agent_input(t):
    return 'Dog'

def verb_input(t):
    return 'Chase'

def obj_input(t):
    return 'Boy'

d = 16
model = spa.Network()

with model:
    agent_inp = spa.Transcode(agent_input, output_vocab=d)
    verb_inp = spa.Transcode(verb_input, output_vocab=d)
    object_inp = spa.Transcode(object_input, output_vocab=d)
    sentence = spa.State(d)
    agent = spa.State(d)
    verb = spa.State(d)
    object = spa.State(d)
    agent_inp >> agent
    verb_inp >> verb
    object_inp >> object
    spa.sym.Agent*agent + spa.sym.Verb*verb + spa.sym.Object * object >> sentence

In this case, the norm of the vector in sentence represents exceeds 1.
What is the way to normalize this?

2

In SPA models we generally rely on the neuron saturation behaviour to provide a soft normalization. Because LIF neurons “flatten out” for higher and higher input values, trying to represent vectors with magnitude significantly greater than 1 will all tend to result in the same representation (with magnitude ~1). You can change where that saturation happens by playing with the Ensemble’s radius parameter.

If you want an exact normalization you can use nengo.Node (e.g. nengo.Node(lambda t, x: x / np.linalg.norm(x))). The output of that node will always be exactly normalized (but it’s not being done in a biologically plausible way, if that is important to you).

3

A quick addendum: the semantic pointer (SP) class has a normalize method and a unitary method. It’s also useful to know that any unitary vector bound with another unitary vector will maintain unit length after binding; this property can be leveraged to help control the lengths of vectors.