Problem when transforming with cosine

Hi,

I have a fairly basic question… I have the following model:

model = nengo.Network()
with model:
    
    stim_teta1 = nengo.Node(Piecewise({0: 0, 1: 1.57}))
    stim_teta2 = nengo.Node(Piecewise({0: 0, 2: 1.57}))
    
    teta = nengo.Ensemble(n_neurons=600, 
                          dimensions=2, 
                          radius=np.sqrt(2 * 6.28**2), 
                          seed = 0)
                          
    nengo.Connection(stim_teta1, teta[0])
    nengo.Connection(stim_teta2, teta[1])
    
    res_x = nengo.Ensemble(n_neurons=600, 
                           dimensions=1, 
                           radius=2,
                           seed = 0)
                           
    def calculate_x(teta):
        **return np.cos(teta[0]) + np.cos(teta[1])**  
        
    nengo.Connection(teta, res_x, function=calculate_x)

You would expect values of 2, 1, 0 in the res_x ensemble. However, I get the following:

I’m sure that there is an easy fix for it… and would appreciate your help…

Hi Elishai,

I ran your model, and confirmed that the circuit is correct by adding
model.config[nengo.Ensemble].neuron_type = nengo.Direct()
before the with model: line, which runs the circuit without simulating neurons.

Because your circuit is set up properly, we know that the neural representation is introducing the error.

  • The easiest thing to try is increasing the number of neurons in the ensemble. But even with 5k neurons I’m not getting great output.
  • [incorrect comment about radii setting edited out, see below instead!]

Thank you Travis for your prompt response.

  1. Great TIP regarding the nengo.Direct! Never heard or read about this option before.

  2. Can you help me understand the radius? This is a two dimensional ensemble, which has to represent two 2*pi inputs. Why isn’t the radius is sqrt(2pi^2 + 2pi^2)=8.88?

If I will change the inputs to:
stim_teta1 = nengo.Node(Piecewise({0: 0, 1: 2*np.pi}))
stim_teta2 = nengo.Node(Piecewise({0: 0, 2: np.pi}))

Then with your proposed radius it seems that we can’t represent the inputs since the radius is not big enough:

My first guess would be to find better encoders. What do you think?

Thanks!
Elishai

Elishai! I’m sorry, the radius that I said above is incorrect. You are correct, I’ll modify my response.

There are a number of ways to improve the representation, as you mention we could change the encoders, intercepts, and eval_points of the neurons to focus on the range 0-2*pi, but I normally implement things another way when the range of values that I want to represent is not symmetric over 0.

If you instead set radius = np.sqrt(n_dims) and scale the input signals to each dimension such that they’re between -1 and 1 then you make full use of the default range represented by the ensemble. As things are currently, setting the radius to represent up to 2*np.pi along each dimension, the ensemble is optimized to represent the whole range -2*np.pi to 2*np.pi, half of which you’re not using. (i’m assuming you only want to use the 0-2*pi range).

I’ve attached code below, with this change the accuracy of the representation improves a fair bit. I still had to increase the number of neurons to 3k though. The extra ensemble I added was just to have the direct mode answer plot at the same time as the spiking estimate.

import numpy as np
import nengo
from nengo.processes import Piecewise

model = nengo.Network()
with model:

def scale_down(x):
    return (x - np.pi) / (np.pi)
stim_teta1 = nengo.Node(Piecewise({0: scale_down(0), 1: scale_down(2*np.pi)}))
stim_teta2 = nengo.Node(Piecewise({0: scale_down(0), 2: scale_down(np.pi)}))

teta = nengo.Ensemble(
    n_neurons=3000,
    dimensions=2,
    radius=np.sqrt(2),
    seed = 0
)
teta_direct = nengo.Ensemble(
    n_neurons=1,
    dimensions=2,
    neuron_type=nengo.Direct(),
)

res_x = nengo.Ensemble(
    n_neurons=600,
    neuron_type=nengo.Direct(),
    dimensions=2,
    radius=2,
    seed = 0
)

for ii, ens in enumerate([teta, teta_direct]) :
    nengo.Connection(stim_teta1, ens[0])
    nengo.Connection(stim_teta2, ens[1])

    def scale_up(x):
        return x * np.pi + np.pi
    def calculate_x(x):
        return np.cos(scale_up(x[0])) + np.cos(scale_up(x[1]))

    nengo.Connection(ens, res_x[ii],  function=calculate_x)

Brilliant. Thank you!

One last thought, how can I distribute encoders throughout 0 to 2pi, eliminating the ones which lies at negative side? I tried using encoders=uniform(0, 2np.pi), but the result was fairly noisy.

Thanks again, Elishai

Hi Elishai,

I do recommend transforming the input into the -1 to 1 range, you get into weird effects trying to morph the neurons to just represent the first quadrant. That said, you can do something like the following

import numpy as np
import nengo
from nengo.processes import Piecewise

seed = np.random.randint(1e5)
print('seed: ', seed)
np.random.seed(seed) 

model = nengo.Network()
with model:
    stim_teta1 = nengo.Node(Piecewise({0: 0, 1: 1.57}))
    stim_teta2 = nengo.Node(Piecewise({0: 0, 2: 1.57}))
    
    teta = nengo.Ensemble(
        n_neurons=3000,
        dimensions=2,
        radius=np.sqrt(2 * 6.28**2),
        eval_points=nengo.dists.Uniform(0, 1),
    )
    
    teta.encoders = nengo.dists.UniformHypersphere().sample(d=2, n=teta.n_neurons)
    teta.intercepts = nengo.dists.Uniform(0, 1).sample(teta.n_neurons)
    indices, _ = np.where(teta.encoders < 0)
    teta.intercepts[indices] *= -1
    
    teta_direct = nengo.Ensemble(
        n_neurons=1,
        dimensions=2,
        neuron_type=nengo.Direct(),
    )
    
    res_x = nengo.Ensemble(
        n_neurons=600,
        neuron_type=nengo.Direct(),
        dimensions=2,
        radius=2,   
    )
    
    for ii, ens in enumerate([teta, teta_direct]) :
        nengo.Connection(stim_teta1, ens[0])
        nengo.Connection(stim_teta2, ens[1])
    
        def calculate_x(x):
            return np.cos(x[0]) + np.cos(x[1])
    
        nengo.Connection(ens, res_x[ii],  function=calculate_x)

the things to note here are

  • the encoders are just set the same way they are usually, but we’re explicitly calculating them beforehand so that we can
  • set the intercepts such that the neurons are active inside the quadrant that we’re interested in, and then
  • specifying the eval_points to only sample from quadrant 1, otherwise you try to optimize your decoders over all 4 quadrants and your representation suffers because you don’t care about any of those.

if you run this you’ll get improved representation, but still not as good as normalizing the input signal to -1 to 1 along each dimension and then scaling it up in your decoding function.
Hope that helps!

Thank you Travis for your prompt help.