# Noise in integrator's neurons

I am looking into noise in an integrating ensemble to compare the effects of a noisy, encoded input to noise originating within the neurons. In theory, the decoded value should have a Brownian motion – the integral of white noise over time – where the std. dev. of the values should increase proportional to sqrt(t)

This does indeed happen in the first case.
ReLu200-ensemble.pdf (1.2 MB)
ReLu200-ensemble-fit.pdf (18.5 KB)

When noise is injected to neurons, it does not though.
ReLu200-perneuron.pdf (1.3 MB)
ReLu200-perneuron-fit.pdf (18.9 KB)

Instead, the std. dev. jumps up immediately and stays there… almost as if the white noise is doing a “set” instead of an “inc” operation on the .neurons signal. This is a bit worrying because it implies the integrator can remember a value forever even with internal noise.

Any idea what I am doing wrong? Is this expected behavior that I am misinterpreting?

Code excerpt:

``````A = Ensemble(200)
tau = .1
Connection(A, A, transform=[[1]], synapse=tau)
if True:
encoded_noise = WhiteNoise(dist, default_size_out=1)
Connection(Node(encoded_noise), A, transform=[[tau]], synapse=tau)
else:
neuron_noise = WhiteNoise(dist, default_size_out=200)
Connection(Node(neuron_noise), A.neurons)  # equivalent to setting "noise" kwarg of Ensemble
``````

Oh wait, I answered my own problem. Let me try that code excerpt again:

``````tau = .1
A = Ensemble(200)
Connection(A, A, transform=[[1]], synapse=tau)
noise_size1 = WhiteNoise(dist, default_size_out=1)
noise_size200 = WhiteNoise(dist, default_size_out=200)
if case==0:
# Same as before, it shows diffusion
Connection(Node(noise_size1), A, transform=[[tau]], synapse=tau)
elif case==1:
# Same as before, it does not show diffusion
Connection(Node(noise_size200), A.neurons)
# A.noise = noise_size200  # Equivalent
# A.noise = noise_size1  # Equivalent because builder will override default_size_out
elif case==2:
# This is the one I want!
Connection(Node(noise_size200, A.neurons),
transform=tau * np.eye(200), synapse=tau
)
``````
1 Like