Noise in integrator's neurons

I am looking into noise in an integrating ensemble to compare the effects of a noisy, encoded input to noise originating within the neurons. In theory, the decoded value should have a Brownian motion – the integral of white noise over time – where the std. dev. of the values should increase proportional to sqrt(t)

This does indeed happen in the first case.
ReLu200-ensemble.pdf (1.2 MB)
ReLu200-ensemble-fit.pdf (18.5 KB)

When noise is injected to neurons, it does not though.
ReLu200-perneuron.pdf (1.3 MB)
ReLu200-perneuron-fit.pdf (18.9 KB)

Instead, the std. dev. jumps up immediately and stays there… almost as if the white noise is doing a “set” instead of an “inc” operation on the .neurons signal. This is a bit worrying because it implies the integrator can remember a value forever even with internal noise.

Any idea what I am doing wrong? Is this expected behavior that I am misinterpreting?

Code excerpt:

A = Ensemble(200)
tau = .1
Connection(A, A, transform=[[1]], synapse=tau) 
if True:
    encoded_noise = WhiteNoise(dist, default_size_out=1)
    Connection(Node(encoded_noise), A, transform=[[tau]], synapse=tau)
else:
    neuron_noise = WhiteNoise(dist, default_size_out=200)
    Connection(Node(neuron_noise), A.neurons)  # equivalent to setting "noise" kwarg of Ensemble

Oh wait, I answered my own problem. Let me try that code excerpt again:

tau = .1
A = Ensemble(200)
Connection(A, A, transform=[[1]], synapse=tau) 
noise_size1 = WhiteNoise(dist, default_size_out=1)
noise_size200 = WhiteNoise(dist, default_size_out=200)
if case==0:
    # Same as before, it shows diffusion
    Connection(Node(noise_size1), A, transform=[[tau]], synapse=tau)
elif case==1:
   # Same as before, it does not show diffusion
    Connection(Node(noise_size200), A.neurons)
    # A.noise = noise_size200  # Equivalent
    # A.noise = noise_size1  # Equivalent because builder will override default_size_out
elif case==2:
   # This is the one I want!
    Connection(Node(noise_size200, A.neurons), 
        transform=tau * np.eye(200), synapse=tau
    )
1 Like