I am looking into noise in an integrating ensemble to compare the effects of a noisy, encoded input to noise originating within the neurons. In theory, the decoded value should have a Brownian motion – the integral of white noise over time – where the std. dev. of the values should increase proportional to sqrt(t)
This does indeed happen in the first case.
ReLu200-ensemble.pdf (1.2 MB)
ReLu200-ensemble-fit.pdf (18.5 KB)
When noise is injected to neurons, it does not though.
ReLu200-perneuron.pdf (1.3 MB)
ReLu200-perneuron-fit.pdf (18.9 KB)
Instead, the std. dev. jumps up immediately and stays there… almost as if the white noise is doing a “set” instead of an “inc” operation on the .neurons signal. This is a bit worrying because it implies the integrator can remember a value forever even with internal noise.
Any idea what I am doing wrong? Is this expected behavior that I am misinterpreting?
Code excerpt:
A = Ensemble(200)
tau = .1
Connection(A, A, transform=[[1]], synapse=tau)
if True:
encoded_noise = WhiteNoise(dist, default_size_out=1)
Connection(Node(encoded_noise), A, transform=[[tau]], synapse=tau)
else:
neuron_noise = WhiteNoise(dist, default_size_out=200)
Connection(Node(neuron_noise), A.neurons) # equivalent to setting "noise" kwarg of Ensemble