I’m trying to make networks with dendrite-like multi-layer perceptron features. I call these MLPs “dendrites” because they are tree graphs like many biological dendrites, and each neuron gets non-permutable inputs from heterogeneous “synapses,” unlike with weighted addition followed by a filter (i.e. `nengo.Synapse`

).

It is perhaps related to BioSpaun from @jgosmann. The difference is that, being MLPs, they have nonlinearities but not conduction delays/states. It also might fall under a more traditional, weight-level neural modeling of the type described in @tbekolay’s post.

Sorry for the long and kind of vague post. I’m not sure if this is a Builder, Solver, or Nengo DL question. Any sort of partial advice would be helpful! I think it comes down to two parts.

### 1. Embedding stateless MLPs within a spiking network

Pseudocode:

```
def produce_one_neuron(*some_arguments):
with Network() as one_neuron:
dend_layer1 = Ensemble(4, 1, neuron_type=Sigmoid)
dend_layer2 = Ensemble(2, 1, neuron_type=Sigmoid)
soma = Ensemble(1, 1, neuron_type=LIF) # stateful neuron type
Connection(dend_layer1.neurons, dend_layer2.neurons, transform=L_4x2)
Connection(dend_layer2.neurons, soma.neurons, transform=L_2x1)
```

where `L_4x2`

means some 4x2 numpy array. See picture below.

The builder order of operations would look something like this pseudocode

```
one_neuron_steps = Simulator(produce_one_neuron()).step_order
one_neuron_steps == [TimeUpdate,
DotInc, SimProcess, # (the synapses)
DotSet, SimNeuron{Sigmoid}, # (dendrite layer 1)
DotSet, SimNeuron{Sigmoid}, # (dendrite layer 2)
DotInc, SimNeuron{SpikingSigmoid}] # (the soma/axon).
```

where `DotSet`

is an imagined `Y[...] = A.dot(X)`

. I have already made the synapse `Process`

- it is element-wise.

The point is that a spike arriving at a synapse at timestep `i`

should be able to have an effect on the soma at timestep `i`

.

It seems you could fold these “dendrites” into a fancy `nengo.Neuron`

with some `step_math`

composed of a bunch of Sigmoid functions; however, this would require a `nengo.Neuron`

with a vector of inputs, instead of just scalar `J`

. Is that possible?

Like the example above, another option is to make each layer of the MLP its own `Ensemble`

; however, that ends up stacking timestep delays and internal states associated with the “`DotInc`

” operators (hence why I made up the “`DotSet`

” operators). Is there a way around that?

### 2. Getting the optimizer to recognize these patterns

Each dendritic subnet as well as each “synapse” `Process`

are conceptually parallelizable – regardless of network topology – but the simulation optimizer does not recognize this if the neurons are constructed individually.

```
with Network() as multi_neuron:
for i in range(10):
produce_one_neuron()
```

The number of operations is theoretically constant in this case:

```
multi_neuron_steps = Simulator(multi_neuron).step_order
len(multi_neuron_steps) =approx= 10 * len(one_neuron_steps) # current behavior
len(multi_neuron_steps) == len(one_neuron_steps) # desired behavior
```

where basically all of the `Dot`

steps should now be sparse `BsrDot`

, just like what happens when you have a network of multiple `Ensembles`

. Instead of step growth (bad), I’m looking for memory chunk growth (good).

I have tested both ways of initializing this type of network: neuron-by-neuron vs. layer-by-layer, and it made a big differece in speed - ridiculously big if you are using a GPU.

This seems like it will need a customization of either an operator or the optimizer. Both of these are pretty confusing to me. Any ideas/pseudocode on how to approach this?