I read an example of MNIST using NengoLoihi, but I don’t understand what is the rules of transform = conv in this example MNIST convolutional network — NengoLoihi 1.1.0.dev0 docs ? According to my knowledge, transform is used to form the connection weight matric in direct connection, however, I am not clear how to form weight from conv?
For context, the full code of that function is this:
def conv_layer(x, *args, activation=True, **kwargs):
# create a Conv2D transform with the given arguments
conv = nengo.Convolution(*args, channels_last=False, **kwargs)
if activation:
# add an ensemble to implement the activation function
layer = nengo.Ensemble(conv.output_shape.size, 1).neurons
else:
# no nonlinearity, so we just use a node
layer = nengo.Node(size_in=conv.output_shape.size)
# connect up the input object to the new layer
nengo.Connection(x, layer, transform=conv)
# print out the shape information for our new layer
print("LAYER")
print(conv.input_shape.shape, "->", conv.output_shape.shape)
return layer, conv
I’m not sure of your exact question…
If you are asking where the conv object comes from, it is created at the top of that function, and is a nengo.Convolution transform object.
If you are asking why a convolution transform object can be used as a weight matrix, it is because the convolution operation is a linear operation. In neural networks, a convolution layer applies a kernel to an input (see this wiki article). The kernel is typically a matrix, and the application process applies the kernel to almost every element in the input (this process is also known as convolution).
Because the convolution operation is a bunch of matrix multiplies, you can combine all of them into one giant matrix multiplication. You’ll need to do some math to figure out what the correct matrix to use in this giant matrix multiplication is, but it is possible. It’s kinda like how b = a \times 2 \times 2 \times 2 is equivalent to doing b = a \times 8. This is what Nengo is doing behind the scenes when you use the nengo.Convolution object. We made the nengo.Convolution object to make it easy for you to specify the parameters of the kernel (shape, rank, element values, etc.), and Nengo will handle the rest of it (i.e., converting the kernel into a connection weight matrix that does the convolution operation).