Nengo DL Converter multichannel input


Quick question, can the nengo dl converter deal with a multichannel input as typically the images are flattened. How do i deal with the situation when i have a multichannel image input.
My approach of just flattening does not appear to be working, using the same method as the Keras example.



Hi @paulkirkland, and welcome back to the Nengo forums. :smiley:

If by multi-channel input, you mean that the Keras layer accepts multiple inputs (e.g., the tf.keras.layers.Add()), then no, NengoDL doesn’t natively know how to deal with these types of Keras layers. As discussed in this forum thread, you’ll need to write your own nengo_dl.TensorNode, or create a custom class for the NengoDL converter (see this forum thread for an example, or refer to the NengoDL documentation).

If you provide some code, or a description of which Keras layers you are intending to convert, I can probably give you more directed suggestions to help you with your implementation.

Side note
I just realized you might also be talking about multi-channel as a description of inputs (e.g., RGB images, where R, G, and B form individual channels). In that case, flattening the images should work. If your inputs work within a Keras network, there should be very few reasons why it wouldn’t also work when converted to a Nengo network using the NengoDL converter.

Once again, some code that demonstrates the behaviour you are observing vs. what you are expecting would be helpful. :smiley:

i think i might have realised my mistake

I am dealing with a tf.keras.Input(shape=(27, 27, 8))

however when i flatten i need to arrange the shape to be 8,27,27 before i do this or the ordering of the stack, row, column gets confused