Hi,
I am trying to implement a network that uses the Heaviside step function as an activation function. So, I created a new neuron type using an example from here Adding new objects to Nengo — Nengo 3.2.0.dev0 docs. My implementation is following:
# Neuron types must subclass `nengo.neurons.NeuronType`
class SmoothStep(nengo.neurons.NeuronType):
# We don't need any additional parameters here;
# gain and bias are sufficient. But, if we wanted
# more parameters, we could accept them by creating
# an __init__ method.
def gain_bias(self, max_rates=0, intercepts=0):
"""Return gain and bias given maximum firing rate and x-intercept."""
return np.array([1.]), np.array([0.])
def step(self, dt, J, output):
output[...] = numpy.heaviside(J,0.)
But due to this function when performing nengo_dl.Simulator simulations, TensorFlow is throwing the following error.
UserWarning: <class 'nengo_utils.SmoothStep'> does not have a native TensorFlow implementation; falling back to Python implementation.
line 1812, in _create_c_op
c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 1580 and 2090 for '{{node TensorGraph/while/iteration_0/CopyBuilder/BroadcastTo}} = BroadcastTo[T=DT_FLOAT, Tidx=DT_INT32](TensorGraph/while/Identity_7, TensorGraph/while/iteration_0/CopyBuilder/BroadcastTo/shape)' with input shapes: [1580], [2] and with input tensors computed as partial shapes: input[1] = [32,2090].
It looks like having a corresponding TF implementation of this would help but I don’t know how to register it with the python version from my application code.
Also is there any other clean way to do this?