When you add two vectors together, such as
DOG+CAT you often get a vector who’s magnitude exceeds 1. What neural options are there for normalization? I remember @xchoo doing a presentation on this at one point. I seem to remember the best bet was with the soft-normalisation performed by neural ensembles anyways.