Hi, sorry for posting many theoretical questions, but I really hope to understand the theory behind Nengo better.
I think I got myself confused about how transformation (Principle 2 of NEF) works for non-linear functions. I was following the derivation presented in Nengo summer school lecture 3: NEF Principle II with Nengo - YouTube (approximately 43min in the video) which learns \textbf{y}=\textbf{x}. The derivation for the weight w_{ij} between 2 neuron ensembles a_i representing \textbf{x} and b_j representing \textbf{y} is:
\begin{equation} \begin{split} b_j &= G_j(\alpha_j\textbf{e}_j\hat{\textbf{x}} + J_j^{bias})\\ &= G_j(\alpha_j\textbf{e}_j\sum_i{a_i d_i} + J_j^{bias})\\ & = G_j(\sum_i{w_{ij}a_i} + J_j^{bias}) \end{split} \end{equation}
This gives us w_{ij} = \sum_i{\alpha_j\textbf{e}_jd_i}.
This is fine so far, but once I tried to reason about \textbf{y} = \textbf{x}^2, I find that I am stuck at b_j = G_j(\alpha_j\textbf{e}_j(\sum_i{a_i d_i})^2 + J_j^{bias}), where I can no longer represent the activations of the second ensemble as a function of linear transformations of the activities and decoders of the first ensemble.
So I guess my question is how are the weights solved for nonlinear transformations such as \textbf{y} = \textbf{x}^2?