Looking at your notebook, I’d say yes. What you are observing is catastrophic forgetting. I think the issue stems from either the learning rate of the PES rule and / or the length of time you present the stimulus for. The higher the learning rate, or the longer you present the stimulus / output pair, the more it “overwrites” what is has learned before.
If you look at this forum thread I posted some code which uses the PES learning rule to learn a linear matrix transformation. In that code, I’m presenting the input / output pair for only 0.2s, whereas in your code, you are doing it for 1s.
Another thing you might want to experiment with is the number of neurons in the pre ensemble. I use a rule-of-thumb of at least 50 \times D (where D is the input dimension) neurons. In the code I used for the matrix transformation, I used 50 \times D^{1.5} (for better performance).