Vector exponentials with unitary semantic pointers?


#1

We often do something like X*X*X*X (where X is a unitary vector) in order to get some sort of spatial scale in semantic pointers. But, that’s a discrete scale. Can we do something like X**3.5 and get a continuous scale? I’m not quite sure what that would mean mathematically…


#2

Yes, we can! To understand this, it is best to think about this in the Fourier space. When, we do X*Y (* being circular convolution), it is an elementwise product of the complex Fourier coefficients. The product of two complex numbers can be interpreted as a rotation and scaling of one of the numbers by the other number. Now, who says that we have to do the full rotation and scaling? Instead, we could do only “half”¹ of it and would get X*(Y**0.5). Of course this can be generalized to arbitrary exponents.

An interesting case is the vector $(0, 1, 0, 0, \dots)$ which will shift all elements by one position in circular convolution. With non-integer exponents, it allows us to shift vector elements by partial positions!

These things are also mentioned in this internal tech report (towards the end, starting with the section “Circular auto-convolutions”). I probably also have some more precise notes about the math involved lying around if you are interested.

¹: I’m not sure out of my head whether actually halfing the angle and scaling is correct. As we are actually taking the exponential of a complex number, the mathematics get a little bit weird and normal power laws like (a**b)**c = a**(b*c) do not apply. The required math can be found here. The most relevant formula is probably $c^{k^m} = e^{m\cdot \mathrm{Ln}\ c^k} = c^{m(k - d [(1/2) + (k/d)])}$ where $\mathrm{Ln}$ is the principal value $\mathrm{Ln}\ z = \mathrm{Ln}\ |z| + i \mathrm{Arg}\ z$ and $[\cdot]$ is the floor function.


#3

Sweet! So something like this should work:

def power(s, e):
    x = np.fft.ifft(np.fft.fft(s.v) ** e).real
    return spa.SemanticPointer(data=x)

That seems to behave as expected… power(s,2)==s*s and power(s,2.5).compare(s*s*s)==power(s, 2.5).compare(s*s)==0.64475


#4

Yep, seems like Python handles complex exponentiation. :slight_smile: The internal tech report also gives you an analytic formula for the similarity using the shift-by-one vector that you can use to verify your results.