Hi @green_clover, I can add a few comments to help further answer your questions.
Regarding (1), if you want to build structured sentence representations, one thing you can try is defining strings that represent collections of role-filler pairs. For example,
OLD_MAN*NOUN + WALKS*VERB might correspond to the sentence “The old man walks.” One advantage of this approach is that you can call
vocab.parse() on such strings to return a semantic pointer object that is structured in accordance with the convolution and addition operators they contain. (i.e., the semantic pointer returned for the above example string would be built by convolving and adding other semantic pointer objects such those corresponding to
Regarding (2), when you use a
spa.Vocabulary to create semantic pointers (e.g., via
vocab.parse(word)), you are taking random vectors from the D-dimensional unit sphere (where D in your code is 256) that satisfy a minimum similarity constraint - all of the semantic pointers in the resulting vocabulary are approximately orthogonal. If you want to use pretrained vectors, you can do something like the following:
for word in words:
vocab is a
word is a string,
words is a list of strings (e.g., the list of nouns you have defined), and
word2vec is a dictionary that maps from strings to numpy arrays. What this will do is initialize the semantic pointers in your vocabulary using pre-existing collection of word vectors. You can use a tool like GenSim to load Google’s word2vec demo file and create the needed dictionary. One thing to check for is whether the words you want to model are all present in the preexisting collection of word2vec examples. Another thing to consider is that word2vec embeddings do not necessarily exhibit the properties needed to support effective binding via convolution.
Regarding (3), I think the challenge here is to find some way of automatically converting real-world sentences into strings that describe collections of role filler pairs that you can call
vocab.parse on. One way you might try to do this is by using an NLP library to get part-of-speech (POS) tags for each word in a sentence, and then automatically generating a new string of the form
Word_1 * POS_1 + Word_2 * POS_2... which you can then use in your SPA model. A more complicated version of this approach might involve using parse tree information in some way, since POS tags do not really capture anything about the sentence’s structure.
Anyway, this is all very much an open area of research, so please let us know if you have any follow up questions!