Viewing a single comment thread. View all comments

FunQuarter3511 OP t1_jcpstej wrote

Reply to comment by p0p4ks in Question on Attention by FunQuarter3511

Fully agree!

I think my issue was that because of the terms query, key, value, I was trying to relate them in a database or hashtable context. But in reality, those terms seem to be misnomers, and backprop will set the key/query pair to whatever is needed such that the dot product for important context will be large and be weighted appropriately.

I was over complicating it.

1