Submitted by FunQuarter3511 t3_11ugj0f in deeplearning
FunQuarter3511 OP t1_jcpstej wrote
Reply to comment by p0p4ks in Question on Attention by FunQuarter3511
Fully agree!
I think my issue was that because of the terms query, key, value, I was trying to relate them in a database or hashtable context. But in reality, those terms seem to be misnomers, and backprop will set the key/query pair to whatever is needed such that the dot product for important context will be large and be weighted appropriately.
I was over complicating it.
Viewing a single comment thread. View all comments