Good ideas and conversation. No ads, no tracking. Login or Take a Tour!
This is really cool. PMI is a super useful measure, so it's nice that this model that works really well turns out to use it.And a paper last year found that this neural model, with its regularizer and everything, was actually implicitly performing matrix factorization on a matrix where cell (i,j) described how often word i and word j occur together. This was incredible, because NLP researchers had been using matrix factorization of this sort to find word embeddings for a long time, and this fancy new technique for learning embeddings suddenly had a clear intuition for why it worked!