a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment
user-inactivated  ·  3110 days ago  ·  link  ·    ·  parent  ·  post: "Why Should I Trust You?": Explaining the Predictions of Any Classifier

Recommenders are pretty simple, hence so many companies implementing them, and so not the type of algorithm that needs something like this. They're hard to implement, because you're juggling really big sparse matrices and you need to be a little clever to make them tractable, but algorithmically there's not much to them. Here's a bare-bones but representative recommendation algorithm:

1) Assign some index to each item in your inventory, and one to each of your customers

2) Form a matrix A such that a[i][j] is 1 if user i purchased item j

3) Perform singular value decomposition on A, and keep only the first some-arbitrary-number of singular values. This gives you a lower-dimensional space that approximates the very-high dimensional space. The intuition here is that because purchases of related things will be correlated, so by mapping into the lower-dimensional space produced by the SVD, which combines correlated axis, you're going from particular items to general interests.

4) Find k users nearest to a given user in the lower-dimensional space. Cosine similarity is the favorite way to define nearness. Rank items based on how many of those k purchased them, and recommend either the first some-arbitrary-constant or those with a rank greater than some-other-arbitrary-constant.

You can generate a "based on watching X, Y and Z" explanation from that by looking at the rankings of the items the user themselves purchased.

The models that are hard to explain aren't that simple.