Really great article, but note especially the link to the original paper(which is only six pages long). I've been fascinated by the Loebner Competition forever and this is a pretty good deconstruction of it.

The article also links to an interesting bit on why IBM's Watson thought Toronto was a US city (and the footnote at the bottom is, hilariously, a form of Roko's basilisk).