Thunderdome match - Eugene Goostman vs Alan Turing

Jedi Mind Trick
Jedi Mind Trick's picture

 

We all know (or soon will, if you keep reading) that the Turing Test is the Holy Grail of artificial intelligence trophies because it tests how effectively a computer/machine can convince humans that it is also human. In the case of Eugene Goostman, the reported result was that over 30% of the humans evaluating the program bought into the idea that the program was, in fact, also human.  This edges Turing’s guidance by the thinnest of margins..and only when you look to take his statements on the matter…and not the formal and historical guidance of the test…as the criteria for pass.

In 1950 Turing famously predicted that within the coming 50 years, computers would be so adept at imitating human thought that "an average interrogator will not have more than a 70% chance of making the right identification after five minutes of questioning." Eugene Goostman, as noted above, seems to have exceeded the threshold…but, as you’d expect…there wouldn’t really be a story here if that was all there was to it.

The sad fact of the matter is there is a bit of gimmickry on the part of Goostman’s programmers and the manner in which they chose to present their creation. Instead of tackling the challenge in a straight forward manner…with real, trained Turing judges, they gamed the test a bit by proclaiming that Goostman was a teenaged boy from Odessa, Ukraine…and asked people selected from tech and news media to participate in a 'test'...this move really gave them allowances, both for goofy responses resulting from misunderstanding of language and for lack of real 'interrogation' discipline on the part of the 'judges'. 

Here is the interface to the program (with all the news, access may be sporadic): http://www.princetonai.com/bot/

While its more interesting than neat that the Goostman crew pulled this off and it TECHNICALLY beats the Turing standard, it doesn't demonstrate machine learning and is by no means an AI. It also doesn’t set off the off the alarm that the Singularity is any nearer or that somewhere, in a basement, a forgotten Apple IIe is toiling away in obsurity on what will become SkyNet.

In many ways, this ‘Goostman’ event is the equivalent of a Spiritualist séance where a medium proves contact with the dead by asking for knocks and then knee kicks the underside of the table. It might freak some people right the hell out…but when you think about it…its utter crap and doesn’t mean that you’re talking to the deceased. (It gets even goofier when you realize that the crew that built Goostman are programmers from a company called Wholesale Change...and they've been at it since 2001...and the whole purpose is to create a convincing chat bot that can satisfactorily replace human chat agents. Think about this a second...they pass the test of being a human by mimicking an adolescent with a language barrier...as a means of proving they have a human like equivalent of a chat agent?!?! LOL. If it wasn't true, it would be funny.)

And, assuredly, somewhere…wherever the dead go…Turing is the one kicking himself for putting the onus on a machine to rise to the occasion of his test instead of predicting that common human behavior would one day become so absurd that a machine could imitate it without true (or synthetic) understanding.  

Tags: