Tuesday, September 08, 2009

Updates on Loebner Prize 2009 Results

Loebner Prize winner David Levy has no plans to reveal how his program Do-Much-More works, at least for the next year or two.

are some blogged comments about the InterSpeech conference, including some remarks by one of the judges, and the usual lament about "relative absence of progress in developing general purpose conversational agents".

Kris Schnee has published a version of the transcripts. Making readable transcripts is made complicated by the Loebner Prize protocol, which works like a keylogger, storing every individual character typed along with a time stamp. The judges and programs/confederates are allowed to type simultaneously, so these interleaved conversations have to be unraveled into transcript form.

Contest sponsor Hugh Loebner has increased the prize money for winning the Bronze medal to $5000 for 2010. The year 2010 marks the 20th annual Loebner contest. Loebner also announced that the contest will be held in the U.S. , and that "beloved Loebner Prize Protocol will be used for at least the next 5 years. This will remove the onus for contestants of having to change the I/O routines every year. I think that after 5 years we can re-examine the communications protocol in light possible advances in technology."

Update: Hugh Loebner has pubished a results page for the 2009 contest here.

Sunday, September 06, 2009

David Levy wins Loebner 2009

Congratulations to David Levy, winner of the Loebner Prize 2009 for "most human computer". The contest was held September 6, 2009 in Brighton, UK. This year the contest drew only three finalists. In addition to Mr. Levy's program, the competitors included previous Loebner winner Rollo Carpenter and newcomer Mohan Embar. (Update: David Levy's program is called Do-Much-More and it is not yet available online).

As has been the case in previous years, no program fooled the judges by passing as a human. Instead, a bronze medal and cash prize are awarded to the program that comes closest to chatting like a human. The programs were instead scored based on rank. The results were as follows:

First Place David Levy - Rank 4.5
Second Place Rollo Carpenter - Rank 5.0
Third Place Mohan Embar - Rank 5.5

As you can see, the scores were very close.

David Levy is an international chess master, whose program Converse previously won the Loebner prize in 1997. He becomes the third two-time Loebner prize winner. Joseph Weintraub's PC Therapist won the bronze medal four times, and A.L.I.C.E. won three times.

blogger templates | Make Money Online