The Now Economy

CommerceNet Internships -- Summer 2008

Fri, 21 Mar 2008 13:51:00 PDT

We are offering a couple internship positions at CommerceNet this summer.

CommerceNet is an entrepreneurial research institute, dedicated to fulfill the promise of the Internet. We are currently seeking Software Engineer interns to implement a data visualization Web application for public health information. Involves JavaScript and Python, both data access and graphics. CommerceNet may also accept proposals for internships to work on well-specified projects of the intern's own design.

What you'll do

Develop open source libraries or widgets for graphing and data visualization
Build public service, community oriented Web site
Be part of a small team or work nearly independently
Develop with minimal guidance, using rapid iteration and feedback loop and with leeway in choices of tools.
Borrow, create or collaborate on visual design and visual elements

Required Skills:

Web Applications development, including CSS and JavaScript
Python or demonstrated ability to pick up languages
MySQL or similar data management experience
Great ability to extrapolate from raw ideas to realistic implementations.
Demonstrated initiative pulling a project forward
Some experience using graphics libraries
Familiarity with Cleveland or Tufte principles would be a bonus

Email cn-hr@commerce.net with questions or cover letter and resume.

Commuknity

Tue, 02 Oct 2007 12:49:00 PDT

A goal of many new Web "2.0" ventures is to build a large or at least persistent community. Success is difficult to measure, but breaking into the top 100,000 sites by traffic measured by Alexa is one goal. It might be a good sign if the site designers send emails to a few people asking them to take a look at the site and after only two days over 3000 people sign up to beta test. You could do worse than to have a list of 60,000 people desperate to join your site before it leaves beta - so desperate that the site admins put a "waiting list checker" page up just so that an impatient person can see how many people are in line to get accounts before he or she does.

IETF highlights: HTTP Bis and BIFF BoF

Tue, 31 Jul 2007 15:52:00 PDT

The 69th IETF was last week in Chicago (windy and good pizza, who knew?). The two highlights for me were the HTTP Bis BoF and the BIFF BoF. A BoF is a "Birds of a Feather" meeting used to gauge interest and feasibility towards forming an IETF working group.

Attacks on Vidoop Authentication

Mon, 07 May 2007 11:05:00 PDT

A new authentication scheme was announced recently at the Web 2.0 Expo: Vidoop http://www.vidoop.com.

Vidoop describes itself as a web single sign-on solution that is resistant to "all prevalent forms of hacking". Specifically, they claim to resist "phishing, keystroke logging, brute force, and many man-in-the-middle attacks" and to resist automated attacks by "requiring human cognition" on the part of the attacker. This language is misleading. In reality, the scheme only resists simple phishing attacks - it does not prevent man-in-the-middle attacks, is vulnerable to brute force attacks, and it is resistant to keyboard loggers only when screen loggers are not present.

We were able to construct a man-in-the-middle (MITM) attack that allows us to capture users' credentials and to login to their accounts. We recorded a video that demonstrates a MITM attack in progress at myvidoop.com. Ian Fischer, a Harvard University student and research intern at CommerceNet, created the attack in a few hours, by modifying freely available proxy software on the Internet. We describe the attacks in more detail below.

How Vidoop works: Vidoop is essentially a combination of a graphical password scheme and client-side cookie. During setup, a user must choose their secret, which is a set of three "image categories" out of 25 categories (e.g., the user might choose cats, dogs, and birds).

To login, the user has to enter their username (or OpenID URI). The server presents a grid of 12 images from different image categories. Each picture has a random character superimposed on it, and three of the images are from the user's pre-selected categories. The user derives his one-time PIN by entering the three letters corresponding to his image categories.

Attacks: We recently conducted a study that analyzed attacks on Bank of America's SiteKey scheme [1]. Vidoop bears some similarities and shares many of the same vulnerabilities. In particular, Vidoop is vulnerable to a man-in-the-middle attack in which the attacker simulates the enrollment process. This is a well-know attack on SiteKey, which was first published in 2005 [2], and has been well analyzed by Jim Youll [3] and more recently demonstrated in this video by Indiana University researchers [4].

Like SiteKey, users must have a Flash cookie and/or HTTP cookie on their machine in order to log in (this cookie acts as a "second factor" that ties the machine to the user's account). If this cookie is erased, or if the user logs in from a new machine, the user needs to "enroll" the machine. The SiteKey enrollment process requires the user to answer a challenge question before receiving their cookie. This opens up a MITM attack, where the phisher lures the user to his website and presents the enrollment message "You are logging in from a computer that we don't recognize". The phisher proceeds to relay the challenge question from the bank to the user, and then relays the user's answer back to the bank. This allows the phisher to ultimately capture the user's SiteKey image and password. Because the user has probably seen the re-enrollment message several times in legitimate circumstances, he is likely to answer the challenge question and might not even know he was the victim of a phishing attack.

In Vidoop's enrollment process, the user has to request an activation code, instead of answering a challenge question (the activation code is delivered via email, a phone call or SMS text message). Once the user enters the activation code, the server will place a cookie on the machine, and allow the user to log in as usual. This opens up the same MITM described above - now, instead of relaying the challenge question to the bank, the phisher simply relays the activation code:

1. The phisher directs the user to phishingsite.com, which looks just like the Bank site, and the user enters his username.

2. The phisher relays the username to the real Bank and is presented with the message "We don't recognize your computer. Please select how you would like to receive your activation code". The phisher relays this message to the user.

3. The user selects the method of delivery, and the phisher relays this choice to the Bank. The user receives the activation code and enters it into the phishing website.

4. The phisher relays the activation code to the Bank, receives the cookie, and the user's authentication grid image.

5. The phisher displays the user's image grid to the user in order to obtain his PIN and secret "image categories". He relays the PIN back to the bank in order to log in.

Vidoop's requirement for out-of-band communication does not increase the cost of launching an automated MITM attack. In the SiteKey attack, the MITM phisher obtains the SiteKey image and password and a secure cookie, which allows him to log in indefinitely. In Vidoop, the MITM attacker obtains the user's PIN, which can only be used immediately to login to the account one time. He also receives the user's image categories and a cookie that allows him to log in in the future. To make use of the cookie, the attacker has to do a little more work.

Vidoop claims that subsequent logins require a human to determine the image categories and to look at the image grid to obtain the user's PIN. The necessity for a human in the loop increases the cost of an attack, and most phishers won't bother to go through the effort. They don't need to! The password space is so small that, once you have a cookie, a brute force attack is trivial. The myvidoop.com PIN is 3 characters chosen from the 26 characters of the alphabet, is order-independent, and is case insensitive, so the attacker only has to search 2,600 combinations (26 choose 3). With four login attempts available, the chances of success are 1 in 650. If the phisher uses automated character-recognition programs, he can reduce the number of combinations to 220 (12 choose 3), or a 1 in 55 chance of success with 4 login attempts. Note that brute force attacks are also easy to mount by anyone that shares the machine with the user.

Vidoop could increase the attacker workload by increasing the size of the PIN (the number of image categories), increasing the image grid, increasing character set (e.g., adding digits and symbols), requiring order dependence and non-repeatability, or by reducing the number of attempts that are allowed. To defeat character recognition, they could eventually employ captcha-type characters. All of these options will significantly reduce the usability of the system.

Vidoop does improve upon SiteKey in its resistance to keyboard logging attacks. If a keyboard logger obtains the PIN, it is only useful for one login and only within the timeout period. Vidoop is not resistant to malware that contains both keyboard loggers and screen loggers, which are becoming increasingly common [5].

Graphical passwords do have other weaknesses. For example, an attacker can predict the type of image categories that are chosen, even with very limited information about the target user [6, 7]. However, targeted attacks are expensive to mount - we've only focused on the attacks that are easy to automate here.

Privacy: There is a gaping privacy hole in their system. Vidoop makes it easy to search for registered usernames, and they openly publish these on their website. An attacker can enter usernames and request that activation codes be sent to them via text message, cell phone or email, depending on the user's preferences (this can be very costly and annoying for both Vidoop and its users). Initially, Vidoop had no time-out or restriction on the number of messages that could be sent by an unknown party. It appears that I can now only send 3 messages to any one person, after which time there is 9 minute timeout before requests can be sent again. By signing up for Vidoop, users essentially give anyone the right to send them Vidoop messages, without requesting their permission and without needing any contact information.

Usability: The cognitive overhead of selecting the Vidoop PIN is higher than recognizing the previously seen SiteKey image (the user must remember their semantic image categories, select images from the appropriate categories, find the associated characters and input them into a text box). However, Vidoop eliminates the need to recall a password, which is still a requirement with SiteKey. Vidoop eliminates the need to answer a challenge question during enrollment, but requires the user to check their email or phone and then input the activation code.

Summary: Before publishing our analysis, we communicated with Vidoop's CTO, Scott Blomquist. He acknowledged that he is aware of these weaknesses and that the scheme is vulnerable to man-in-the-middle attacks. In comparison to simple password authentication, Vidoop does raise the bar for phishers. However, we find their advertising, and in particular their claims that they resist man-in-the-middle attacks and "all prevalent forms of hacking", to be disingenuous.

[1] The Emperor's New Security Indicators, Stuart Schecter, Rachna Dhamija, Andy Ozment, Ian Fischer, to appear in the Proceedings IEEE Symposium on Security and Privacy, May 2007.

[2] The Battle Against Phishing: Dynamic Security Skins, Rachna Dhamija and J. D. Tygar, in Proceedings of the Symposium on Usable Privacy and Security (SOUPS), July 2005.

[3] Fraud Vulnerabilities in SiteKey Security at Bank of America, Jim Youll, July 2006.

[4] Deciet Augmented Man-in-the-middle Attack against Bank of America SiteKey Service, blog post and video, Christopher Soghoian, April 10, 2007.

[5] Anti-phishing Working Group, http://www.apwg.org/

[6] Deja Vu: A User Study. Using Images for Authentication, Rachna Dhamija and Adrian Perrig, in Proceedings of the 9th USENIX Security Symposium, August 2000.

[7] On User Choice in Graphical Password Schemes, Darren Davis, Fabian Monrose, and Michael K. Reiter, in Proceedings of the 13th USENIX Security Symposium, August 2004.

Needed: Web 2.0 hackers

Tue, 03 Apr 2007 16:17:00 PDT

A longer, though not necessarily more accurate job description, can be found here.

Open Source, Open Standards

Fri, 08 Dec 2006 15:19:00 PDT

I've been told in the past that Open Source and Open Standards are practically the same thing, they go so well together. While they're both good things, unfortunately, they're not quite as naturally reinforcing as you'd like. There's cost and style of participation, IPR concerns, and proliferation of standards ("The good thing about standards is, there's so many to choose from" - ref).

EMail Standards Waves

Thu, 30 Nov 2006 15:37:00 PDT

A month ago at the last IETF meeting, I talked to a bunch of email standards experts about the current wave of Internet email standards work. In these conversations I also built a mental picture of the previous waves.

Wave 1 was what really made email work over the Internet. The Simple Mail Transfer Protocol and the basic email message format were defined in 1982 with the major innovation of using domain names to find out where to deliver an email. This allowed email from one organization to reach an individual in another company. In 1989, this was somewhat updated with RFC1123, which made email addresses look the way they do today: mailbox@domain.example. Once mail got to the right server, POP (first described in RFC918 in 1984) allowed any mail client to look through the server's queue of new mail and decide what to do with each message. Although that first POP was described so early, many did not use it in those years and it didn't get much attention until POP3. Instead one would typically log into the SMTP server and look at its mailboxes there.

Wave 2 was POP3, IMAP and MIME, 1988 to 1994 or so. POP3 gained far more adoption than POP. IMAP defined a way to access the server-side repository for all one's mail: not just the queue of new messages, but a hierarchy of mailboxes (called "folders" in many clients) which can be used to store mail for access by several clients. MIME brought Media to electronic mail: the ability to include image file formats, to use HTML instead of text, to attach Word documents and executables, and other variations necessary to business and eventually much beloved by spammers. MIME also introduced the very first non-ASCII characters in the body of email. MIME turned out to be big for other purposes too, like the Web.

There's arguably a wave 2.5 or 3, adding security features from 1994 to 1999, including S/MIME, TLS support and authentication features for IMAP and SMTP. SASL was added to SMTP in 1999 although didn't get put into IMAP until 2003. This mini-wave didn't change peoples' lives much except for those whose companies rolled out complicated and hard-to-use S/MIME infrastructures, but the continued deployment of IMAP and MIME over this period did change the email habits of many.

Today's wave is starting to get complicated (oh, just starting? heh). It's adding internationalization capability, step by painful step (to various IMAP functions, to various mail headers like an email Subject line, and most painfully, to email addresses themselves). It's making IMAP and other mail infrastructure more usable by mobile clients (all the work of the Lemonade WG). It's addressing security and spam, among other things new ways to sign messages (DKIM). There is also some refactoring and architectural work going on which may be very interesting in the long run - for example, features to assign URLs and attach metadata to IMAP messages. This kind of work already allows increasing innovation in how email clients can deal with mail (particularly mail overload and spam).

The people I work with today include:

Dave Crocker, who edited RFC822 (mail message format) in Wave 1
Joyce Reynolds, author of the first experimental version of POP, RFC918 in Wave 1
Mark Crispin, author of the first version of IMAP, RFC1064 in Wave 2, and other revisions of IMAP
Nathaniel Borenstein and Ned Freed, who did the first three versions of MIME, starting with RFC1341 in 1992
Marshall Rose, who updated POP many times (POP3 in RFC1081, RFC1225, RFC1460, RFC1725 and RFC1939) in Wave 2
Randy Gellens and Chris Newman, who have contributed significant updates to POP and IMAP in Wave 2
Paul Hoffman, who defined SMTP over TLS in RFC2487 in 1999, and who ran the Internet Mail Consortium
John Klensin and Pete Resnick, who edited the modern versions of SMTP and the Internet Message format (RFC2821 and RFC2822 respectively).
The same and many more participating in today's wave, all of whom I greatly enjoy working with.

Of course, although talking to some of these guys helped me put together this picture of email standardization waves, any errors here are mine (and please let me know of errors so I can update this).

What compliance means to an engineer

Thu, 16 Nov 2006 19:27:00 PDT

I've been seeing the word "compliance" tossed around a lot for HTTP and other standards lately, with much ambiguity. Let's say you read RFC2068 and implemented a client very carefully. Does it make your client implementation "uncompliant" if a new standard updates or obsoletes RFC2068 and adds requirements, as RFC2616 did?

My answer is "that's not even a meaningful question". Compliance can be a very loose concept.

If your software claims compliance to HTTP, there can be a lot of variation in how that actually works because different versions of HTTP have significant differences.
If your software claims HTTP/1.1 compliance, we have a somewhat better idea what that means. A client advertising HTTP/1.1 support in its requests can be assumed to understand the "Cache-Control" response header from the server, because all the specs that ever defined HTTP/1.1 (RFC2068 and RFC2616) define that header. However, we can't tell if such a client supports the "s-maxage" directive on the Cache-Control header (the maximum age allowed for a shared cache entry) because that was only defined in RFC2616.
If your software claims RFC2068 compliance we don't know whether it understands "s-maxage", but we can assume that it supports "maxage".
If your software claims RFC2616 compliance we can assume that it understands "s-maxage" as well as "maxage". But support for RFC2616 isn't advertised over the wire to servers, so we can't tell the difference from clients that only implement RFC2068.

With this knowledge, you can ask if the new caching features in RFC2616 made existing clients uncompliant with RFC2068. Of course not. RFC2068 didn't change - there's a reason the IETF doesn't replace its standards in-place but defines new RFC numbers. Do the new caching features make the client uncompliant with RFC2616? Well it never claimed to be compliant with a spec that was probably published after the client was written. The important question to ask is whether a new feature or requirement is backwards-compatible (and if it's not, whether the feature is important enough to break backwards-compatibility). Let's consider The Cache-Control header a little further: a response with "Cache-Control: no-store" can be sent to any client that advertised HTTP/1.1 support, because that directive works the same way in both specs. If the response has "Cache-Control: s-maxage=1600", then we're not so sure if all HTTP/1.1 clients support it, but that might be OK - only shared caches can possibly do the wrong thing if they don't implement RFC2616 yet, and the server could limit the out-of-date cache entries of pre-2616 shared caches by having a backup plan, e.g. "Cache-Control: s-maxage=1600, maxage=36000".

This new feature was a reasonable choice in the standardization of RFC2616. If the writers of RFC2616 had been prevented from making any requirements that weren't already met in deployed clients, they would not have been able to add features like "maximum age limits for shared cache entries". The limitation would have unduly restricted their ability to improve RFC2616. Instead, the authors considered whether each feature was worth the new code and other design/test effort, and the backwards-compatibility considerations, and whether there were reasonable work-arounds or fall-backs.

It's a very engineering approach but that's what we do at the IETF. We don't do scientific theories of protocol compliance that must be true for all instances of protocol X. We do engineering.

A Skeptic's View of Identity 2.0

Tue, 17 Oct 2006 13:00:00 PDT

I signed up to do a talk called "Beyond Passwords" at ApacheCon US 2006, which took place in Austin last week. I had originally intended to talk rather blandly about current standards efforts. But in the end I took a much more contrarian approach and examined the promises of Identity 2.0, how policies and implementation progress are likely to affect the real benefits, and the risks or threats. It is a skeptical guide to a potential relying party - a Web service that is considering relying on some 3rd-party to authenticate and identify its users - on how to evaluate the benefits and the costs.

CalDAV to Proposed Standard

Mon, 09 Oct 2006 17:51:00 PDT

I am very pleased to announce that an effort I've spent nearly three years on is becoming an IETF Proposed Standard. CalDAV will have its own RFC number shortly, and the approval announcement was just last week.

[Lisa Dusseault] Introducing myself

Tue, 03 Oct 2006 13:01:00 PDT

As my first The Now Economy post, I thought I'd introduce myself and what I do.

I just joined CommerceNet as a Fellow a couple weeks ago. Just before that I was working at OSAF as a development manager and standards architect. I'd been doing that job for about two years, and simultaneously chairing the IMAPEXT and CALSIFY working groups at the IETF, when I was chosen by the IETF's Nominations Committee to serve as the Applications Area Director for a two year period. I'm interested in all the work going on in the Applications Area and I enjoy doing standards work which has so much leverage (even though it has a distant success horizon of deployed and useful implementations of new standards), so I was very happy to accept this position and enjoy doing it so far.

Gambling Bill doesn't cover Prediction Markets

Mon, 24 Jul 2006 11:43:00 PDT

The Internet Gambling Act (HR4411) that passed the House earlier this month is a curious compromise. There's been a lot of discussion elsewhere about the fact that this bill won't stop gambling on the Internet, and doesn't even seem intended to try; I'll leave that issue alone.� The only point I wanted to make is that the bill only covers sports betting (and pure lotteries).

The definitions say

(1)The term `bet or wager'--
(A) means the staking [...]  of value upon the outcome of a contest of others, a sporting
event, or a game subject to chance [...];
(B) includes the purchase of a chance [...] to win a lottery (which [...] is predominantly
subject to chance);
(C) includes any scheme described in [Unlawful sports gambling law];
(D) includes any [..] information pertaining to the [...] movement of funds [for] the
business of betting or wagering; and
(E) does not include [commodities, securities, derivatives, insurance, and fantasy sports]

Of course, the lack of coverage of prediction markets in this bill doesn't make them legal, it just leaves them out of the effects of this bill. Most of the effect of the bill is to make it harder to move funds into and out of gambling accounts, rather than to prohibit anything in particular.� It's not clear whether credit card handlers or banks would notice the distinction, but this may leave an opportunity for someone to run a market long enough to challenge the law. I think it's odd that they drew the line so narrowly. But there's time for the Senate to change that if they pass something, and if there are any differences between the House and Senate versions, anything can happen in reconciliation.

Increasing Liquidity in Multi-Outcome Claims

Wed, 19 Jul 2006 15:20:00 PDT

Previous articles have described a few simple formats of prediction market: simple double auctions, markets with open-ended prices, the symmetry of complementary purchases, and how to integrate an order book with an automated market maker. In this article, I describe the mechanics of multi-outcome markets, both as most markets currently implement them, and as I expect to implement them in the Zocalo Prediction Market. I've presented this idea before, so you can get another look at the idea by reviewing my slides. ( PowerPoint 3MB, PDF 2.6MB).

The basic idea is that instead of two exclusive outcomes, you want the market to give a prediction about an event that might turn out in one of three or more ways. Canonical examples include an election with multiple candidates running, or a tournament among some number of teams. The straightforward approach is to create a pair of assets for each candidate, representing respectively, that competitor's chances of winning and losing. This way you end up with N separate markets, and each one has a price for buying and selling the particular candidate that gives their chance of winning. The same general idea can be used to turn a continuous variable (how many widgets will we sell? What will the temperature in San Jose be on August 27th?) into a series of discrete choices. I'll talk about those kinds of markets later.

Collective Intelligence

Fri, 14 Jul 2006 17:37:00 PDT

MIT's Center for Coordination Science has recast itself as the Center for Collective Intelligence. Tom Malone (who spoke at the New York Prediction Market Summit), and Tomaso Poggio (who co-authored Securities Trading of Concepts) are two of the principals. The new center's framing question is How can people and computers be connected so that?collectively?they act more intelligently than any individuals, groups, or computers have ever done?. Prediction Markets are explicitly on the agenda. Their proposed (and ongoing) research includes:

How can large groups of people produce high quality written documents?
How can groups of people make accurate predictions of future events? For instance, in prediction markets, people buy and sell predictions about uncertain future events, and the prices that emerge in these markets are often better predictors than opinion polls or individual experts. When and how do these prediction markets work best? How can they be combined with simulations, neural nets, and other techniques? (emphasis added)
How can we harness the intelligence of thousands of people around the world to help solve the problems of global climate change?
How can we create an on-line, searchable library of books from many languages and historical eras?
How can we help create commercially sustainable products and services for low-income communities around the world?

This focus is strongly related to Marty Tenebaum's proposals in his AAAI invited talk and PARC Forum on collaboration between people and intelligent agents on the modern web. (In addition to being at the heart of what's most valuable about Web 2.0.)

Other distinguished faculty include Deborah Anacona, Rodney Brooks, and Alex Pentland. Marty's son Josh Tenenbaum is also on the faculty.

The center is new enough that their websites don't appear to be in their final locations. cci.mit.edu still has the old center's pages, while the new center is at learning.mit.edu. If the links move around, that's probably why.

Congratulations on the creation of the new center, and good luck!

�ADDENDUM: Apparently I jumped the gun with this annoucement.� The center is now up at cci.mit.edu, and learning.mit.edu has been discontinued.� I changed the link under the first mention of the center's new name.

Zocalo Release for Windows

Fri, 14 Jul 2006 17:24:00 PDT

In response to a couple of requests for installation help, I spent most of the week figuring out how to install under Windows, and how to generate appropriate zip files. I have uploaded a new release of the Zocalo Prediction Market Software to SourceForge. There are now 5 different files you can choose from in order to run Experiments or Prediction Markets, on Windows (zip) or unix-based platforms (tar.gz), or to get a copy of the entire source tree.

I'm embarassed to admit that I broke the Experiments in the last release, but they should be fixed.

On another note, while I was poking around SourceForge the other day, I noticed that they keep statistics on downloads and visits. The statistics on downloads didn't have any interesting consistency, and I think the "project web traffic" means my use to check-in changes and maintain the code, but this graph of traffic sure looks interesting. A steadily growing number of visits is good. Thanks for your interest. I plan to keep improving Zocalo; I hope to justify your continued interest.

New Zocalo Release: Accounts and Transaction History

Fri, 30 Jun 2006 12:28:00 PDT

I released a major new version of Zocalo on SourceForge yesterday. This release is the one I've been targeting as good enough for people to test out with interested groups. The user accounts now have password security, and only administrators can create new accounts and set up claims. (Later, I'll make it possible to configure Zocalo so anyone can create an account and the administrator will be able determine who can create claims. Public access to account creation is waiting until I hook up email for verification. At that point, it will also be possible to send trade results via email.) Claim owners now also have the ability to pay out claims, there's a new screen for reviewing transaction history, and users get feedback when a trade takes place.

If you have been waiting to use Zocalo to run prediction markets until a few more features are added, this is the first version (since the releases for economics experiments) that has enough functionality to be worth playing with and showing to your friends and colleagues. If you want to make suggestions about what features should be added first, I'd love to hear them. I know of many things that need to be added, but I only have my own guesses at this point about which missing features are show stoppers and what minor features would suffice for some users to deploy significant markets.

In the absence of feedback pointing to particular features that are useful sooner, I'm expecting my next step to be integrating email, so that account creation can be opened up, and the system can send transaction details to traders. I will also do some cleanup and simplification of the order entry form, since I've alrady had a request for that.

Individual Prediction And Prediction Markets

Fri, 23 Jun 2006 14:17:00 PDT

Eliezor Yudkowsky has written a good review on individual biases for a volume assessing the possibility of global catastrophic risks from AI, Nanotech, Biotech, etc. Yudkowsky's particular concern is AI, and the likelihood that an AI will take over the world soon after its ascendance if it's not carefully designed to care about humanity's wishes. In the process of arguing on this subject, he has spent quite a bit of time becoming an expert on various aspects of epistemology: Bayesian reasoning, cognitive biases, in general how to think, act, and argue rationally. His writings in this area are particularly clear and usually directed to an audience that might not have thought carefully about reasoning.

The Future of Zocalo

Wed, 31 May 2006 11:45:00 PDT

CommerceNet has been very generous in funding my work on Zocalo for more than a year. From the beginning, it was proposed as an appointment that would last for a year or two, as part of CommerceNet's program to bring a variety of people and projects into the labs, both to expose the particular projects to CommerceNet and its partners and to attract other entrepreneurs to visit and raise their new ideas and ventures to CommerceNet's view. While I've been here, I think both sides have benefited: I've been able to get more exposure for Zocalo than I would have otherwise, and in getting visibility for Prediction Markets, I've also raised CommerceNet's profile in some interesting arenas.

The next version of Zocalo that I release (sometime in June, I expect) will add the main features that were missing for usable long-term prediction markets. (Secure accounts, transaction history, and access controls for claim creation are checked in to sourceforge's subversion repository, paying off claims is coming soon.) I expect to run a couple of private trials (more beta sites would be welcome) and hope to install the software on a public website so people will be able to try it out. I'm also happy that there's another developer interested in adding some features who is talking issues over with me via email and on sourceforge.

I will continue to work at CommerceNet over the next few months, while I transition back to working on Zocalo on my own. (I was working on the code on my own nickel before CommerceNet offered to hire me and allow the code to remain open source.) I expect to work on Zocalo full-time for the rest of the year, and then spend up to half-time consulting to support continued development. The more consulting I can find that's related to Zocalo, the less time I'll have to spend on other work.

It's been great fun interacting with all the people and companies that have been through here. We had some great interns last summer, and are expecting more this summer. I can't talk about all the companies that have incubated here, but the folks that Renkoo and NewRoo were certainly cool to work with.

Zocalo's future? I'm going to continue betting on it for a while.

Chicago Prediction Market Summit

Fri, 05 May 2006 13:38:00 PDT

I'm joining the roster at the Chicago Prediction Market summit. I'll try to explain how Zocalo can improve the prospects for adoption of Prediction Markets in business by making the technology more accessible, and by doing a better job of publicizing results so more companies may be convinced that this is valuable technology. I'll try to argue that the academic results are in and pretty uniformly positive; the thing that is lacking that would enable or encourage more widespread adoption is evidence that these markets produce valuable input into organizations' deliberative processes. The existing companies selling PMs into business haven't been able to talk enough about how these markets have helped their customers.

I'm looking forward to hearing what Cass Sunstein has to say. His article in the Hahn & Tetlock book is quite interesting. He points out that groups that make decisions as a result of meetings and discussions make common mistakes in their reasoning, and that markets seem to have opposite tendencies. In the article, he seems to suggest that markets should be used instead, but the point about countervailing tendencies suggests that finding ways to use them together might be a better approach.

Arik Johnson's background is in competitive intelligence; he's going to talk about how markets can be used in exploring the competitive landscape. It will be good to see Robin Hanson, Justin Wolfers, and Michael Gorham again (all were at the DIMACS event in early 2005), as well as perenials John Delaney (from Tradesports), David Perry (Consensus Point), and Emile Servan-Schreiber (NewsFutures).

Some VC and Research Lab links

Fri, 05 May 2006 12:57:00 PDT

It's a Friday afternoon, and I'd like to clean up my desktop with a list o' links I've found interesting over the past few weeks:

The Personal Bee aggregator for VC & Startup news. Feels a bit like some of the concepts behind Newroo/Fox. Here are several stories I got to from there:

the MIT/Lemelson Prize for inventors goes to an LCD pioneer from Menlo Park.

Sequoia, via Kedrosky:
E&Y: Why is it crazy that LPs are willing to invest so much in venture capital?
Leone: The returns have been miserable. If you take away a couple of exits, such as Google and MySpace, there haven't been meaningful returns generated. There are [venture] firms that have never generated a positive return or have not even returned capital in 10 years that are raising money successfully. And that surprises the heck out of me. People talk about the top quartile - its not about the top quartile, it's barely about the top decile, or even a smaller subset than that.

Khosla Ventures actually does still invest in computer-related stuff, not just the cool new life- and green- sciences, from this BusinessWeek blogpost by Justin Hibbard:
one of his inaugural portfolio companies was SkyBlue Technologies, Inc. The Redwood City (Calif.) startup was founded a year ago by Stanford U. computer science professor Monica S. Lam and her fellow researchers, who are developing open-source virtualization software that lets systems administrators remotely manage PCs. Traditionally, companies have used programs like CA's Unicenter or HP OpenView for this task. Virtualization sacrifices some performance to keep the management program running independently from the PC operating system, which can become unstable. It's a clever use of an under-exploited technology that has had a recent resurgence on server computers and has produced at least one recent hit startup, VMWare. SkyBlue calls its class of software ready-to-run (R2R) and has launched a portal site, itCasting, to promote collaboration on R2R software. William J. Raduchel, CEO of Ruckus Network and former CTO at AOL Time Warner, is on SkyBlue's board. The company raised $1 million last August and $2.26 million from Khosla and others in March.

[In other recent manageability news, Intel announced vPro, a desktop featureset hopefully-analgous to Centrino, raising the possibility of yet-more feature wars such as XML processing smarts on the server side. ]

The New York Times recently had a piece on academics investigating the IBM-sponsored "services science" field

ComputerWorld's Gary Anthes, a dedicated reporter on the research-and-innovation beat, wrote a piece on the looming anniversaries of the oldest CS departments.:

John Canny, chairman of the electrical engineering and computer science department, University of California, Berkeley: "Computers aren't very valuable yet, because the applications they perform are still elementary and routine. It's actually remarkable how much we spend on IT, considering how little it does. The most widespread applications are still e-mail and Microsoft Office. That should tell us something.
What we really need to be thinking about is what people are doing with computers and how we could help them to do those things much better. Since most people are doing knowledge tasks, that means machines understanding their owners' work processes much more deeply, finding semantically appropriate resources with or without being asked, critiquing choices and suggesting better ones, and tracking synergies with other groups within a large organization. Computers will leverage the human resources in the company more at a knowledge level. They will directly tie what they do to the creative processes of employees. The economic impact of that would be much bigger than anything we have seen so far. "
Jaime Carbonell, director of the Language Technologies Institute, Carnegie Mellon University: "Artificial intelligence. Although those words may be somewhat out of fashion these days, much of the deep excitement and universally useful apps descend therefrom. For example: speech understanding and synthesis in handheld devices, in cars, in laptops; machine translation of text and spoken language; new search engines that find what you want, not just Web pages that contain query words; self-healing software, including adaptive networks that reconfigure for reliability; robotics for mine safety, planetary exploration; prosthetics for medical/nursing care and manufacturing; game theory for electronic commerce, auctions and their design to ensure fairness and market liquidity and maximize aggregate social wealth."
Bernard Chazelle, professor of computer science, Princeton University: I roll my eyes when I hear students say, "CS is boring, so I'll go into finance." Do they know how dull it is to spend all-nighters running the numbers for a merger-and-acquisition deal? No.
Canny: We're losing in quality - principally to bioengineering, which is now the best students' top choice - and diversity. It's a problem of social relevance. Minorities and women moved fastest into areas such as law and medicine that have obvious and compelling social impact. We've never cared much about social impact in CS.
Chazelle: Much of the curriculum is antiquated. Why are we still demanding fluency in assembly language today for our CS majors? Some curricula seem built almost entirely around the mastery of Java. This is criminal.
The curriculum is changing to fulfill the true promise of CS, which is to provide a conceptual framework for other fields. Students need to understand there's more, vastly more, to CS than writing the next version of Windows. For example, at Princeton, we have people who major in CS because they want to do life sciences or policy work related to security, or even high-tech music. In all three cases, we offer tracks that allow them to acquire the technical background to make them intellectually equipped to pursue these cross-disciplinary activities at the highest level.
Carbonell: CS needs a great communicator who lives the excitement, is deeply respected by his or her peers, and can reach out and communicate clearly with any educated person via his books. We have no such person in CS. Perhaps Raj Reddy [a Carnegie Mellon computer science professor] has the right kind of talents.

Finally, please don't miss Bill Burnham's excellent survey of opportunites to push 'persistent search' forward.

Market Design: Book and Market Maker

Fri, 28 Apr 2006 14:38:00 PDT

There are two basic ways of handling incoming orders: keeping standing orders in a book against which market orders can be entered, or having a market maker that accepts orders at its current price and adjusts the price after each order. It's also possible to combine the two, though that hasn't been done in very many of the existing markets. (Pennock mentions that the combination is used in finance, but doesn't give examples.)

Book orders match our mental model of the stock market. The basic idea is that traders are allowed to enter an order to buy or sell at any price. If they offer to buy at a price above (or sell below) the current clearing price (and there are enough shares available), then they make an immediate trade. If they offer to buy at a lower price (or sell higher) then the order is recorded and displayed to later traders. It's also possible that the market will partially satisfy the new order because the new order specifies more volume than the matching book orders. Different policies might apply in those cases: partially filling the order and placing the remainder as a book order; dropping the remainder; or producing an error message and requiring that the trader specify either that the whole order should be filled or that a price limit shouldn't be exceeded. On the stock markets, these are called markets orders and limit orders. A market order buys from existing book orders, accepting the market price, while a limit order specifies the highest price at which the trader is willing to buy (or the lowest sell price).

There is one difference in interpretation that matters more in prediction markets than stock markets. Most prediction markets interpret the limit price as the highest incremental price that should apply: no shares should be purchased beyond that price. But it's also possible to interpret the price as a limit on the aggregate price: if some shares are available below that price, then some could be purchased above the price as long as the average price for the order is below the limit. On a prediction market this might be against the trader's intent, since it could have the effect of pushing the current price significantly above the limit the trader specified.

There is some risk to entering a book order: market conditions might change, new news might become available, making the trader wish she hadn't left the standing order. Presumably for this reason, many traders only enter market orders. But in thin markets (most prediction markets are thin; even most stocks are thinly traded most of the time) there can be a gap between the best buy offer and the best sell offer. If there's always a 5% gap between the best offers on each side, then price takers (those who accept the price the market makes available) pay 5% more than price makers (those who enter book orders, thereby setting the market's prices). Market volatility determines whether that price difference is a bigger effect than the effect of the changing markets. This makes it valuable for market operators to encourage traders to place book orders. TradeSports charges fees only for price takers.

Markets orders are usually limited by the amount of (real or play) money a trader has in her account. Book orders can be limited in different ways. Most markets allow total orders in all markets up to the user's balance. The Foresight Exchange (FX) allows orders in each market up to the user's balance. When trading reduces the user's cash, FX reduces book orders in any market exceeding the new limit. TradeSports allows total orders up to the user's balance, and gives additional allowance for the expected value of current holdings. Robin Hanson's Combinatorial Information Market Design does an even better job of allowing the trader to invest in mutually exclusive outcomes because of the way it tracks assets.

These are all ways of further encouraging more book orders. As long as the market operator ensures that the user's balance isn't exceeded, they don't entail more risk. To the trader, they will sometimes cause orders to disappear just as you try to trade with them, but this isn't different from someone else accepting the offer before you, or the owner canceling the order while you submit yours. TradeSports' margin trading allows more trading, but requires that the market operator take on some risk.

The alternative to maintaining an order book is to have an automated market maker. Market makers in stock markets ensure the market is liquid by always having a published price at which they will buy or sell. They make money by maintaining a spread between the prices at which they buy and sell; the quoted price is the price at which you can buy from them. They set the price according to their beliefs about the current demand for the asset, and change the price as their opinion (or their exposure) changes.

Automated market makers set the price according to a rule that tells them how much to raise (lower) the price when a buy (sell) order is processed. A really simple rule would be to buy or sell a single coupon at the current price, and change the price by a constant amount. Run this way, the market maker's price would bounce around a lot as different traders traded with it, so it often takes a larger volume of trading to move the price as you approach the extremes.

If the market maker bases its prices on a rule that produces consistent prices whether buying or selling, it can sell an unlimited number of shares while limiting its losses. This only works if the market maker is integrated with the market; a user agent can arbitrage, but can't limit the amount of its losses if it doesn't have priority in getting its orders into the queue.

The market maker can also follow completely different rules. Dave Pennock's Dynamic Parimutuel Market (DPM) adjusts the price to ensure that traders on the winning side of a question will split the total value invested up to that point. The standard price rules ensure that the currently quoted market odds are also the incremental price at which you can buy or sell shares, which gives traders an incentive to trade whenever the price is different from their estimate of the odds. The DPM market doesn't have this incentive structure, as the price and the current payoff vary from one another.

A weakness of algorithmic market makers is that they don't adjust well as the volume of trading changes. The algorithm is parameterized by a constant that controls how quickly prices move in response to trading. When the market has many participants, you want the prices to move more slowly in response to trading than when there are only a few traders. If you choose the wrong constant, then it will be too hard to move the price in a think market (the price will remain near 50 even when sentiment wants to move it away), or in a thick market it might move back and forth too often. Order books work well in thick markets (there are plenty of offers for people to trade with), but less well with fewer traders. Combining the two should produce markets that work well in both cases. (Robin Hanson made a similar point about combining simple scoring rules with simple markets to produce his market scoring rules.)

In order to run a market with both a market maker and book orders, you have to ensure that the market maker's orders get priority and that the book orders are satisfied in the correct order as the market maker's price changes. The intuitive model is that you track queues of buy orders and sell orders, each sorted by the offered price. The market maker can freely trade and adjust its price as long as the price remains between the highest offer to buy and the lowest offer to sell. If the market receives a new offer that would move the price past either best offer, the market maker trades first, until it reaches the price of the best offer, then the book orders are used up. If the new offer hasn't reached its limit, the process iterates. (Robin Hanson and I discussed how to implement this for his Combinatorial Market in early 2003, resulting in his draft article.) This is all fairly straightforward if the market sells fractional shares (the stock exchanges work that way, why would it confuse traders?). I don't know of a way to use continuous prices if the market insists on trading only whole shares.

Using a market maker will increase the number of trades that are possible, especially when the markets are thin or many traders are reluctant to enter book orders. Adding book orders makes it easier for the market to adjust to increasing volume. If the initial constant used by the market maker is appropriate for a thin market, the book orders will play a more significant role as the market gets thicker, ensuring that prices don't gyrate too wildly.

Book Only

Maker only

Both

Zocalo currently integrates the two. When creating a market, book orders are always allowed, and the market creator can decide whether to add a market maker or not. While writing this article, it occurred to me that I should also allow markets with a market maker and no book orders.