EEDAR's study suggested not releasing demos resulted in better sales and marketers should start designing Achievements. MTV Multiplayer's astute readers brought up some smart criticisms of the EEDAR's results. Said Multiplayer reader JFK:

"Gears of War, COD4, Rock Band and Halo had no demos, but they didn’t need them due to their marketing and general market anticipation. The very AAA games were more likely to not need demos to get sales."

Gregory Short and Geoffery Zatkin, the heads of EEDAR, spoke with Multiplayer yesterday, and don't necessarily dispute JFK's claims.

They claim their job is pointing out industry trends, the data isn't a guarantee. "Causation is a hard thing. There are so many variables in the industry. What works now may not work later on causing something. What we want to do is point out really strong correlations," said Zatkin.

"If you have a crappy game and you release a crappy demo, it's going to hurt you," said Short. "And if you have a game with a $100 million marketing budget, you probably don't need to release a demo. What our evidence showed us was, if you only have an average game and an average marketing budget, putting out a demo that's bad is going to hurt you even more than [if it] were a triple A title [where] marketing might make up for it."

Fair enough, but the study didn't really tell us much about EEDAR's methodology, how EEDAR interpreted the available data and, perhaps most importantly, just who are Gregory Short and Geoffery Zatkin.

Are they nefarious marketers looking for new and unique ways to manipulate the everyday gamer into buying things they don't actually want? Far from it, actually.

Zatkin worked on the original "EverQuest."

Both Short and Zatkin come from Sony Online Entertainment, but from opposite ends, Zatkin from development and Short on the business side. The two formed EEDAR after seeing most decisions in the games business are made based on experience, not empirical data. They wanted to change that.

"Every game I've ever worked on, you have $10, 20 30 million dollar products. You have a lot of really cautious people because that's a lot of friggin' money," said Zatkin. "As a designer, every project I've worked on, we've never had the kind of information we've really wanted."

Zatkin wanted hard data on what gamers expect from their interactive experiences. Each genre comes with a certain set of expectations by the gamer. Zatkin believes these expectations account for roughly 70% of every game's design, while the other 30% is strictly creative. The problem: no one can agree on what that 70% should be, resulting in a lot of wasted time iterating on bad game design.

Maybe there's a specific expectations for the mini-map, a preference for a particular speed of camera movement, or a favored death mechanic in an MMO. There isn't necessarily "one" winning model, but EEDAR attempts to determine the two or three best options available to developers. "What we want to help people do is narrow down what that is so they can spend more time on the creative part of the game," said Zatkin.

"Every game I've ever worked on, you have $10, 20 30 million dollar products...that's a lot of friggin' money"

Under the current model, Short argues, companies take a gamble on every game that's released. "We [developers] get $20 million dollars and we debate over what we think is fun and what we think is gonna be cool and then we throw it to the wolves [the consumer] and hope we get some money back," he said. "That's pretty much the way the game industry works right now. It's a little scary."

I suggested that nailing design to a few proven formulas could erode innovation, but Short don't look at it that way. "Iterative work is inefficiently spent. [We help companies] make informed decisions earlier. Then everyone's not only going to have a better quality of life in the work place, but also make better games."

Zatkin and Short were both developers and saw how marketing interferes with the creative process. "People have been interfering with games to improve their marketability for years," said Short. "But they've been doing it on experience, not on empirical facts. It's far more dangerous to be interfering in ignorance than from a position of information."

Readers, what do you make of EEDAR and their mission? Will the itemization and classification of successful game elements result in better games?