So, the Dawnguard beta keys have gone out. It's been officially stated that less than one half of a percent of applicants were accepted. The rumor going around (I lack any definitive source) is that only 40 beta keys were sent out.
What
a fucking joke.
This is not a beta, this is a PR stunt so they can say,
"Hey, look, we're trying to make sure the problems that plagued our
other DLC releases don't happen here". About all a 40 person beta can
do is make sure something like missing textures (Which happened with The Pitt
DLC for Fallout 3) doesn't happen. You aren't going to get any
substantive feedback on issues with just 40 people. As a developer on AutoIt which has public betas all the time I can stated pretty definitively that even with open betas simple and obvious things get missed and that's with software nowhere near the complexity of an Elder Scrolls game.
First let's talk logistics. Mose people do not provide feedback on betas (stated with experience as both a developer and a tester). If 40 random
people were selected then the odds of feedback from more than one or two of those people is incredibly slim. The other option is
that they specifically selected people from the forums that they are familiar with. If that is the case then this could have been some sort of community reward or a private beta or something that didn't involve signups.
Let's
talk coverage. Let's assume all 40 participants will give feedback.
You still have virtually no coverage. There are 3 ways to enter the
beta broadly speaking. You are either human, a vampire or a
lycanthrope. All of these need tested since this DLC centers around
vampires. Best case scenario you have 13 of each. We also know there are a couple factions, the vampire side and the humans fighting them. Will things get tested such as siding with the humans a a vampire? Siding with the vampires as a lycanthrope? Siding with the vampires as a human without becoming a vampire (if this is possible)? Can you cover all those things with 40 people?
If this DLC is
playable for any level then there needs to be coverage of various level
ranges to ensure it plays correctly. They also must want to have testers for various scenarios like the civil war, main quest completion and faction completion since they asked about these things during signup. Or perhaps those questions were only asked so they could narrow it down to who they think will be good testers.
Then there's getting it completely tested. With just 40 people how many of them will play it to completion?
Now
let's do some math. If less than one half of a percent of applications
signed up and we think 40 people were chosen then around 8,000 people
signed up. This is for a game that has sold 3.5 million units on the Xbox 360 in North America alone. I don't know what the statistics say about how many people buy DLC.
Let's pick a nice simple number, 20%. On 3.5 million (Remember, this
is just North America, the game has almost 6 million Xbox 360 sales
globally) that's 700,000 people who purchase the DLC. If they gave the
DLC to every single person who signed up that is only 1.14% of total
sales lost. That's assuming that everybody who signed up buys the DLC.
When
this was first announced Bethesda generated some good will with me. I
thought they would do a beta like other companies and take a few
thousand people so they could have a better chance of getting broad
coverage and getting feedback. In a game with so many variables like
Skyrim, you need that sort of coverage. Hearing just how few people
were selected burned all that good will up. I'll stop short of saying I
won't buy the DLC because of this because I will. I'm still excited
for it. I do, however, wonder how many sales they will lose over this.
This just screams PR stunt. Everybody with any sense knows that 40
people for a product that sold 3.5 million copies in one territory does
not constitute a beta.