You know, there are two kinds of things statistics can be used for in science, to determine correlation between events and to determine causality. What we're doing here is determining correlation. We know that Legacy is involved in the majority of wins, but we don't know if Legacy causes or is even a major contributing factor to the wins. To determine that, we'd need to run experiments. We'd need to run a number of games using the same heroes/villains/environments and then run the same games with one hero swapped out and observe the results. Where are all the science students out there yearning to set up hundreds of controlled experiments so that we can actually say to ourselves, "this isn't just related to that somehow, this CAUSED that"?
Seems to me that you would have an additional problem: the randomization of the cards. You'd either need to run each game several times before you ever switch out or set up a villain who will take a long time (thereby increasing the chance that everyone would run through at least half of their deck).
I think what we’re dealing g with here (and I could be wrong I stuck at math) is the law of large nimbers. that even with all the random factors involved with playing a game of sotm, eventually its all going to average out and we’ll get some sort of information that is palpable to understand. that is with enough games, yes it will become evident that having legacy in your party is more likely to win you the game in general.
Is having Legacy in your game more likely to win you the game, or do people who have played the game longer and are therefore more experienced, more likely to choose Legacy for their games, winning because of their experience, not because of Legacy? Correlative data like we have can't answer that question, but a designed experiment could.
It's the same with the Advanced mode wins -- a surprisingly high percentage of those games are wins, but that's most likely because only people who are very experienced even play Advanced mode. If you designed an experiment that took a randomly selected group of players (so, a mix of experience levels) and had half play Normal mode and half play Advanced, I think the data would show a drastically reduced win percentage for Advanced mode. But without the designed experiment to remove the factor of experience from the equation, you can never be sure.
Of course, if I cared that much, I'd design and run the experiments myself... ;)
I still think it comes down to the Law of Large Numbers Spiff - even with what your saying, i understand, but given a signifigantly large enough of data points, it will all average out in the end. This is a very important law of statistics, from what (little) i understand. Over enough data things like player experience, heroes paired with, and other factors wont matter because there is enough data that the 'random ness' finally results in an average.
of course were talking 1000's of datapoints per entry and were barely in the 100-200 range for most of them, far less than that for many, so its a long point before where get there, and while were working, your points are still extremely valid in dealing with these numbers
Spiff is talking about systematic errors. Statistical errors are saved by large numbers, but in a academic study you always need to consider systematic errors. Now with large numbers you can at least estimate our bias.
For example, one would think that experience would improve play skill. If all the players inputing data were more expierenced than their win percentage should be higher than the win percentage of new players.
You could attempt to measure this bias using the data in a variety of ways. One way is to look at how an average inputees win% increases on a month by month basis. If we spot a trend then we could modify the data by the inputees time inputing data.
In general there could be thousands of varieties of bias. Most will be really small, does win % drop on Friday do to drinking while playing for example?
There is one bias we will never be able to test: do people who input data play differently then people who don't.
To test this we would need the field studies spiff suggests. Personally I think the effect of this is small. I play with my 7 year old, we often lose because of poor play. I included data from games with a lot of newbies, and games were I play solo. I think the variety of entries will balance most of the bias, but to be sure we would have to check.
Perhaps people doing demos could input their results? Perhaps with some sort of marker for that in the data?
Personally I think overall systematic errors will be small, but I could be wrong.
As an aside, it would be interesting to see how solo games compare with group games. How much more do you win when you're the only one you have to argue strategy with?
I added last night a # o players optional field
That's going to be exciting to start tracking, I wonder how many of my games make up the solo player market? Guess we'll be finding out soon enough!
I was excited to see the # of players option as well. I think all but two of my games have been solo and I was wondering if that would make a difference
You added a new thing and I got confused :(
4/3/2013 16:18:25
4/3/2013 19:19:48
The second entry is correct, I accidentally submited it as 2 players, not 2 difficulty, please delete the first entry
I feel bad about not entering all of the games I play at cons, but there is JUST NO WAY. It is SO MANY GAMES all back to back to back to back.
Well, just keep a pad of paper with you and write it down as soon as it happens! Yeah, you're right. That's possible, BUT at a con I am often busily moving into spiel about the company and products and thinking about who is going to sit down next and getting the cards and tokens organized and ready for the next group.
So, a proposal -- if you are going to be at a con and are not already busy doing other things, feel free to volunteer to be stats secretary -- going around amongst the tables and recording the results. This would be at more than just the booth, mind you, as there can easily be many, many games going on in free play areas (or in lobbies of hotels, etc.)
Another thought is that the >G staff could set up some sort of a kiosk in the booth where people could record their games, but a note of caution on this -- there usually is neither power nor wifi available at the booth, so they are relying on re-charging devices and (perhaps costly) cellular data.
How useful are stats from people playing at cons, though? Those games will be from people who barely know the game or the rules and who may not have even decided if they like the game or not. I don't think we're losing much by not including those games.
Fixed
And Re: Cons - Yeah. I dont expect its going to be that easy. Im already working Origins, and id much more rather be teaching than secretariing 
ill write down a few for the sake of doing it but I doubt ill get all of them
How about offering Sentinels Points for people to come to the booth and register their game by filling up a hardcopy of the form. After the con, some one can do the data entry. This avoids the computer/wifi problem. And encourages players to contribute to the stats.
Sentinels points – would certainly be a motivator, but I wonder if it's unnecessary motivation – getting to fill in the stats could be its own reward in a way, it's fun to fill in the form.
Data entry – uggh
As for con games not being typical, that may be much less so than it used to be, as a LOT of the games played at the con are by experienced players (who have drawn in a newbie or two) who are looking to get more Sentinels Points. There are EIGHT new promo cards, so if you want them all, you need 800 Sentinels Points. Now, you might have gotten a few hundred from buying the core and/or expansions, but even getting all of those only gives you 500 Sentinels Points.
Get the intern to do them. Unity comes to mind. 
Given that its a fan based project, not official, i doubt we'll get any official support on the line, like SP or a form. Its a fun side project, but thats all it is - fan based, for fun.
And i -hate- data entry. hatehatehatehatehate. it takes me a few days to even get around to putting in the 2-3 games we play every tuesday, cause i just hate doing more than one at a time :p
pfft, data entry is easy brain-off stuff for me. give me the lists and i'll input them. might not all get done right away but it'll get done in a semi-timely manner.
Interestingly, I just looked up the Google Game games again and it appears that Argent Adept currently has a 100% win rate on there. Of course I don't think all games have been counted, plus putting them under Google Games is a fairly new thing and everything else prior to that was put under whom ever did the statistics. It is also 13 games aswell, so I think that's fairly good. Team Leader Tachyon is shortly behind with 11 wins and being at 100% as well. Redeemer Fanatic (7), Scholar (9), and Wraith (8) all also have 100% win rate as well.
One of the most shocking things about it is that Legacy has a 58% win rate, only 7 out of 12 games has he won. Kind of interesting since the only other hero under 70% win rate is Absolute Zero at 66%. Both Abz and Abz: EW are both tied at 6 victories in 9 games.