Deconvoluting Holzhauer

[mas3cf]

I did some of my own analysis of James's data available from J-Scorer at this link: https://j-scorer.com/shared/JamesOnTV

My method is super-secret and proprietary, but here's what I concluded. These are all best guesses based on statistical fit of available data. Get ready to feel small in 3, 2, 1....

1. The absolutely superhuman: He seems to know about 90% of J! material, and in an average game, he'd know 54 of the answers for an at-home Coryat of around 46000.

2. The extremely good: he probably won on the buzzer about 2/3 of the times he tried.

3. The mediocre: contrary to popular belief, he wasn't exceptional at staying clam. He buzzed in about 25% of the times he didn't have the right answer, which is fairly average for a J! contestant. It's just that it was so rare for him not to know, that his ratio of good to bad buzzes was very high.

4. His best topics (no surprise): sports and geography, where he seems to know about 95+% of J! material.

5. His worst topic (of the ones with OK sample size): animals, where it looks like he only knows about 50%. Still very good for a worst subject.

So there you have it... what it takes to be a legend.

[Robert K S]

He buzzed in about 25% of the times he didn't have the right answer

Can you tell us how this number was computed? How do we know how many times he didn't have the correct response and didn't buzz in? As opposed to had the correct response and didn't buzz in? How do you factor in when he was beaten on the buzz? Hopefully this aspect isn't too super-secret and proprietary to share.

[hbomb1947]

He buzzed in about 25% of the times he didn't have the right answer

Can you tell us how this number was computed? How do we know how many times he didn't have the correct response and didn't buzz in? As opposed to had the correct response and didn't buzz in? How do you factor in when he was beaten on the buzz? Hopefully this aspect isn't too super-secret and proprietary to share.

The number sounds made-up to me. It's impausible that James was "fairly average" at staying clam, when he averaged fewer than 1 incorrect response per episode over a very large sample size (not counting DD's, which of course he was forced to respond to). Ken Jennings averaged nearly 3.

[mas3cf]

He buzzed in about 25% of the times he didn't have the right answer

Can you tell us how this number was computed? How do we know how many times he didn't have the correct response and didn't buzz in? As opposed to had the correct response and didn't buzz in? How do you factor in when he was beaten on the buzz? Hopefully this aspect isn't too super-secret and proprietary to share.

That's actually the simple part. The easy way to verify is that in a number of different categories, his correct rate is in the 99% range (meaning he likely knows 95+% of the answers) but he answers about 65-70% of the clues. Assuming his buzz speed is the same across categories, it stands to reason he was beaten on the buzz about 1/3 of the time. When you do statistical modelling, it converges on that intuitive result.

The alternative extreme would be that he knows 2/3 of the material, wins on the buzzer every time, and stays clam 95% of the times he doesn't know. I find that highly improbable given his success rate on DDs and FJ.

When I use my formula and buzzer win rate and clam rate, then enter the back-calculated knowledge rates for the individual categories, it does a good job of predicting his distribution of responses by row as well as his rate of bad buzzes (although the latter is tricky because it's largely buried in statistical noise).

[mas3cf]

He buzzed in about 25% of the times he didn't have the right answer

Can you tell us how this number was computed? How do we know how many times he didn't have the correct response and didn't buzz in? As opposed to had the correct response and didn't buzz in? How do you factor in when he was beaten on the buzz? Hopefully this aspect isn't too super-secret and proprietary to share.

The number sounds made-up to me. It's impausible that James was "fairly average" at staying clam, when he averaged fewer than 1 incorrect response per episode over a very large sample size (not counting DD's, which of course he was forced to respond to). Ken Jennings averaged nearly 3.

Take what you will from it, sounds like you are taking nothing from it. When there are only 6 clues you don't know in an episode, and you pull the trigger 25% of those times, that's 1.5 attempts. If you win 2/3 of those, you'll get 1 wrong per episode. Numbers don't lie!

[jeff6286]

Yeah the 25% number seems high but I think it's perfectly plausible, albeit still largely based on guesswork. I agree that 90% is a good estimate for how many clues James knows in an average game. I'd put his range at something like 50-57, with an average around 54. So the number of clues he doesn't know is going to be around 3-10, so yeah he probably does buzz on 25% of those, meaning 1 or 2 in most games, of which he might win the buzzer race and neg or he might get beaten.

If Ken negged 3 times per game then maybe he didn't know as much as James, and was actually buzzing on 3 of 12 clues he didn't know (if he had a similar 25% average). This is largely speculative to be sure.

[mas3cf]

Yeah the 25% number seems high but I think it's perfectly plausible, albeit still largely based on guesswork. I agree that 90% is a good estimate for how many clues James knows in an average game. I'd put his range at something like 50-57, with an average around 54. So the number of clues he doesn't know is going to be around 3-10, so yeah he probably does buzz on 25% of those, meaning 1 or 2 in most games, of which he might win the buzzer race and neg or he might get beaten.

If Ken negged 3 times per game then maybe he didn't know as much as James, and was actually buzzing on 3 of 12 clues he didn't know (if he had a similar 25% average). This is largely speculative to be sure.

My sense is that Ken was stronger on the buzzer than James (who himself is no slouch) but James has the edge in knowledge base. If I could find detailed Ken stats, I could crunch the numbers for a best guess...

[gnash]

1. The absolutely superhuman: He seems to know about 90% of J! material, and in an average game, he'd know 54 of the answers for an at-home Coryat of around 46000.

That's not superhuman at all. I can't do that, but I can name at least 3 boardies for whom 54 correct would count as a below-average day.

[gnash]

Assuming his buzz speed is the same across categories

That's a strong and probably unrealistic assumption. In my experience, buzzer speed is almost certainly affected by confidence.