# Winning Jeopardy

Jeopardy is a popular TV show in the US where participants answer questions to win money. It’s been running for a few decades, and is a major force in popular culture.

Let’s say we want to compete on Jeopardy, and we’re looking for any edge we can get to win. In this project, we’ll work with a dataset of Jeopardy questions to figure out some patterns in the questions that could help us win.

**Project Goal**
In this project, we’ll work with a dataset of Jeopardy questions to figure out some patterns in the questions that could help us win.

This project was motivated as part of the guided project series on dataquest. The analysis is an extended version of the guided project.

## Jeopardy Questions

Let’s explore the jeopardy questions. We will see that the dataset contains the following columns:

`Show Number`

– the Jeopardy episode number of the show this question was in.`Air Date`

– the date the episode aired.`Round`

– the round of Jeopardy that the question was asked in. Jeopardy has several rounds as each episode progresses.`Category`

– the category of the question.`Value`

– the number of dollars answering the question correctly is worth.`Question`

– the text of the question.`Answer`

– the text of the answer.

Let’s start our exploration with first of all loading the complete Jeopardy Question csv file from this reddit post. The csv file was downloaded and added to the folder location.

```
import pandas as pd
from string import punctuation
import re
import matplotlib.pyplot as plt
from scipy.stats import chisquare
# read in the dataset
jeopardy = pd.read_csv('jeopardy.csv')
jeopardy.columns
```

```
Index(['Show Number', ' Air Date', ' Round', ' Category', ' Value',
' Question', ' Answer'],
dtype='object')
```

We can see that the names of each column contain an additional space. Let’s remove those and keep the name only.

```
# clean column names from additional spaces
jeopardy.columns = jeopardy.columns.str.replace(' ', '')
jeopardy.columns
```

```
Index(['ShowNumber', 'AirDate', 'Round', 'Category', 'Value', 'Question',
'Answer'],
dtype='object')
```

```
jeopardy.shape
```

```
(216930, 7)
```

```
jeopardy.head()
```

ShowNumber | AirDate | Round | Category | Value | Question | Answer | |
---|---|---|---|---|---|---|---|

0 | 4680 | 2004-12-31 | Jeopardy! | HISTORY | $200 | For the last 8 years of his life, Galileo was ... | Copernicus |

1 | 4680 | 2004-12-31 | Jeopardy! | ESPN's TOP 10 ALL-TIME ATHLETES | $200 | No. 2: 1912 Olympian; football star at Carlisl... | Jim Thorpe |

2 | 4680 | 2004-12-31 | Jeopardy! | EVERYBODY TALKS ABOUT IT... | $200 | The city of Yuma in this state has a record av... | Arizona |

3 | 4680 | 2004-12-31 | Jeopardy! | THE COMPANY LINE | $200 | In 1963, live on "The Art Linkletter Show", th... | McDonald's |

4 | 4680 | 2004-12-31 | Jeopardy! | EPITAPHS & TRIBUTES | $200 | Signer of the Dec. of Indep., framer of the Co... | John Adams |

Our dataset contains 216930 questions.

## Normalizing Text

Before we start with our analysis, we need to normalize the text, that is ensure lowercase words and remove punctuation. There are two ways we can go about this. Let’s see which one is more efficient.

```
# function to normalize the text using python string library
def normalize_text(s):
s = str(s).lower()
s = s.translate(str.maketrans('', '', punctuation))
return s
# function to normalize the text using regex
def normalize_text_reg(s):
s = str(s).lower()
s = re.sub("[^A-Za-z0-9\s]", "", s)
return s
s = 'Hello.?! World.*'
%timeit normalize_text(s)
%timeit normalize_text_reg(s)
```

```
3.26 µs ± 38.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
3.26 µs ± 194 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

We can observe that using the function with regular expression seems to be much faster. We will use it to treat and normalize our `Question`

and `Answer`

columns.

```
# normalize our columns
jeopardy['clean_question'] = jeopardy['Question'].apply(normalize_text_reg)
jeopardy['clean_answer'] = jeopardy['Answer'].apply(normalize_text_reg)
```

```
jeopardy.head()
```

ShowNumber | AirDate | Round | Category | Value | Question | Answer | clean_question | clean_answer | |
---|---|---|---|---|---|---|---|---|---|

0 | 4680 | 2004-12-31 | Jeopardy! | HISTORY | $200 | For the last 8 years of his life, Galileo was ... | Copernicus | for the last 8 years of his life galileo was u... | copernicus |

1 | 4680 | 2004-12-31 | Jeopardy! | ESPN's TOP 10 ALL-TIME ATHLETES | $200 | No. 2: 1912 Olympian; football star at Carlisl... | Jim Thorpe | no 2 1912 olympian football star at carlisle i... | jim thorpe |

2 | 4680 | 2004-12-31 | Jeopardy! | EVERYBODY TALKS ABOUT IT... | $200 | The city of Yuma in this state has a record av... | Arizona | the city of yuma in this state has a record av... | arizona |

3 | 4680 | 2004-12-31 | Jeopardy! | THE COMPANY LINE | $200 | In 1963, live on "The Art Linkletter Show", th... | McDonald's | in 1963 live on the art linkletter show this c... | mcdonalds |

4 | 4680 | 2004-12-31 | Jeopardy! | EPITAPHS & TRIBUTES | $200 | Signer of the Dec. of Indep., framer of the Co... | John Adams | signer of the dec of indep framer of the const... | john adams |

## Normalizing Columns

Now that we have normalized the string columns, let’s also normalize `Value`

and `Air Date`

columns. We will remove the $ sign from the `Value`

column and convert the column to numeric; we will also convert `Air Date`

to datetime.

```
# function to normalize value column
def normalize_value(s):
s = re.sub("[^A-Za-z0-9\s]", "", s)
try:
s = int(s)
except:
s = 0
return s
# clean value column
jeopardy['clean_value'] = jeopardy['Value'].apply(normalize_value)
jeopardy['AirDate'] = pd.to_datetime(jeopardy['AirDate'])
```

## Answers in Questions

In order to figure out whether to study past questions, study general knowledge, or not study it all, it would be helpful to figure out two things:

- How often the answer is deducible from the question.
- How often new questions are repeats of older questions.

We can answer the first question by seeing how many times words in the answer also occur in the question. We can answer the 2nd question by seeing how often meaningful words occur. Meaningful in this context means words that are not stopwords or words that make sense. Let’s tackle the first question first.

```
# function to count words from answer in the question
def count_terms(row):
# split the column values by space
split_answer = row['clean_answer'].split()
split_question = row['clean_question'].split()
# initialize count of words
match_count = 0
# remove 'the' from answers
if 'the' in split_answer:
split_answer.remove('the')
# check if answer is 0
if len(split_answer) == 0:
return 0
# count ocurrences of answer values in question
for i in split_answer:
if i in split_question:
match_count += 1
# return the calculated count
return match_count / len(split_answer)
# count the words
jeopardy['answer_in_question'] = jeopardy.apply(count_terms, axis=1)
# calculate the mean
jeopardy['answer_in_question'].mean()
```

```
0.05792070323661065
```

```
jeopardy.head()
```

ShowNumber | AirDate | Round | Category | Value | Question | Answer | clean_question | clean_answer | clean_value | answer_in_question | |
---|---|---|---|---|---|---|---|---|---|---|---|

0 | 4680 | 2004-12-31 | Jeopardy! | HISTORY | $200 | For the last 8 years of his life, Galileo was ... | Copernicus | for the last 8 years of his life galileo was u... | copernicus | 200 | 0.0 |

1 | 4680 | 2004-12-31 | Jeopardy! | ESPN's TOP 10 ALL-TIME ATHLETES | $200 | No. 2: 1912 Olympian; football star at Carlisl... | Jim Thorpe | no 2 1912 olympian football star at carlisle i... | jim thorpe | 200 | 0.0 |

2 | 4680 | 2004-12-31 | Jeopardy! | EVERYBODY TALKS ABOUT IT... | $200 | The city of Yuma in this state has a record av... | Arizona | the city of yuma in this state has a record av... | arizona | 200 | 0.0 |

3 | 4680 | 2004-12-31 | Jeopardy! | THE COMPANY LINE | $200 | In 1963, live on "The Art Linkletter Show", th... | McDonald's | in 1963 live on the art linkletter show this c... | mcdonalds | 200 | 0.0 |

4 | 4680 | 2004-12-31 | Jeopardy! | EPITAPHS & TRIBUTES | $200 | Signer of the Dec. of Indep., framer of the Co... | John Adams | signer of the dec of indep framer of the const... | john adams | 200 | 0.0 |

We can see that on average the answer shows up in the question around 6% of the time. This tells us that its quite difficult to deduce the answer from the question.

## Recycled Questions

Let’s move to tackling our second question, where we need to find out how often new questions are repeats of older ones.

We also should remove uninformative words (e.g stopwords) in order to better understand the question overlap. For this we will use `nltk`

package to remove stopwords in the english language.

```
from nltk.corpus import stopwords
# sorty jeopary values in ascending order
jeopardy.sort_values('AirDate', inplace=True)
# stopwords and used terms
terms_used = set()
stop_words = set(stopwords.words('english'))
# function to calculate the number of times
# a term from previous questions has been used in new ones
def calculate_overlap(row):
"""
Returns a word match count (proportion) for each row containing
a question back to the 'question_overlap' columns.
The count is calculated used a set of words used in previous questions
Parameters:
row (pandas.Series): row from pandas dataframe
Returns:
match_count (float): match count proportion
"""
# split the question by space
split_question = row.split()
# keep only non-stop words
split_question = [word for word in split_question if word not in stop_words]
# initialize count
match_count = 0
# loop through words in split_question: count matches
for w in split_question:
if w in terms_used:
match_count += 1
# add words to terms used set
terms_used.add(w)
if len(split_question) > 0:
match_count /= len(split_question)
return match_count
# run the function on the clean_question column
jeopardy['question_overlap'] = jeopardy['clean_question'].apply(calculate_overlap)
# print the mean overlap
print("Average overlap of questions based on individual words is {}".format(jeopardy['question_overlap'].mean()))
```

```
Average overlap of questions based on individual words is 0.9285999315781669
```

Our results show that there is an 92% overlap between terms in new questions vs terms in old questions. This is quite a strong indicator that new questions tend to include terms from old questions. Nevertheless, this doesn’t help us as much since we are only looking at individual terms.

We could possibly investigate further and look at either phrases or find out how similar questions to each other. To keep this analysis brief, we will not cover it today.

## Low value vs high value questions

Nevertheless, we can also look at questions that pertain high value in terms of money. This strategy can help us earn more money when on Jeopardy. We can actually figure out which terms correspond to high-value questions using a chi-squared test. We’ll first need to narrow down the questions into two categories:

- Low value – Any row where Value is less than 800.
- High value – Any row where Value is greater than 800.

Let’s write the code which will help us identify value pairs for terms.

```
# function to find value
def calculate_value(row):
value = 1 if row['clean_value'] > 800 else 0
return value
# determine questions with high value
jeopardy['high_value'] = jeopardy.apply(calculate_value, axis=1)
# function to calculate value pairs (high, low count) for any word
def calc_word(word):
high_count = 0
low_count = 0
for index, row in jeopardy.iterrows():
q = row['clean_question'].split()
if word in q:
if row['high_value'] == 1:
high_count += 1
else:
low_count += 1
return high_count, low_count
```

```
# empty dictionary to hold the word and
# counts of high and low value questions the term appears in
observed_expected = {}
# sample terms to run the value function on
comparison_terms = list(terms_used)[:10]
# calculate value pairs (high and low count) for each word in the sample
for word in comparison_terms:
observed_expected[word] = calc_word(word)
```

```
observed_expected
```

```
{'flighty': (2, 6),
'pennants': (0, 2),
'miltons': (2, 3),
'hrefhttpwwwjarchivecommedia20110602j12jpg': (0, 1),
'civilizations': (4, 19),
'gethsemane': (1, 6),
'theatera': (1, 1),
'hotheaded': (1, 1),
'hrefhttpwwwjarchivecommedia20080403dj30jpg': (1, 0),
'targetblankolivia': (0, 1)}
```

## Applying chi-squared test

Now that we’ve found the observed counts for a few terms, we can compute the expected counts and the chi-squared value.

```
# find total sum of high and low value questions
high_value_count = jeopardy["high_value"].sum()
low_value_count = jeopardy.shape[0] - high_value_count
# dictionary to hold chisquare and p-value for each term
chi_squared = {}
# loop through the dictionary
for key, value in observed_expected.items():
# calculate total value of high and low
total = value[0] + value[1]
# calculate word occurence proportion across the dataset
total_prop = total / jeopardy.shape[0]
# find expected term for high and low value rows
high_exp, low_exp = total_prop * high_value_count, total_prop * low_value_count
# calculate chi-squared statistic value and associated p-value
chi_squared[key] = chisquare(value, (high_exp, low_exp))
```

```
chi_squared
```

```
{'flighty': Power_divergenceResult(statistic=0.043292301416985354, pvalue=0.8351758561462266),
'pennants': Power_divergenceResult(statistic=0.7899529284667026, pvalue=0.3741143592744989),
'miltons': Power_divergenceResult(statistic=0.3363947754070794, pvalue=0.5619176551024535),
'hrefhttpwwwjarchivecommedia20110602j12jpg': Power_divergenceResult(statistic=0.3949764642333513, pvalue=0.5296950912486695),
'civilizations': Power_divergenceResult(statistic=1.3519672340861204, pvalue=0.24493449653850868),
'gethsemane': Power_divergenceResult(statistic=0.6787070195906365, pvalue=0.4100323130964867),
'theatera': Power_divergenceResult(statistic=0.46338644448358013, pvalue=0.49604555208958945),
'hotheaded': Power_divergenceResult(statistic=0.46338644448358013, pvalue=0.49604555208958945),
'hrefhttpwwwjarchivecommedia20080403dj30jpg': Power_divergenceResult(statistic=2.5317964247338085, pvalue=0.11157312838169751),
'targetblankolivia': Power_divergenceResult(statistic=0.3949764642333513, pvalue=0.5296950912486695)}
```

The results of the chi-squared test show that all of the sample terms we chose are not statistical significant. So far this shows that none of the terms above correspond to high value questions if we defined test significance acceptance level of $p=0.05$

## Exploring Categories

Apart from investigating the values of questions, we can also prepare for jeopardy by knowing the likelihood of certain categories appearning in each round. In our dataset we have a column `Category`

which we will use to answer the following questions:

- Which categories appear most often
- What is the probability of each category appearing in each round.

Now, the first question is pretty straight forward. We simply need to count occurences of each category. There are many unique categories, thus we will only look at top 10.

```
# count number of categories
unique_cats = len(jeopardy["Category"].unique())
print("Number of unique categories {}".format(unique_cats))
```

```
Number of unique categories 27995
```

```
# count category ocurrences
cat_occurs = jeopardy["Category"].value_counts().sort_values(ascending=False).head(10)
fig, ax = plt.subplots(figsize=(18, 12))
ax.bar(cat_occurs.index, cat_occurs)
[ax.spines[a].set_visible(False) for a in ax.spines]
plt.xticks(rotation=45)
plt.show()
```

The above top 10 categories have more questions than the rest of categories. The category ‘Before & After’ seems to be the most popular one. For more information on what this category means, you can read here.

Now, to find the probability of each category appearing in each round, we need to first group the questions by round and then look at the categories and basically calculate the proportion of questions for a specific category in that round. Let’s get to it.

```
# list of possible rounds
jeopardy["Round"].value_counts()
```

```
Jeopardy! 107384
Double Jeopardy! 105912
Final Jeopardy! 3631
Tiebreaker 3
Name: Round, dtype: int64
```

Jeopardy has only three main rounds with a potential tiebreaker in case of a tie. Let’s calculate the occurence of each category by round.

```
jeopardy["Round"].value_counts()
```

```
Jeopardy! 107384
Double Jeopardy! 105912
Final Jeopardy! 3631
Tiebreaker 3
Name: Round, dtype: int64
```

```
# get number of categories by round
catbyround = jeopardy[["Round", "Category", "Question"]].groupby(["Round", "Category"]).agg({'Question': 'count'})
# calculate the proportion of questions for each category in that round
catbyround["Percentage"] = catbyround.groupby(level=0).apply(lambda x: 100 * x / float(x.sum()))
# check results
catbyround
```

Since we have lots of categories in each round, we need to think of a way to condense it into a more readable format. For that we can only look at the top 10 categories in each round in terms of probability of occurence.

```
# groupby Round then take the top 10 largest
catbyround['Percentage'].groupby(level=0, group_keys=False).nlargest(10)
```

```
Round Category
Double Jeopardy! BEFORE & AFTER 0.424881
LITERATURE 0.359733
SCIENCE 0.279477
WORLD GEOGRAPHY 0.239822
OPERA 0.236045
WORLD HISTORY 0.223771
BALLET 0.217161
COLLEGES & UNIVERSITIES 0.207720
ART 0.202999
ISLANDS 0.202999
Final Jeopardy! U.S. PRESIDENTS 1.377031
WORD ORIGINS 0.936381
STATE CAPITALS 0.853759
AUTHORS 0.743597
WORLD LEADERS 0.716056
FAMOUS AMERICANS 0.660975
FAMOUS NAMES 0.660975
HISTORIC NAMES 0.660975
BUSINESS & INDUSTRY 0.633434
ORGANIZATIONS 0.523272
Jeopardy! POTPOURRI 0.237466
STUPID ANSWERS 0.237466
SPORTS 0.235603
ANIMALS 0.216978
AMERICAN HISTORY 0.211391
SCIENCE 0.202079
STATE CAPITALS 0.195560
TELEVISION 0.186247
U.S. CITIES 0.181591
BUSINESS & INDUSTRY 0.172279
Tiebreaker CHILD'S PLAY 33.333333
LITERARY CHARACTERS 33.333333
THE AMERICAN REVOLUTION 33.333333
Name: Percentage, dtype: float64
```

The above table is a very nice summary of the probability of each category ocurring within each round. Some notable observations are:

- It seems tiebreakers only have three category which equal probability of ocurrence
- US Presidents is a very popular topic in Final Jeopardy! Round with 1.37% chance of ocurrence, which relative to other categories is high
- Literature, science ans sports seem to be quite popular topics touched in every round

## Summary

Here, we have explored the wonderful world of jeopardy and tried to find interesting information that could give us an edge in better preparing for the game and ultimately win in it.

For a complete jupyter notebook file please refer to the github page of the project.

## Leave a comment