Notebook

HaFHL Stats¶

Preamble¶

I recently made some statistics in Excel for my Fantasy Hockey Pool, the Hipster and Friends Hockey League (HaFHL). I started wondering what kind of effort would be involved to replicate what I had done in Python. You can find the results of this question below. You can find the original Excel version here

Parsing the CSV Data¶

Let's first import the modules we're going to need to use plus define some handy lists we're going to need later on:

In [1]:

import csv
import StringIO
import sys

player_names = ["Dwayne Gretskeet", "South Park Cows", "the favorites", 
                "Pmoda", "Fresh Baked Brownies", "JoshHum", "Pika Pikachus", 
                "BenDover", "Mikhail's Team", "First Place", "Team Leaddz", 
                "EWeb FTW", "HERRO. THIS IS DOG", "Matthew's Team"]

scoring_categories = ["G", "A", "+/-", "PIM", "PPP", "SOG", "W", "GAA", "SV%",  "SHO"]

And now do the actual parsing into some data structures:

In [2]:

csv_data = """Player,G,A,+/-,PIM,PPP,SOG,W,GAA,SV%,SHO
Dwayne Gretskeet,154,191,26,379,109,1325,38,2.32,0.926,4
South Park Cows,141,191,46,282,115,1157,31,2.57,0.92,4
Mikhail's Team,138,160,40,268,97,1233,13,2.88,0.9,3
the favorites,132,153,82,307,76,1269,33,2.23,0.918,5
Pmoda,124,186,-25,318,97,1283,32,2.11,0.919,4
JoshHum,121,198,-22,227,115,1263,21,2.93,0.914,2
BenDover,119,206,74,248,88,1064,22,2.52,0.905,0
Pika Pikachus,113,201,59,177,103,1022,21,2.65,0.91,2
Fresh Baked Brownies,95,185,36,262,91,1092,25,2.01,0.926,5
Matthew's Team,89,169,9,256,97,1052,13,3.11,0.898,1
First Place,88,214,-51,186,108,932,29,2.64,0.909,2
HERRO. THIS IS DOG,83,148,51,245,70,912,19,2.84,0.914,2
EWeb FTW,81,149,-28,164,70,826,26,2.4,0.909,5
Team Leaddz,81,156,14,279,72,941,16,2.61,0.906,2
"""
f = StringIO.StringIO(csv_data)
reader = csv.DictReader(f)

per_player_stats = {i['Player']: {x: i[x] for x in scoring_categories} for i in reader}

Awesome. Let's take a look at our data (it should look pretty similar to our original CSV) by defining a print function for our data structure:

In [3]:

from IPython.core.display import HTML

def print_stats(stats, columns, keyword):
    output = '<table>'
    output += '<tr><th>' + keyword + '</th>'
    for col in columns:
        output += '<th>' + col + '</th>'
    output += "</tr>"
    
    for k,v in stats.iteritems():
        output += '<tr><td>' + k + '</td>'
        for col in columns:
            output += '<td>' + str(v[col]) + '</td>'
        output += '</tr>'
    output += "</table>"
    return HTML(output)
        
def print_players(players, columns):
    return print_stats(players, columns, "Player")
    
def print_categories(categories, columns):
    return print_stats(categories, columns, "Category")

print_players(per_player_stats, scoring_categories)

Out[3]:

Player	G	A	+/-	PIM	PPP	SOG	W	GAA	SV%	SHO
Pmoda	124	186	-25	318	97	1283	32	2.11	0.919	4
Matthew's Team	89	169	9	256	97	1052	13	3.11	0.898	1
South Park Cows	141	191	46	282	115	1157	31	2.57	0.92	4
JoshHum	121	198	-22	227	115	1263	21	2.93	0.914	2
Dwayne Gretskeet	154	191	26	379	109	1325	38	2.32	0.926	4
Fresh Baked Brownies	95	185	36	262	91	1092	25	2.01	0.926	5
First Place	88	214	-51	186	108	932	29	2.64	0.909	2
Pika Pikachus	113	201	59	177	103	1022	21	2.65	0.91	2
the favorites	132	153	82	307	76	1269	33	2.23	0.918	5
Mikhail's Team	138	160	40	268	97	1233	13	2.88	0.9	3
BenDover	119	206	74	248	88	1064	22	2.52	0.905	0
Team Leaddz	81	156	14	279	72	941	16	2.61	0.906	2
HERRO. THIS IS DOG	83	148	51	245	70	912	19	2.84	0.914	2
EWeb FTW	81	149	-28	164	70	826	26	2.4	0.909	5

Nice! Now let's get into the real fun: ranking the data. What is a rank you ask? Well, let's take Goals as an example: The person with the most Goals will have a ranking of 1, second most -- 2, etc. We're going to do this for each stat category. First, let's define a function to rank each category:

In [4]:

def rank(unranked, reverse=True):
    ranked = {}
    for x in unranked:
        ranked_values = sorted(unranked.values())
        if reverse:
            ranked_values.reverse()
        ranked[x] = ranked_values.index(unranked[x]) + 1 # add 1 to account for zero indexing
    return ranked

Great, but our data is structured such that it is indexed by player. But our ranking function takes a dictionary whose key is the stat category. Let's transpose our data so that this is the case:

In [5]:

def transpose_players_to_categories(players):
    stat_categories = []
    for player in players:
        stat_categories = players[player].keys()
        break
    
    categories = {category: {} for category in stat_categories}

    for stat in players:
        for cat in stat_categories:
            try:
                categories[cat][stat] = int(str(players[stat][cat]))
            except ValueError:
                categories[cat][stat] = float(str(players[stat][cat]))
    
    return categories
    
unranked_stats = transpose_players_to_categories(per_player_stats)

OK, so now we have our data in the right format, but it's still just the totals. We haven't ranked it yet. Let's print it to take a look at what our data looks like:

In [6]:

print_categories(unranked_stats, player_names)

Out[6]:

Category	Dwayne Gretskeet	South Park Cows	the favorites	Pmoda	Fresh Baked Brownies	JoshHum	Pika Pikachus	BenDover	Mikhail's Team	First Place	Team Leaddz	EWeb FTW	HERRO. THIS IS DOG	Matthew's Team
A	191	191	153	186	185	198	201	206	160	214	156	149	148	169
+/-	26	46	82	-25	36	-22	59	74	40	-51	14	-28	51	9
PIM	379	282	307	318	262	227	177	248	268	186	279	164	245	256
G	154	141	132	124	95	121	113	119	138	88	81	81	83	89
SOG	1325	1157	1269	1283	1092	1263	1022	1064	1233	932	941	826	912	1052
SV%	0.926	0.92	0.918	0.919	0.926	0.914	0.91	0.905	0.9	0.909	0.906	0.909	0.914	0.898
PPP	109	115	76	97	91	115	103	88	97	108	72	70	70	97
W	38	31	33	32	25	21	21	22	13	29	16	26	19	13
GAA	2.32	2.57	2.23	2.11	2.01	2.93	2.65	2.52	2.88	2.64	2.61	2.4	2.84	3.11
SHO	4	4	5	4	5	2	2	0	3	2	2	5	2	1

Let's rank the data now. We're also going to rank all of our data in the opposite order in a seperate dictionary. I'll come back to why this is necessary a little later: (Notice that for GAA, lower is actually better, so we rank it in the other direction)

In [7]:

ranked_stats = {}
for stat in unranked_stats:
    if stat == "GAA":
        ranked_stats[stat] = rank(unranked_stats[stat], False)
    else:
        ranked_stats[stat] = rank(unranked_stats[stat])
        
backwards = {}
for stat in unranked_stats:
    if stat != "GAA":
        backwards[stat] = rank(unranked_stats[stat], False)
    else:
        backwards[stat] = rank(unranked_stats[stat])

Perfect, let's display it:

In [8]:

print_categories(ranked_stats, player_names)

Out[8]:

Category	Dwayne Gretskeet	South Park Cows	the favorites	Pmoda	Fresh Baked Brownies	JoshHum	Pika Pikachus	BenDover	Mikhail's Team	First Place	Team Leaddz	EWeb FTW	HERRO. THIS IS DOG	Matthew's Team
A	5	5	12	7	8	4	3	2	10	1	11	13	14	9
+/-	8	5	1	12	7	11	3	2	6	14	9	13	4	10
PIM	1	4	3	2	7	11	13	9	6	12	5	14	10	8
G	1	2	4	5	9	6	8	7	3	11	13	13	12	10
SOG	1	6	3	2	7	4	10	8	5	12	11	14	13	9
SV%	1	3	5	4	1	6	8	12	13	9	11	9	6	14
PPP	3	1	11	6	9	1	5	10	6	4	12	13	13	6
W	1	4	2	3	7	9	9	8	13	5	12	6	11	13
GAA	4	7	3	2	1	13	10	6	12	9	8	5	11	14
SHO	4	4	1	4	1	8	8	14	7	8	8	1	8	13

D'oh, so close! The data is still transposed. That's no good. Let's transpose it back to it's original format (and also transpose our backwards version while we're at it):

In [9]:

players = {player: {} for player in per_player_stats}
for stat in ranked_stats:
    for player in players:
        players[player][stat] = ranked_stats[stat][player]
        
players_backwards = {player: {} for player in per_player_stats}
for stat in backwards:
    for player in players_backwards:
        players_backwards[player][stat] = backwards[stat][player]

Awesome! Let's print out the results:

In [10]:

print_players(players, scoring_categories)

Out[10]:

Player	G	A	+/-	PIM	PPP	SOG	W	GAA	SV%	SHO
Pmoda	5	7	12	2	6	2	3	2	4	4
Matthew's Team	10	9	10	8	6	9	13	14	14	13
South Park Cows	2	5	5	4	1	6	4	7	3	4
JoshHum	6	4	11	11	1	4	9	13	6	8
Dwayne Gretskeet	1	5	8	1	3	1	1	4	1	4
Fresh Baked Brownies	9	8	7	7	9	7	7	1	1	1
First Place	11	1	14	12	4	12	5	9	9	8
Mikhail's Team	3	10	6	6	6	5	13	12	13	7
the favorites	4	12	1	3	11	3	2	3	5	1
Pika Pikachus	8	3	3	13	5	10	9	10	8	8
BenDover	7	2	2	9	10	8	8	6	12	14
Team Leaddz	13	11	9	5	12	11	12	8	11	8
HERRO. THIS IS DOG	12	14	4	10	13	13	11	11	6	8
EWeb FTW	13	13	13	14	13	14	6	5	9	1

Super. We now have the basis for the rest of the work we're going to do. Let's try to add everyone's average rank:

In [11]:

for player in players:
    avg = sum(players[player].values())/float(len(players[player].values()))
    avg = int(avg * 100) / 100.0 # trick to round to at most 2 decimal places
    players[player]["AVG"] = avg
print_players(players, ["AVG"])

Out[11]:

Player	AVG
Pmoda	4.7
Matthew's Team	10.6
South Park Cows	4.09
JoshHum	7.3
Dwayne Gretskeet	2.9
Fresh Baked Brownies	5.7
First Place	8.5
Mikhail's Team	8.1
the favorites	4.5
Pika Pikachus	7.7
BenDover	7.8
Team Leaddz	10.0
HERRO. THIS IS DOG	10.19
EWeb FTW	10.1

Next, let's rank each player according to their average rank. At the same time I'm going to manually add in each player's actual rank according to Yahoo's scoring.

In [12]:

ranked_avg = rank(transpose_players_to_categories(players)["AVG"], False)

for player in players:
    players[player]["Rank"] = ranked_avg[player]

players["South Park Cows"]["Actual"] = 1
players["the favorites"]["Actual"] = 2
players["Dwayne Gretskeet"]["Actual"] = 3
players["Pmoda"]["Actual"] = 4
players["JoshHum"]["Actual"] = 5
players["Fresh Baked Brownies"]["Actual"] = 6
players["BenDover"]["Actual"] = 7
players["Mikhail's Team"]["Actual"] = 8
players["HERRO. THIS IS DOG"]["Actual"] = 9
players["First Place"]["Actual"] = 10
players["Pika Pikachus"]["Actual"] = 11
players["Matthew's Team"]["Actual"] = 12
players["EWeb FTW"]["Actual"] = 13
players["Team Leaddz"]["Actual"] = 14

    
print_players(players, ["AVG", "Rank", "Actual"])

Out[12]:

Player	AVG	Rank	Actual
Pmoda	4.7	4	4
Matthew's Team	10.6	14	12
South Park Cows	4.09	2	1
JoshHum	7.3	6	5
Dwayne Gretskeet	2.9	1	3
Fresh Baked Brownies	5.7	5	6
First Place	8.5	10	10
Mikhail's Team	8.1	9	8
the favorites	4.5	3	2
Pika Pikachus	7.7	7	11
BenDover	7.8	8	7
Team Leaddz	10.0	11	14
HERRO. THIS IS DOG	10.19	13	9
EWeb FTW	10.1	12	13

Alright, let's add everyone's diff between their Rank, and they Actual, Yahoo provided score:

In [13]:

for player in players:
    players[player]["Diff"] = players[player]["Rank"] - players[player]["Actual"]
    
print_players(players, ["AVG", "Rank", "Actual", "Diff"])

Out[13]:

Player	AVG	Rank	Actual	Diff
Pmoda	4.7	4	4	0
Matthew's Team	10.6	14	12	2
South Park Cows	4.09	2	1	1
JoshHum	7.3	6	5	1
Dwayne Gretskeet	2.9	1	3	-2
Fresh Baked Brownies	5.7	5	6	-1
First Place	8.5	10	10	0
Mikhail's Team	8.1	9	8	1
the favorites	4.5	3	2	1
Pika Pikachus	7.7	7	11	-4
BenDover	7.8	8	7	1
Team Leaddz	10.0	11	14	-3
HERRO. THIS IS DOG	10.19	13	9	4
EWeb FTW	10.1	12	13	-1

Alright, let's add the final few pieces of the puzzle to get back to where the Excel version was at. First the easy part, number of First and Top 5s per player:

In [14]:

for player in players:
    number_ones = 0
    top_fives = 0
    for cat in players[player]:
        if cat in scoring_categories:
            if players[player][cat] == 1:
                number_ones += 1
            if players[player][cat] < 6:
                top_fives += 1
    players[player]["First"] = number_ones
    players[player]["Top 5"] = top_fives
    
print_players(players, ["AVG", "Rank", "Actual", "Diff", "First", "Top 5"])

Out[14]:

Player	AVG	Rank	Actual	Diff	First	Top 5
Pmoda	4.7	4	4	0	0	7
Matthew's Team	10.6	14	12	2	0	0
South Park Cows	4.09	2	1	1	1	8
JoshHum	7.3	6	5	1	1	3
Dwayne Gretskeet	2.9	1	3	-2	5	9
Fresh Baked Brownies	5.7	5	6	-1	3	3
First Place	8.5	10	10	0	1	3
Mikhail's Team	8.1	9	8	1	0	2
the favorites	4.5	3	2	1	2	8
Pika Pikachus	7.7	7	11	-4	0	3
BenDover	7.8	8	7	1	0	2
Team Leaddz	10.0	11	14	-3	0	1
HERRO. THIS IS DOG	10.19	13	9	4	0	1
EWeb FTW	10.1	12	13	-1	1	2

It turns out getting the number of Last Place and Bottom 5 finishes is a bit trickier than First and Top 5. We always know that the player with Rank == 1 is in First Place. Likewise, if someone's Rank is between 1 and 5 they are in the Top 5. But is the opposite true for Last Place and Bottom 5? No -- because of ties. There are 14 Players, but if multiple people are tied for last, they're also tied for second to last, and are ranked as 13th rather than 14th. Hey, remember that "backwards" stuff we did a while back? Well, now is when it's important. Ranked the opposite way, we're able to essentially treat Last and Bottom 5 exactly the same way as we did First and Top 5.

In [15]:

for player in players_backwards:
    lasts = 0
    bottom_fives = 0
    for cat in players_backwards[player]:
        if cat in scoring_categories:
            if players_backwards[player][cat] == 1:
                lasts += 1
            if players_backwards[player][cat] < 6:
                bottom_fives += 1
    players[player]["Last"] = lasts
    players[player]["Bottom 5"] = bottom_fives
    
print_players(players, ["AVG", "Rank", "Actual", "Diff", "First", "Top 5", "Last", "Bottom 5"])

Out[15]:

Player	AVG	Rank	Actual	Diff	First	Top 5	Last	Bottom 5
Pmoda	4.7	4	4	0	0	7	0	1
Matthew's Team	10.6	14	12	2	0	0	3	6
South Park Cows	4.09	2	1	1	1	8	0	0
JoshHum	7.3	6	5	1	1	3	0	5
Dwayne Gretskeet	2.9	1	3	-2	5	9	0	0
Fresh Baked Brownies	5.7	5	6	-1	3	3	0	0
First Place	8.5	10	10	0	1	3	1	6
Mikhail's Team	8.1	9	8	1	0	2	1	4
the favorites	4.5	3	2	1	2	8	0	2
Pika Pikachus	7.7	7	11	-4	0	3	0	5
BenDover	7.8	8	7	1	0	2	1	3
Team Leaddz	10.0	11	14	-3	0	1	1	7
HERRO. THIS IS DOG	10.19	13	9	4	0	1	2	8
EWeb FTW	10.1	12	13	-1	1	2	4	7

Alright, let's take a look at everything all together now:

In [16]:

print_players(players, scoring_categories + ["AVG", "Rank", "Actual", "Diff", "First", "Top 5", "Last", "Bottom 5"])

Out[16]:

Player	G	A	+/-	PIM	PPP	SOG	W	GAA	SV%	SHO	AVG	Rank	Actual	Diff	First	Top 5	Last	Bottom 5
Pmoda	5	7	12	2	6	2	3	2	4	4	4.7	4	4	0	0	7	0	1
Matthew's Team	10	9	10	8	6	9	13	14	14	13	10.6	14	12	2	0	0	3	6
South Park Cows	2	5	5	4	1	6	4	7	3	4	4.09	2	1	1	1	8	0	0
JoshHum	6	4	11	11	1	4	9	13	6	8	7.3	6	5	1	1	3	0	5
Dwayne Gretskeet	1	5	8	1	3	1	1	4	1	4	2.9	1	3	-2	5	9	0	0
Fresh Baked Brownies	9	8	7	7	9	7	7	1	1	1	5.7	5	6	-1	3	3	0	0
First Place	11	1	14	12	4	12	5	9	9	8	8.5	10	10	0	1	3	1	6
Mikhail's Team	3	10	6	6	6	5	13	12	13	7	8.1	9	8	1	0	2	1	4
the favorites	4	12	1	3	11	3	2	3	5	1	4.5	3	2	1	2	8	0	2
Pika Pikachus	8	3	3	13	5	10	9	10	8	8	7.7	7	11	-4	0	3	0	5
BenDover	7	2	2	9	10	8	8	6	12	14	7.8	8	7	1	0	2	1	3
Team Leaddz	13	11	9	5	12	11	12	8	11	8	10.0	11	14	-3	0	1	1	7
HERRO. THIS IS DOG	12	14	4	10	13	13	11	11	6	8	10.19	13	9	4	0	1	2	8
EWeb FTW	13	13	13	14	13	14	6	5	9	1	10.1	12	13	-1	1	2	4	7

Conclusion¶

Aaanndddd, we're back to where we were with the Excel version. Wow, that took a bit longer than expected. It was fun, but took quite a bit more effort than the Excel version did. Lessons learned? Probably, but it's 5am and I'm going to sleep. Thanks for reading!

Category	Dwayne Gretskeet	South Park Cows	the favorites	Pmoda	Fresh Baked Brownies	JoshHum	Pika Pikachus	BenDover	Mikhail's Team	First Place	Team Leaddz	EWeb FTW	HERRO. THIS IS DOG	Matthew's Team
A	5	5	12	7	8	4	3	2	10	1	11	13	14	9
+/-	8	5	1	12	7	11	3	2	6	14	9	13	4	10
PIM	1	4	3	2	7	11	13	9	6	12	5	14	10	8
G	1	2	4	5	9	6	8	7	3	11	13	13	12	10
SOG	1	6	3	2	7	4	10	8	5	12	11	14	13	9
SV%	1	3	5	4	1	6	8	12	13	9	11	9	6	14
PPP	3	1	11	6	9	1	5	10	6	4	12	13	13	6
W	1	4	2	3	7	9	9	8	13	5	12	6	11	13
GAA	4	7	3	2	1	13	10	6	12	9	8	5	11	14
SHO	4	4	1	4	1	8	8	14	7	8	8	1	8	13

Category	Dwayne Gretskeet	South Park Cows	the favorites	Pmoda	Fresh Baked Brownies	JoshHum	Pika Pikachus	BenDover	Mikhail's Team	First Place	Team Leaddz	EWeb FTW	HERRO. THIS IS DOG	Matthew's Team
A	5	5	12	7	8	4	3	2	10	1	11	13	14	9
+/-	8	5	1	12	7	11	3	2	6	14	9	13	4	10
PIM	1	4	3	2	7	11	13	9	6	12	5	14	10	8
G	1	2	4	5	9	6	8	7	3	11	13	13	12	10
SOG	1	6	3	2	7	4	10	8	5	12	11	14	13	9
SV%	1	3	5	4	1	6	8	12	13	9	11	9	6	14
PPP	3	1	11	6	9	1	5	10	6	4	12	13	13	6
W	1	4	2	3	7	9	9	8	13	5	12	6	11	13
GAA	4	7	3	2	1	13	10	6	12	9	8	5	11	14
SHO	4	4	1	4	1	8	8	14	7	8	8	1	8	13

Category	Dwayne Gretskeet	South Park Cows	the favorites	Pmoda	Fresh Baked Brownies	JoshHum	Pika Pikachus	BenDover	Mikhail's Team	First Place	Team Leaddz	EWeb FTW	HERRO. THIS IS DOG	Matthew's Team
A	5	5	12	7	8	4	3	2	10	1	11	13	14	9
+/-	8	5	1	12	7	11	3	2	6	14	9	13	4	10
PIM	1	4	3	2	7	11	13	9	6	12	5	14	10	8
G	1	2	4	5	9	6	8	7	3	11	13	13	12	10
SOG	1	6	3	2	7	4	10	8	5	12	11	14	13	9
SV%	1	3	5	4	1	6	8	12	13	9	11	9	6	14
PPP	3	1	11	6	9	1	5	10	6	4	12	13	13	6
W	1	4	2	3	7	9	9	8	13	5	12	6	11	13
GAA	4	7	3	2	1	13	10	6	12	9	8	5	11	14
SHO	4	4	1	4	1	8	8	14	7	8	8	1	8	13