This notebook will demonstrate:
# Usually people put all of their imports together in the first cell of the notebook.
# I will talk more about this when it becomes relevant, but you can ignore it at first.
import pandas
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
ls
Ipython_Pandas_and_Matplotlib.ipynb demo notebook.ipynb data/ venv/
below is basic python code
1+15
16
a = 2
a + 5
7
print "Hello, CSTA"
Hello, CSTA
import time
while True:
print 'hello'
time.sleep(2)
hello hello hello hello hello hello hello hello hello
--------------------------------------------------------------------------- KeyboardInterrupt Traceback (most recent call last) <ipython-input-10-34b45b36b43d> in <module>() 1 while True: 2 print 'hello' ----> 3 time.sleep(2) KeyboardInterrupt:
here are some more resources on iPython Notebook basics:
http://nbviewer.ipython.org/github/ipython/ipython/blob/1.x/examples/notebooks/Part%201%20-%20Running%20Code.ipynb http://nbviewer.ipython.org/github/ipython/ipython/blob/1.x/examples/notebooks/Part%202%20-%20Basic%20Output.ipynb
Did I show you how cool tab completion is in iPython notebooks?
a.
pandas.
x = np.random.normal(size=10000).cumsum()
plt.plot(x)
Use github and http://nbviewer.ipython.org to share your notebooks!
You can download (or "export") your fitbit data here: http://www.fitbit.com/premium/export
Be aware that: (1) you can only get daily summary data this way and (2) this service costs money, but there is a one-week, one-time free trial. For the purposes of the PACC, the free trial should be enough (you can use my data to practice running your scripts on data in the same format and then download all of your data at once toward the end of the challenge).
There is also an API that you can use to get to your data. We'll send more instructions on that at the beginning of Part 2.
Below is what your fitbit data looks like from this export. Not a typical CSV file - this will confuse Pandas (especially the header lines in the middle of the file).
with open('data/fitbit_export_20150311.csv') as mydata:
print mydata.read()
Body Date,Weight,BMI,Fat "2014-11-25","115.72","21.49","0" "2014-11-26","115.72","21.49","0" "2014-11-27","115.72","21.49","0" "2014-11-28","115.72","21.49","0" "2014-11-29","115.72","21.49","0" "2014-11-30","115.72","21.49","0" "2014-12-01","115.72","21.49","0" "2014-12-02","115.72","21.49","0" "2014-12-03","115.72","21.49","0" "2014-12-04","115.72","21.49","0" "2014-12-05","115.72","21.49","0" "2014-12-06","115.72","21.49","0" "2014-12-07","115.72","21.49","0" "2014-12-08","115.72","21.49","0" "2014-12-09","115.72","21.49","0" "2014-12-10","115.72","21.49","0" "2014-12-11","115.72","21.49","0" "2014-12-12","115.72","21.49","0" "2014-12-13","115.72","21.49","0" "2014-12-14","115.72","21.49","0" "2014-12-15","115.72","21.49","0" "2014-12-16","115.72","21.49","0" "2014-12-17","115.72","21.49","0" "2014-12-18","115.72","21.49","0" "2014-12-19","115.72","21.49","0" "2014-12-20","115.72","21.49","0" "2014-12-21","115.72","21.49","0" "2014-12-22","115.72","21.49","0" "2014-12-23","115.72","21.49","0" "2014-12-24","115.72","21.49","0" "2014-12-25","115.72","21.49","0" "2014-12-26","115.72","21.49","0" "2014-12-27","115.72","21.49","0" "2014-12-28","115.72","21.49","0" "2014-12-29","115.72","21.49","0" "2014-12-30","115.72","21.49","0" "2014-12-31","115.72","21.49","0" "2015-01-01","115.72","21.49","0" "2015-01-02","115.72","21.49","0" "2015-01-03","115.72","21.49","0" "2015-01-04","115.72","21.49","0" "2015-01-05","115.72","21.49","0" "2015-01-06","115.72","21.49","0" "2015-01-07","115.72","21.49","0" "2015-01-08","115.72","21.49","0" "2015-01-09","115.72","21.49","0" "2015-01-10","115.72","21.49","0" "2015-01-11","115.72","21.49","0" "2015-01-12","115.72","21.49","0" "2015-01-13","115.72","21.49","0" "2015-01-14","115.72","21.49","0" "2015-01-15","115.72","21.49","0" "2015-01-16","115.72","21.49","0" "2015-01-17","115.72","21.49","0" "2015-01-18","115.72","21.49","0" "2015-01-19","115.72","21.49","0" "2015-01-20","115.72","21.49","0" "2015-01-21","115.72","21.49","0" "2015-01-22","115.72","21.49","0" "2015-01-23","115.72","21.49","0" "2015-01-24","115.72","21.49","0" "2015-01-25","115.72","21.49","0" "2015-01-26","115.72","21.49","0" "2015-01-27","115.72","21.49","0" "2015-01-28","115.72","21.49","0" "2015-01-29","115.72","21.49","0" "2015-01-30","115.72","21.49","0" "2015-01-31","115.72","21.49","0" "2015-02-01","115.72","21.49","0" "2015-02-02","115.72","21.49","0" "2015-02-03","115.72","21.49","0" "2015-02-04","115.72","21.49","0" "2015-02-05","115.72","21.49","0" "2015-02-06","115.72","21.49","0" "2015-02-07","115.72","21.49","0" "2015-02-08","115.72","21.49","0" "2015-02-09","115.72","21.49","0" "2015-02-10","115.72","21.49","0" "2015-02-11","115.72","21.49","0" "2015-02-12","115.72","21.49","0" "2015-02-13","115.72","21.49","0" "2015-02-14","115.72","21.49","0" "2015-02-15","115.72","21.49","0" "2015-02-16","115.72","21.49","0" "2015-02-17","115.72","21.49","0" "2015-02-18","115.72","21.49","0" Foods Date,Calories In "2014-11-25","0" "2014-11-26","0" "2014-11-27","0" "2014-11-28","0" "2014-11-29","0" "2014-11-30","0" "2014-12-01","0" "2014-12-02","0" "2014-12-03","0" "2014-12-04","0" "2014-12-05","0" "2014-12-06","0" "2014-12-07","0" "2014-12-08","0" "2014-12-09","0" "2014-12-10","0" "2014-12-11","0" "2014-12-12","0" "2014-12-13","0" "2014-12-14","0" "2014-12-15","0" "2014-12-16","0" "2014-12-17","0" "2014-12-18","0" "2014-12-19","0" "2014-12-20","0" "2014-12-21","0" "2014-12-22","0" "2014-12-23","0" "2014-12-24","0" "2014-12-25","0" "2014-12-26","0" "2014-12-27","0" "2014-12-28","0" "2014-12-29","0" "2014-12-30","0" "2014-12-31","0" "2015-01-01","0" "2015-01-02","0" "2015-01-03","0" "2015-01-04","0" "2015-01-05","0" "2015-01-06","0" "2015-01-07","0" "2015-01-08","0" "2015-01-09","0" "2015-01-10","0" "2015-01-11","0" "2015-01-12","0" "2015-01-13","0" "2015-01-14","0" "2015-01-15","0" "2015-01-16","0" "2015-01-17","0" "2015-01-18","0" "2015-01-19","0" "2015-01-20","0" "2015-01-21","0" "2015-01-22","0" "2015-01-23","0" "2015-01-24","0" "2015-01-25","0" "2015-01-26","0" "2015-01-27","0" "2015-01-28","0" "2015-01-29","0" "2015-01-30","0" "2015-01-31","0" "2015-02-01","0" "2015-02-02","0" "2015-02-03","0" "2015-02-04","0" "2015-02-05","0" "2015-02-06","0" "2015-02-07","0" "2015-02-08","0" "2015-02-09","0" "2015-02-10","0" "2015-02-11","0" "2015-02-12","0" "2015-02-13","0" "2015-02-14","0" "2015-02-15","0" "2015-02-16","0" "2015-02-17","0" "2015-02-18","0" Activities Date,Calories Burned,Steps,Distance,Floors,Minutes Sedentary,Minutes Lightly Active,Minutes Fairly Active,Minutes Very Active,Activity Calories "2014-11-25","1,222","0","0","0","1,440","0","0","0","0" "2014-11-26","1,222","0","0","0","1,440","0","0","0","0" "2014-11-27","1,222","0","0","0","1,440","0","0","0","0" "2014-11-28","1,222","0","0","0","1,440","0","0","0","0" "2014-11-29","1,222","0","0","0","1,440","0","0","0","0" "2014-11-30","1,222","0","0","0","1,440","0","0","0","0" "2014-12-01","1,222","0","0","0","1,440","0","0","0","0" "2014-12-02","1,222","0","0","0","1,440","0","0","0","0" "2014-12-03","1,222","0","0","0","1,440","0","0","0","0" "2014-12-04","1,222","0","0","0","1,440","0","0","0","0" "2014-12-05","1,222","0","0","0","1,440","0","0","0","0" "2014-12-06","1,222","0","0","0","1,440","0","0","0","0" "2014-12-07","1,222","0","0","0","1,440","0","0","0","0" "2014-12-08","1,222","0","0","0","1,440","0","0","0","0" "2014-12-09","1,222","0","0","0","1,440","0","0","0","0" "2014-12-10","1,222","0","0","0","1,440","0","0","0","0" "2014-12-11","1,222","0","0","0","1,440","0","0","0","0" "2014-12-12","1,222","0","0","0","1,440","0","0","0","0" "2014-12-13","1,222","0","0","0","1,440","0","0","0","0" "2014-12-14","1,222","0","0","0","1,440","0","0","0","0" "2014-12-15","1,222","0","0","0","1,440","0","0","0","0" "2014-12-16","1,222","0","0","0","1,440","0","0","0","0" "2014-12-17","1,222","0","0","0","1,440","0","0","0","0" "2014-12-18","1,222","0","0","0","1,440","0","0","0","0" "2014-12-19","1,222","0","0","0","1,440","0","0","0","0" "2014-12-20","1,222","0","0","0","1,440","0","0","0","0" "2014-12-21","1,222","0","0","0","1,440","0","0","0","0" "2014-12-22","1,222","0","0","0","1,440","0","0","0","0" "2014-12-23","1,222","0","0","0","1,440","0","0","0","0" "2014-12-24","1,222","0","0","0","1,440","0","0","0","0" "2014-12-25","1,222","0","0","0","1,440","0","0","0","0" "2014-12-26","1,729","3,620","1.45","0","1,247","141","52","0","474" "2014-12-27","1,545","4,490","1.8","0","1,322","66","32","20","374" "2014-12-28","1,611","4,723","1.9","0","1,254","106","78","2","492" "2014-12-29","1,639","5,335","2.16","0","1,258","119","49","14","519" "2014-12-30","1,442","2,555","1.03","0","1,339","70","24","7","273" "2014-12-31","1,576","4,482","1.8","0","1,290","91","46","13","428" "2015-01-01","1,492","2,593","1.04","0","1,305","96","39","0","332" "2015-01-02","1,517","4,196","1.68","0","1,297","54","30","20","340" "2015-01-03","1,524","3,397","1.36","0","764","108","30","6","380" "2015-01-04","1,740","6,839","2.75","0","1,209","145","64","22","657" "2015-01-05","2,064","13,118","5.37","0","1,023","149","111","52","1,036" "2015-01-06","1,902","10,727","4.45","0","705","107","74","53","818" "2015-01-07","1,886","10,022","4.1","0","570","134","75","46","829" "2015-01-08","2,300","18,950","7.68","0","500","121","122","105","1,314" "2015-01-09","2,171","15,803","6.56","0","583","135","64","95","1,136" "2015-01-10","1,789","7,965","3.21","0","736","132","55","37","697" "2015-01-11","1,564","4,524","1.82","0","867","86","31","20","412" "2015-01-12","1,872","10,063","4.07","0","735","120","74","47","794" "2015-01-13","2,053","12,232","4.98","0","588","184","98","50","1,044" "2015-01-14","1,866","9,979","4.26","0","719","118","66","48","798" "2015-01-15","2,177","16,538","6.72","0","1,119","137","93","91","1,185" "2015-01-16","1,468","3,229","1.33","0","1,335","66","26","13","308" "2015-01-17","1,222","0","0","0","1,440","0","0","0","0" "2015-01-18","1,222","0","0","0","1,440","0","0","0","0" "2015-01-19","1,222","0","0","0","1,440","0","0","0","0" "2015-01-20","1,222","0","0","0","1,440","0","0","0","0" "2015-01-21","1,222","0","0","0","1,440","0","0","0","0" "2015-01-22","1,222","0","0","0","1,440","0","0","0","0" "2015-01-23","1,222","0","0","0","1,440","0","0","0","0" "2015-01-24","1,222","0","0","0","1,440","0","0","0","0" "2015-01-25","1,222","0","0","0","1,440","0","0","0","0" "2015-01-26","1,222","0","0","0","1,440","0","0","0","0" "2015-01-27","1,222","0","0","0","1,440","0","0","0","0" "2015-01-28","1,222","0","0","0","1,440","0","0","0","0" "2015-01-29","1,222","0","0","0","1,440","0","0","0","0" "2015-01-30","1,222","0","0","0","1,440","0","0","0","0" "2015-01-31","1,222","0","0","0","1,440","0","0","0","0" "2015-02-01","1,222","0","0","0","1,440","0","0","0","0" "2015-02-02","1,222","0","0","0","1,440","0","0","0","0" "2015-02-03","1,222","0","0","0","1,440","0","0","0","0" "2015-02-04","1,222","0","0","0","1,440","0","0","0","0" "2015-02-05","1,222","0","0","0","1,440","0","0","0","0" "2015-02-06","1,222","0","0","0","1,440","0","0","0","0" "2015-02-07","1,222","0","0","0","1,440","0","0","0","0" "2015-02-08","1,222","0","0","0","1,440","0","0","0","0" "2015-02-09","1,222","0","0","0","1,440","0","0","0","0" "2015-02-10","1,222","0","0","0","1,440","0","0","0","0" "2015-02-11","1,222","0","0","0","1,440","0","0","0","0" "2015-02-12","1,222","0","0","0","1,440","0","0","0","0" "2015-02-13","1,222","0","0","0","1,440","0","0","0","0" "2015-02-14","1,222","0","0","0","1,440","0","0","0","0" "2015-02-15","1,222","0","0","0","1,440","0","0","0","0" "2015-02-16","1,222","0","0","0","1,440","0","0","0","0" "2015-02-17","1,222","0","0","0","1,440","0","0","0","0" "2015-02-18","1,222","0","0","0","1,440","0","0","0","0" Sleep Date,Minutes Asleep,Minutes Awake,Number of Awakenings,Time in Bed "2014-11-25","0","0","0","0" "2014-11-26","0","0","0","0" "2014-11-27","0","0","0","0" "2014-11-28","0","0","0","0" "2014-11-29","0","0","0","0" "2014-11-30","0","0","0","0" "2014-12-01","0","0","0","0" "2014-12-02","0","0","0","0" "2014-12-03","0","0","0","0" "2014-12-04","0","0","0","0" "2014-12-05","0","0","0","0" "2014-12-06","0","0","0","0" "2014-12-07","0","0","0","0" "2014-12-08","0","0","0","0" "2014-12-09","0","0","0","0" "2014-12-10","0","0","0","0" "2014-12-11","0","0","0","0" "2014-12-12","0","0","0","0" "2014-12-13","0","0","0","0" "2014-12-14","0","0","0","0" "2014-12-15","0","0","0","0" "2014-12-16","0","0","0","0" "2014-12-17","0","0","0","0" "2014-12-18","0","0","0","0" "2014-12-19","0","0","0","0" "2014-12-20","0","0","0","0" "2014-12-21","0","0","0","0" "2014-12-22","0","0","0","0" "2014-12-23","0","0","0","0" "2014-12-24","0","0","0","0" "2014-12-25","0","0","0","0" "2014-12-26","0","0","0","0" "2014-12-27","0","0","0","0" "2014-12-28","0","0","0","0" "2014-12-29","0","0","0","0" "2014-12-30","0","0","0","0" "2014-12-31","0","0","0","0" "2015-01-01","0","0","0","0" "2015-01-02","0","0","0","0" "2015-01-03","420","144","14","571" "2015-01-04","0","0","0","0" "2015-01-05","0","0","0","0" "2015-01-06","462","32","12","591" "2015-01-07","497","73","17","570" "2015-01-08","435","60","12","564" "2015-01-09","527","32","11","591" "2015-01-10","445","25","12","480" "2015-01-11","402","17","11","480" "2015-01-12","422","58","17","480" "2015-01-13","401","9","7","445" "2015-01-14","524","21","10","564" "2015-01-15","0","0","0","0" "2015-01-16","0","0","0","0" "2015-01-17","0","0","0","0" "2015-01-18","0","0","0","0" "2015-01-19","0","0","0","0" "2015-01-20","0","0","0","0" "2015-01-21","0","0","0","0" "2015-01-22","0","0","0","0" "2015-01-23","0","0","0","0" "2015-01-24","0","0","0","0" "2015-01-25","0","0","0","0" "2015-01-26","0","0","0","0" "2015-01-27","0","0","0","0" "2015-01-28","0","0","0","0" "2015-01-29","0","0","0","0" "2015-01-30","0","0","0","0" "2015-01-31","0","0","0","0" "2015-02-01","0","0","0","0" "2015-02-02","0","0","0","0" "2015-02-03","0","0","0","0" "2015-02-04","0","0","0","0" "2015-02-05","0","0","0","0" "2015-02-06","0","0","0","0" "2015-02-07","0","0","0","0" "2015-02-08","0","0","0","0" "2015-02-09","0","0","0","0" "2015-02-10","0","0","0","0" "2015-02-11","0","0","0","0" "2015-02-12","0","0","0","0" "2015-02-13","0","0","0","0" "2015-02-14","0","0","0","0" "2015-02-15","0","0","0","0" "2015-02-16","0","0","0","0" "2015-02-17","0","0","0","0" "2015-02-18","0","0","0","0" food_log_20141125 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141126 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141127 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141128 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141129 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141130 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141201 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141202 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141203 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141204 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141205 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141206 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141207 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141208 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141209 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141210 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141211 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141212 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141213 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141214 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141215 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141216 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141217 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141218 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141219 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141220 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141221 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141222 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141223 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141224 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141225 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141226 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141227 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141228 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141229 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141230 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20141231 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150101 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150102 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150103 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150104 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150105 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150106 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150107 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150108 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","63.2 fl oz" food_log_20150109 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0.9 fl oz" food_log_20150110 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150111 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150112 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150113 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150114 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150115 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150116 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150117 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150118 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150119 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150120 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150121 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150122 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150123 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150124 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150125 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150126 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150127 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150128 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150129 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150130 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150131 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150201 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150202 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150203 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150204 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150205 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150206 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150207 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150208 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150209 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150210 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150211 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150212 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150213 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150214 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150215 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150216 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150217 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz" food_log_20150218 Daily Totals "","Calories","0" "","Fat","0 g" "","Fiber","0 g" "","Carbs","0 g" "","Sodium","0 mg" "","Protein","0 g" "","Water","0 fl oz"
I created a file for each of the different types of fitbit data that I actually had entries for (sleep, steps, scale) by copy/pasting into new text files. Now they look like this:
(Note that I'm using terminal commands here to show this info)
ls data/fitbit*
data/fitbit_export_20150311.csv data/fitbit_sleep.csv data/fitbit_scale.csv data/fitbit_steps.csv
!head data/fitbit_steps.csv
Date,Calories Burned,Steps,Distance,Floors,Minutes Sedentary,Minutes Lightly Active,Minutes Fairly Active,Minutes Very Active,Activity Calories "2014-11-25","1,222","0","0","0","1,440","0","0","0","0" "2014-11-26","1,222","0","0","0","1,440","0","0","0","0" "2014-11-27","1,222","0","0","0","1,440","0","0","0","0" "2014-11-28","1,222","0","0","0","1,440","0","0","0","0" "2014-11-29","1,222","0","0","0","1,440","0","0","0","0" "2014-11-30","1,222","0","0","0","1,440","0","0","0","0" "2014-12-01","1,222","0","0","0","1,440","0","0","0","0" "2014-12-02","1,222","0","0","0","1,440","0","0","0","0" "2014-12-03","1,222","0","0","0","1,440","0","0","0","0"
You could just process all of this by hand ...
day_to_steps = {}
with open('data/fitbit_steps.csv') as my_steps:
my_steps.readline() # get past the header
for line in my_steps:
line = line.replace('"', '') # get rid of those pesky quotation marks that prevent the numbers from being read as ints
line = line.split(',')
day_to_steps[line[0]] = int(line[2])
print sum(day_to_steps.values())
26135
But it starts to get old pretty fast. Not the best. Here's where pandas comes in.
steps = pandas.read_csv('data/fitbit_steps.csv')
steps
<class 'pandas.core.frame.DataFrame'> Int64Index: 86 entries, 0 to 85 Data columns (total 10 columns): Date 86 non-null values Calories Burned 86 non-null values Steps 86 non-null values Distance 86 non-null values Floors 86 non-null values Minutes Sedentary 86 non-null values Minutes Lightly Active 86 non-null values Minutes Fairly Active 86 non-null values Minutes Very Active 86 non-null values Activity Calories 86 non-null values dtypes: float64(1), int64(4), object(5)
steps.Steps[42:60] # indexing (treating it like an array)
step_sum = 0
for s in steps.Steps:
step_sum = step_sum + int(s)
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-22-058ff7e3a4a8> in <module>() 1 step_sum = 0 2 for s in steps.Steps: ----> 3 step_sum = step_sum + int(s) ValueError: invalid literal for int() with base 10: '3,620'
steps_better = pandas.read_csv('data/fitbit_steps.csv', thousands=',')
step_sum = 0
for s in steps_better.Steps:
step_sum = step_sum + int(s)
step_sum
175380
volleyball_player = pandas.read_csv('data/player_data_volleyball.csv')
soccer_player = pandas.read_csv('data/player_data_soccer.csv')
volleyball_player
volleyball_player['GEN. SORNESS'][:10] # you might want to make the column names not include spaces/special characters,
# but here's another way to get at the data (besides the '.')
0 3 1 4 2 3 3 3 4 4 5 3 6 4 7 2 8 2 9 2 Name: GEN. SORNESS, dtype: int64
More on getting started with Pandas (includes some plotting basics):
volleyball_player.head() # prettier formatting
Athlete ID | Day of Week (1 is Sunday) | Date | FATIGUE | SLEEP QUALITY | GEN. SORNESS | STRESS | MOOD | HOURS SLEPT | |
---|---|---|---|---|---|---|---|---|---|
0 | 15 | 1 | 8/24/14 | 3 | 4 | 3 | 4 | 4 | 8 |
1 | 15 | 2 | 8/25/14 | 3 | 4 | 4 | 4 | 4 | 8 |
2 | 15 | 3 | 8/26/14 | 3 | 4 | 3 | 3 | 4 | 8 |
3 | 15 | 4 | 8/27/14 | 4 | 3 | 3 | 3 | 4 | 8 |
4 | 15 | 5 | 8/28/14 | 3 | 4 | 4 | 4 | 4 | 8 |
volleyball_player.describe() # to get general statistics about your data
Athlete ID | Day of Week (1 is Sunday) | FATIGUE | SLEEP QUALITY | GEN. SORNESS | STRESS | MOOD | HOURS SLEPT | |
---|---|---|---|---|---|---|---|---|
count | 1157.000000 | 1157.000000 | 1157.000000 | 1157.000000 | 1157.000000 | 1157.000000 | 1157.000000 | 1157.000000 |
mean | 7.976664 | 3.946413 | 3.066551 | 3.648228 | 3.138289 | 3.041487 | 3.828868 | 8.177182 |
std | 4.368649 | 1.969639 | 0.763503 | 0.704551 | 0.685915 | 0.792487 | 0.565835 | 1.209885 |
min | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 4.000000 |
25% | 4.000000 | 2.000000 | 3.000000 | 3.000000 | 3.000000 | 3.000000 | 4.000000 | 7.500000 |
50% | 8.000000 | 4.000000 | 3.000000 | 4.000000 | 3.000000 | 3.000000 | 4.000000 | 8.000000 |
75% | 12.000000 | 6.000000 | 4.000000 | 4.000000 | 4.000000 | 4.000000 | 4.000000 | 9.000000 |
max | 15.000000 | 7.000000 | 5.000000 | 5.000000 | 5.000000 | 5.000000 | 5.000000 | 12.000000 |
soccer_player.describe()
ATHLETE_ID | DAY_OF_WEEK | FATIGUE | SLEEP_QUALITY | GENERAL_SORNESS | STRESS | MOOD | HOURS_SLEPT | |
---|---|---|---|---|---|---|---|---|
count | 2457.000000 | 2457.000000 | 2457.000000 | 2457.000000 | 2457.000000 | 2457.000000 | 2457.000000 | 2457.000000 |
mean | 14.985755 | 4.056980 | 3.492063 | 3.741555 | 3.341880 | 3.564917 | 3.878714 | 8.115181 |
std | 8.425292 | 1.967159 | 0.790336 | 0.757398 | 0.876498 | 0.762490 | 0.636772 | 1.137960 |
min | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 1.000000 | 2.000000 |
25% | 8.000000 | 2.000000 | 3.000000 | 3.000000 | 3.000000 | 3.000000 | 4.000000 | 7.500000 |
50% | 15.000000 | 4.000000 | 4.000000 | 4.000000 | 3.000000 | 4.000000 | 4.000000 | 8.000000 |
75% | 22.000000 | 6.000000 | 4.000000 | 4.000000 | 4.000000 | 4.000000 | 4.000000 | 9.000000 |
max | 29.000000 | 7.000000 | 5.000000 | 5.000000 | 5.000000 | 5.000000 | 5.000000 | 15.000000 |
sorted_date = volleyball_player.sort_index(by='Date')
sorted_date.tail()
Athlete ID | Day of Week (1 is Sunday) | Date | FATIGUE | SLEEP QUALITY | GEN. SORNESS | STRESS | MOOD | HOURS SLEPT | |
---|---|---|---|---|---|---|---|---|---|
258 | 12 | 3 | 9/9/14 | 2 | 2 | 3 | 4 | 4 | 6 |
859 | 4 | 3 | 9/9/14 | 4 | 3 | 4 | 4 | 4 | 7 |
613 | 7 | 3 | 9/9/14 | 3 | 3 | 3 | 3 | 4 | 7 |
1020 | 2 | 3 | 9/9/14 | 3 | 4 | 3 | 4 | 4 | 8 |
96 | 14 | 3 | 9/9/14 | 4 | 5 | 4 | 4 | 4 | 8 |
sorted_sleep = volleyball_player.sort_index(by='SLEEP QUALITY')
sorted_sleep.head()
Athlete ID | Day of Week (1 is Sunday) | Date | FATIGUE | SLEEP QUALITY | GEN. SORNESS | STRESS | MOOD | HOURS SLEPT | |
---|---|---|---|---|---|---|---|---|---|
925 | 3 | 4 | 8/27/14 | 3 | 1 | 3 | 2 | 3 | 6 |
598 | 7 | 2 | 8/25/14 | 2 | 1 | 3 | 3 | 3 | 4 |
793 | 5 | 5 | 9/25/14 | 2 | 2 | 2 | 3 | 3 | 7 |
489 | 9 | 1 | 10/5/14 | 2 | 2 | 3 | 3 | 3 | 9 |
908 | 4 | 4 | 10/29/14 | 2 | 2 | 4 | 3 | 4 | 7 |
date_avgs = volleyball_player.groupby('Date').aggregate(np.average)
date_avgs.head()
Athlete ID | Day of Week (1 is Sunday) | FATIGUE | SLEEP QUALITY | GEN. SORNESS | STRESS | MOOD | HOURS SLEPT | |
---|---|---|---|---|---|---|---|---|
Date | ||||||||
10/1/14 | 8.000000 | 4 | 3.200000 | 3.666667 | 3.066667 | 2.866667 | 3.800000 | 8.333333 |
10/10/14 | 7.857143 | 6 | 3.285714 | 3.714286 | 3.357143 | 3.000000 | 3.928571 | 8.357143 |
10/11/14 | 8.000000 | 7 | 2.933333 | 3.600000 | 2.933333 | 2.933333 | 3.666667 | 8.000000 |
10/12/14 | 7.692308 | 1 | 3.076923 | 3.692308 | 3.000000 | 3.000000 | 3.846154 | 7.769231 |
10/13/14 | 8.000000 | 2 | 3.066667 | 3.533333 | 2.933333 | 3.066667 | 3.866667 | 8.700000 |
# empty cells for live demos - ask questions!
Here are the basics of plotting stuff in matplotlib.
Scatter plots & line graphs
x = [1,2,3]
y = [2,4,6]
plt.plot(x,y)
[<matplotlib.lines.Line2D at 0x106b9a6d0>]
plt.plot(x,y, marker='o')
[<matplotlib.lines.Line2D at 0x106ce7090>]
plt.scatter(x,y)
<matplotlib.collections.PathCollection at 0x106c016d0>
x1 = volleyball_player.MOOD
y1 = volleyball_player.STRESS
plt.scatter(x1,y1)
<matplotlib.collections.PathCollection at 0x106bfbe10>
Histograms
my_data = [1,1,1,2,2,3,1,1,2,3,4,4,2,1]
plt.hist(my_data, bins=2)
(array([ 10., 4.]), array([ 1. , 2.5, 4. ]), <a list of 2 Patch objects>)