Import python packages. NumPy for numerical calculuation, Pandas for reading the csv file, matplotlib for plotting functionality, and PyLab for inline visual tools. Notice the aliasing for ease of use later in the program.
%pylab inline
import numpy as np
import pylab as pl
import pandas as pd
import matplotlib.pyplot as plt
Populating the interactive namespace from numpy and matplotlib
Read station data from comma delimited file that is in data directory. data is an object that is created from the method read_csv in the pandas package.
data = pd.read_csv('./data/CR1000_Franz_open_20131011.dat',skiprows=(0,2,3),header=0)
I have left these here so you can mess around with attributes and methods that can be used on the object. These do not need to be run. You could just just down to the plotting section if all you are trying to do is create a plot.
data
TIMESTAMP | RECORD | AirTC | RH | WndDir | WndSpd | SlrW_Avg | Rain_Tot | Batt_volt_Min | WndSpd_Max | Press | |
---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2013-07-09 14:40:00 | 0 | 32.44 | 74.33 | 212.70 | 0.209 | 832.00 | 0 | 13.17 | 1.183 | 976.7668 |
1 | 2013-07-09 14:50:00 | 1 | 32.54 | 68.82 | 227.70 | 2.502 | 819.00 | 0 | 13.17 | 2.502 | 976.7566 |
2 | 2013-07-09 15:00:00 | 2 | 33.07 | 70.03 | 184.60 | 0.549 | 804.00 | 0 | 13.16 | 3.045 | 976.7617 |
3 | 2013-07-09 15:10:00 | 3 | 33.84 | 67.18 | 170.80 | 0.758 | 791.70 | 0 | 13.16 | 1.254 | 976.6046 |
4 | 2013-07-09 15:20:00 | 4 | 33.17 | 68.23 | 240.50 | 0.791 | 779.20 | 0 | 13.16 | 3.182 | 976.6036 |
5 | 2013-07-09 15:30:00 | 5 | 33.02 | 70.31 | 228.80 | 0.412 | 767.10 | 0 | 13.15 | 3.770 | 976.6066 |
6 | 2013-07-09 15:40:00 | 6 | 33.03 | 66.42 | 94.20 | 1.163 | 741.40 | 0 | 13.15 | 2.888 | 976.6049 |
7 | 2013-07-09 15:50:00 | 7 | 33.12 | 65.56 | 256.10 | 2.639 | 657.30 | 0 | 13.14 | 3.182 | 976.6006 |
8 | 2013-07-09 16:00:00 | 8 | 33.60 | 64.68 | 230.40 | 0.934 | 682.20 | 0 | 13.14 | 2.110 | 976.6076 |
9 | 2013-07-09 16:10:00 | 9 | 33.02 | 64.95 | 219.30 | 1.927 | 680.10 | 0 | 13.14 | 3.097 | 976.6124 |
10 | 2013-07-09 16:20:00 | 10 | 34.19 | 63.33 | 212.80 | 0.679 | 641.90 | 0 | 13.14 | 1.751 | 976.6101 |
11 | 2013-07-09 16:30:00 | 11 | 33.51 | 63.28 | 180.30 | 2.071 | 614.70 | 0 | 13.14 | 2.757 | 976.4834 |
12 | 2013-07-09 16:40:00 | 12 | 33.84 | 64.59 | 136.60 | 0.562 | 666.20 | 0 | 13.14 | 3.561 | 976.4681 |
13 | 2013-07-09 16:50:00 | 13 | 33.87 | 66.62 | 227.90 | 0.529 | 579.10 | 0 | 13.14 | 1.555 | 976.4637 |
14 | 2013-07-09 17:00:00 | 14 | 33.90 | 62.77 | 177.80 | 1.176 | 566.40 | 0 | 13.14 | 2.208 | 976.4629 |
15 | 2013-07-09 17:10:00 | 15 | 34.40 | 61.97 | 337.50 | 0.444 | 513.40 | 0 | 13.14 | 1.810 | 976.4729 |
16 | 2013-07-09 17:20:00 | 16 | 33.12 | 65.60 | 221.90 | 0.046 | 418.60 | 0 | 13.14 | 1.895 | 976.4885 |
17 | 2013-07-09 17:30:00 | 17 | 33.17 | 65.87 | 146.50 | 0.072 | 341.10 | 0 | 13.14 | 1.202 | 976.3358 |
18 | 2013-07-09 17:40:00 | 18 | 33.65 | 65.09 | 297.30 | 0.359 | 366.40 | 0 | 13.14 | 2.038 | 976.4780 |
19 | 2013-07-09 17:50:00 | 19 | 33.77 | 63.20 | 227.20 | 1.418 | 413.30 | 0 | 13.15 | 1.738 | 976.3517 |
20 | 2013-07-09 18:00:00 | 20 | 34.04 | 63.59 | 283.70 | 1.365 | 416.20 | 0 | 13.14 | 1.895 | 976.3397 |
21 | 2013-07-09 18:10:00 | 21 | 33.12 | 65.36 | 283.40 | 0.699 | 269.30 | 0 | 13.14 | 1.869 | 976.3406 |
22 | 2013-07-09 18:20:00 | 22 | 33.79 | 64.55 | 196.90 | 0.836 | 335.70 | 0 | 13.14 | 1.065 | 976.3682 |
23 | 2013-07-09 18:30:00 | 23 | 34.16 | 62.91 | 183.50 | 0.274 | 318.70 | 0 | 13.14 | 0.830 | 976.3685 |
24 | 2013-07-09 18:40:00 | 24 | 34.06 | 65.00 | 301.10 | 0.418 | 279.10 | 0 | 13.15 | 0.830 | 976.3771 |
25 | 2013-07-09 18:50:00 | 25 | 34.04 | 66.54 | 278.60 | 0.000 | 260.70 | 0 | 13.15 | 1.104 | 976.5154 |
26 | 2013-07-09 19:00:00 | 26 | 33.40 | 67.43 | 241.90 | 0.314 | 227.60 | 0 | 13.15 | 0.921 | 976.6382 |
27 | 2013-07-09 19:10:00 | 27 | 33.28 | 67.12 | 201.90 | 0.000 | 194.60 | 0 | 13.15 | 0.843 | 976.6376 |
28 | 2013-07-09 19:20:00 | 28 | 32.72 | 69.72 | 276.40 | 0.137 | 162.20 | 0 | 13.15 | 0.908 | 976.6537 |
29 | 2013-07-09 19:30:00 | 29 | 31.83 | 71.41 | 169.90 | 0.464 | 111.80 | 0 | 13.15 | 0.993 | 976.5190 |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
13509 | 2013-10-11 10:10:00 | 13509 | 18.42 | 81.80 | 150.50 | 0.751 | 401.20 | 0 | 13.44 | 3.678 | 973.9662 |
13510 | 2013-10-11 10:20:00 | 13510 | 18.94 | 80.60 | 137.20 | 1.137 | 394.60 | 0 | 13.43 | 2.822 | 973.6671 |
13511 | 2013-10-11 10:30:00 | 13511 | 19.27 | 79.69 | 144.50 | 3.038 | 508.50 | 0 | 13.42 | 3.959 | 973.2455 |
13512 | 2013-10-11 10:40:00 | 13512 | 19.54 | 81.10 | 144.20 | 0.830 | 435.10 | 0 | 13.41 | 2.999 | 973.3963 |
13513 | 2013-10-11 10:50:00 | 13513 | 19.18 | 80.40 | 120.30 | 2.979 | 320.30 | 0 | 13.40 | 4.508 | 973.6475 |
13514 | 2013-10-11 11:00:00 | 13514 | 19.76 | 79.57 | 131.50 | 1.993 | 352.50 | 0 | 13.40 | 3.704 | 973.5187 |
13515 | 2013-10-11 11:10:00 | 13515 | 20.17 | 79.50 | 113.50 | 1.300 | 355.10 | 0 | 13.39 | 3.084 | 973.3695 |
13516 | 2013-10-11 11:20:00 | 13516 | 20.83 | 77.98 | 157.10 | 2.038 | 440.70 | 0 | 13.38 | 3.071 | 973.5199 |
13517 | 2013-10-11 11:30:00 | 13517 | 21.27 | 77.88 | 123.50 | 1.810 | 479.50 | 0 | 13.37 | 4.116 | 973.4694 |
13518 | 2013-10-11 11:40:00 | 13518 | 21.05 | 77.48 | 97.00 | 2.496 | 324.90 | 0 | 13.37 | 3.332 | 973.1865 |
13519 | 2013-10-11 11:50:00 | 13519 | 20.76 | 79.25 | 125.50 | 0.836 | 337.30 | 0 | 13.36 | 3.842 | 973.0438 |
13520 | 2013-10-11 12:00:00 | 13520 | 20.40 | 80.20 | 138.50 | 2.156 | 169.90 | 0 | 13.36 | 3.855 | 973.0426 |
13521 | 2013-10-11 12:10:00 | 13521 | 20.42 | 80.10 | 141.70 | 0.902 | 154.60 | 0 | 13.36 | 2.888 | 973.1787 |
13522 | 2013-10-11 12:20:00 | 13522 | 20.62 | 80.30 | 111.30 | 2.078 | 168.40 | 0 | 13.37 | 3.430 | 973.0422 |
13523 | 2013-10-11 12:30:00 | 13523 | 20.72 | 80.10 | 142.00 | 1.509 | 169.40 | 0 | 13.37 | 3.410 | 972.6608 |
13524 | 2013-10-11 12:40:00 | 13524 | 20.47 | 80.70 | 183.40 | 1.731 | 78.75 | 0 | 13.34 | 2.483 | 972.8020 |
13525 | 2013-10-11 12:50:00 | 13525 | 20.85 | 80.40 | 178.10 | 2.568 | 143.60 | 0 | 13.37 | 3.678 | 972.6698 |
13526 | 2013-10-11 13:00:00 | 13526 | 21.19 | 80.30 | 122.50 | 1.300 | 162.20 | 0 | 13.37 | 3.456 | 972.8157 |
13527 | 2013-10-11 13:10:00 | 13527 | 21.36 | 80.20 | 92.80 | 0.915 | 148.10 | 0 | 13.37 | 3.214 | 972.6649 |
13528 | 2013-10-11 13:20:00 | 13528 | 21.41 | 80.70 | 233.70 | 0.673 | 142.10 | 0 | 13.36 | 4.580 | 972.3804 |
13529 | 2013-10-11 13:30:00 | 13529 | 21.12 | 80.60 | 170.80 | 1.098 | 77.83 | 0 | 13.36 | 2.953 | 972.3934 |
13530 | 2013-10-11 13:40:00 | 13530 | 21.13 | 80.30 | 147.30 | 1.359 | 78.87 | 0 | 13.36 | 2.352 | 972.1140 |
13531 | 2013-10-11 13:50:00 | 13531 | 21.36 | 80.30 | 159.40 | 2.607 | 100.30 | 0 | 13.37 | 3.045 | 971.9573 |
13532 | 2013-10-11 14:00:00 | 13532 | 21.65 | 79.65 | 191.20 | 1.235 | 150.00 | 0 | 13.37 | 4.658 | 971.8445 |
13533 | 2013-10-11 14:10:00 | 13533 | 21.90 | 79.20 | 135.10 | 1.294 | 164.80 | 0 | 13.36 | 4.403 | 971.8303 |
13534 | 2013-10-11 14:20:00 | 13534 | 21.92 | 79.13 | 97.80 | 1.620 | 164.00 | 0 | 13.35 | 3.633 | 971.6807 |
13535 | 2013-10-11 14:30:00 | 13535 | 21.75 | 79.61 | 21.27 | 1.555 | 103.90 | 0 | 13.35 | 4.430 | 971.6780 |
13536 | 2013-10-11 14:40:00 | 13536 | 21.73 | 79.49 | 117.60 | 0.444 | 103.40 | 0 | 13.35 | 3.567 | 971.6774 |
13537 | 2013-10-11 14:50:00 | 13537 | 21.90 | 79.80 | 345.80 | 0.784 | 120.50 | 0 | 13.35 | 3.378 | 971.2759 |
13538 | 2013-10-11 15:00:00 | 13538 | 22.04 | 79.39 | 119.10 | 2.058 | 143.00 | 0 | 13.33 | 3.802 | 971.2694 |
13539 rows × 11 columns
data.dtypes
TIMESTAMP object RECORD int64 AirTC float64 RH float64 WndDir float64 WndSpd float64 SlrW_Avg float64 Rain_Tot float64 Batt_volt_Min float64 WndSpd_Max float64 Press float64 dtype: object
data.columns
Index([u'TIMESTAMP', u'RECORD', u'AirTC', u'RH', u'WndDir', u'WndSpd', u'SlrW_Avg', u'Rain_Tot', u'Batt_volt_Min', u'WndSpd_Max', u'Press'], dtype='object')
data.values
array([['2013-07-09 14:40:00', 0, 32.439999999999998, ..., 13.17, 1.1830000000000001, 976.76679999999999], ['2013-07-09 14:50:00', 1, 32.539999999999999, ..., 13.17, 2.5019999999999998, 976.75660000000005], ['2013-07-09 15:00:00', 2, 33.07, ..., 13.16, 3.0449999999999999, 976.76170000000002], ..., ['2013-10-11 14:40:00', 13536, 21.73, ..., 13.35, 3.5669999999999997, 971.67740000000003], ['2013-10-11 14:50:00', 13537, 21.899999999999999, ..., 13.35, 3.3780000000000001, 971.27589999999998], ['2013-10-11 15:00:00', 13538, 22.039999999999999, ..., 13.33, 3.802, 971.26940000000002]], dtype=object)
Ok, Let's finally get down to the fun part, plotting. This is the simplest way to make a plot.
plt.plot(data.AirTC)
plt.show()
You can also specify the X and Y axes by providing two arguments to the plot method.
plt.plot(data.RECORD,data.AirTC)
plt.show()
No plot is complete without a title and axis labels, so let's add those. Notice how the degree symbol is generated in Python.
plt.title('2-m Air Temperature')
plt.xlabel('Record Number')
plt.ylabel('Temperature ($^\circ$C)')
plt.plot(data.RECORD,data.AirTC)
plt.show()
A more advanced technique, but one that is very important in this case, would be making the x-axis something more meaningful. This would require converting the timestamp (data.TIMESTAMP) provided in the data to something that matplotlib understands which is a datetime string. This requires the dateutil package, so we need to import it first.
import dateutil as du
To see how this conversion works, let's look at the data timestamp before and after we run the dateutil parse method.
data.TIMESTAMP.head(5)
0 2013-07-09 14:40:00 1 2013-07-09 14:50:00 2 2013-07-09 15:00:00 3 2013-07-09 15:10:00 4 2013-07-09 15:20:00 Name: TIMESTAMP, dtype: object
The command below used to convert the data timestamp contains an implied Python loop. It basically states that the parse function should be applied for all values in the data['TIMESTAMP'] list. As the list is traversed, each value gets assigned to x and x is then passed to the parse method. The result of this command is a list.
new_timestamp = [du.parser.parse(x) for x in data['TIMESTAMP']]
Let's now take a look at the list we created. It is now an instance of the datetime class. matplotlib understands how to handle datetime instances. Converting strings to datetime instances and back is one of the primary functionalities of the dateutil package.
new_timestamp[0:5]
[datetime.datetime(2013, 7, 9, 14, 40), datetime.datetime(2013, 7, 9, 14, 50), datetime.datetime(2013, 7, 9, 15, 0), datetime.datetime(2013, 7, 9, 15, 10), datetime.datetime(2013, 7, 9, 15, 20)]
MATLAB and pyplot have the concept of the current figure and the current axes. Because of this, matplotlib does as well. The first line below creates an instance of figure and axis objects for the current plot. We can then use attributes and methods associated with these objects. The autofmt_xdate method rotates the the tick labels and right aligns them.
fig, ax = plt.subplots()
ax.plot(new_timestamp,data.AirTC)
fig.autofmt_xdate()
plt.show()
The last thing I want to explore here is how to generate more than one plot. For example, let's say that we want to compare the average solar radiation to any precipitation that fell. It might be useful to put these plots on the same page. Here is one way it can be done. I should add here that the subplots_adjust method creates space between the subplots to prevent labels from overlapping or the plots from being too close together.
print data.columns
Index([u'TIMESTAMP', u'RECORD', u'AirTC', u'RH', u'WndDir', u'WndSpd', u'SlrW_Avg', u'Rain_Tot', u'Batt_volt_Min', u'WndSpd_Max', u'Press'], dtype='object')
fig,(ax0,ax1) = plt.subplots(nrows=2)
ax0.plot(new_timestamp,data.SlrW_Avg)
ax0.set_title('Hourly Average Solar Radiation')
ax0.set_ylabel('W/m^2')
ax1.plot(new_timestamp,data.Rain_Tot)
ax1.set_title('Hourly Precipitation')
ax1.set_ylabel('mm')
fig.autofmt_xdate()
plt.subplots_adjust(hspace=0.5)
plt.show()
fig,(ax0,ax1, ax2) = plt.subplots(nrows=3)
ax0.plot(new_timestamp,data.SlrW_Avg)
ax0.set_title('Hourly Average Solar Radiation')
ax0.set_ylabel('W/m^2')
ax1.plot(new_timestamp,data.Rain_Tot)
ax1.set_title('Hourly Precipitation')
ax1.set_ylabel('mm')
ax2.plot(new_timestamp,data.Press)
ax2.set_title('Station Pressure')
ax2.set_ylabel('mb')
fig.autofmt_xdate()
plt.subplots_adjust(hspace=0.5)
plt.show()