%matplotlib inline
import matplotlib.pyplot as plt
import pandas as pd
Data obtained from http://biostat.mc.vanderbilt.edu/DataSets
data = pd.read_csv('http://biostat.mc.vanderbilt.edu/wiki/pub/Main/DataSets/titanic3.csv')
pclassの型を数値から文字列に変換
data['pclass'] = data['pclass'].map(str)
survived: 1(生存),0(死亡)
pclass: 乗客の社会経済的地位(1:上流,2:中流,3:下流)
name: 氏名
sex: 性別
age: 年齢
sibsp: 同乗したSibling/Spouseの数
parch: 同乗したParent/Childrenの数
ticket: チケットナンバー
fare: 乗船料金
cabin: 船室番号
embarked: 乗船場(C = Cherbourg, Q = Queenstown; S = Southampton)
boat: Lifeboat
body: Body Identification Number
home.dest: Home/Destination
data.describe
<bound method DataFrame.describe of pclass survived name \ 0 1 1 Allen, Miss. Elisabeth Walton 1 1 1 Allison, Master. Hudson Trevor 2 1 0 Allison, Miss. Helen Loraine 3 1 0 Allison, Mr. Hudson Joshua Creighton 4 1 0 Allison, Mrs. Hudson J C (Bessie Waldo Daniels) 5 1 1 Anderson, Mr. Harry 6 1 1 Andrews, Miss. Kornelia Theodosia 7 1 0 Andrews, Mr. Thomas Jr 8 1 1 Appleton, Mrs. Edward Dale (Charlotte Lamson) 9 1 0 Artagaveytia, Mr. Ramon 10 1 0 Astor, Col. John Jacob 11 1 1 Astor, Mrs. John Jacob (Madeleine Talmadge Force) 12 1 1 Aubart, Mme. Leontine Pauline 13 1 1 Barber, Miss. Ellen "Nellie" 14 1 1 Barkworth, Mr. Algernon Henry Wilson 15 1 0 Baumann, Mr. John D 16 1 0 Baxter, Mr. Quigg Edmond 17 1 1 Baxter, Mrs. James (Helene DeLaudeniere Chaput) 18 1 1 Bazzani, Miss. Albina 19 1 0 Beattie, Mr. Thomson 20 1 1 Beckwith, Mr. Richard Leonard 21 1 1 Beckwith, Mrs. Richard Leonard (Sallie Monypeny) 22 1 1 Behr, Mr. Karl Howell 23 1 1 Bidois, Miss. Rosalie 24 1 1 Bird, Miss. Ellen 25 1 0 Birnbaum, Mr. Jakob 26 1 1 Bishop, Mr. Dickinson H 27 1 1 Bishop, Mrs. Dickinson H (Helen Walton) 28 1 1 Bissette, Miss. Amelia 29 1 1 Bjornstrom-Steffansson, Mr. Mauritz Hakan ... ... ... ... 1279 3 0 Vestrom, Miss. Hulda Amanda Adolfina 1280 3 0 Vovk, Mr. Janko 1281 3 0 Waelens, Mr. Achille 1282 3 0 Ware, Mr. Frederick 1283 3 0 Warren, Mr. Charles William 1284 3 0 Webber, Mr. James 1285 3 0 Wenzel, Mr. Linhart 1286 3 1 Whabee, Mrs. George Joseph (Shawneene Abi-Saab) 1287 3 0 Widegren, Mr. Carl/Charles Peter 1288 3 0 Wiklund, Mr. Jakob Alfred 1289 3 0 Wiklund, Mr. Karl Johan 1290 3 1 Wilkes, Mrs. James (Ellen Needs) 1291 3 0 Willer, Mr. Aaron ("Abi Weller") 1292 3 0 Willey, Mr. Edward 1293 3 0 Williams, Mr. Howard Hugh "Harry" 1294 3 0 Williams, Mr. Leslie 1295 3 0 Windelov, Mr. Einar 1296 3 0 Wirz, Mr. Albert 1297 3 0 Wiseman, Mr. Phillippe 1298 3 0 Wittevrongel, Mr. Camille 1299 3 0 Yasbeck, Mr. Antoni 1300 3 1 Yasbeck, Mrs. Antoni (Selini Alexander) 1301 3 0 Youseff, Mr. Gerious 1302 3 0 Yousif, Mr. Wazli 1303 3 0 Yousseff, Mr. Gerious 1304 3 0 Zabour, Miss. Hileni 1305 3 0 Zabour, Miss. Thamine 1306 3 0 Zakarian, Mr. Mapriededer 1307 3 0 Zakarian, Mr. Ortin 1308 3 0 Zimmerman, Mr. Leo sex age sibsp parch ticket fare cabin \ 0 female 29.00 0 0 24160 211.3375 B5 1 male 0.92 1 2 113781 151.5500 C22 C26 2 female 2.00 1 2 113781 151.5500 C22 C26 3 male 30.00 1 2 113781 151.5500 C22 C26 4 female 25.00 1 2 113781 151.5500 C22 C26 5 male 48.00 0 0 19952 26.5500 E12 6 female 63.00 1 0 13502 77.9583 D7 7 male 39.00 0 0 112050 0.0000 A36 8 female 53.00 2 0 11769 51.4792 C101 9 male 71.00 0 0 PC 17609 49.5042 NaN 10 male 47.00 1 0 PC 17757 227.5250 C62 C64 11 female 18.00 1 0 PC 17757 227.5250 C62 C64 12 female 24.00 0 0 PC 17477 69.3000 B35 13 female 26.00 0 0 19877 78.8500 NaN 14 male 80.00 0 0 27042 30.0000 A23 15 male NaN 0 0 PC 17318 25.9250 NaN 16 male 24.00 0 1 PC 17558 247.5208 B58 B60 17 female 50.00 0 1 PC 17558 247.5208 B58 B60 18 female 32.00 0 0 11813 76.2917 D15 19 male 36.00 0 0 13050 75.2417 C6 20 male 37.00 1 1 11751 52.5542 D35 21 female 47.00 1 1 11751 52.5542 D35 22 male 26.00 0 0 111369 30.0000 C148 23 female 42.00 0 0 PC 17757 227.5250 NaN 24 female 29.00 0 0 PC 17483 221.7792 C97 25 male 25.00 0 0 13905 26.0000 NaN 26 male 25.00 1 0 11967 91.0792 B49 27 female 19.00 1 0 11967 91.0792 B49 28 female 35.00 0 0 PC 17760 135.6333 C99 29 male 28.00 0 0 110564 26.5500 C52 ... ... ... ... ... ... ... ... 1279 female 14.00 0 0 350406 7.8542 NaN 1280 male 22.00 0 0 349252 7.8958 NaN 1281 male 22.00 0 0 345767 9.0000 NaN 1282 male NaN 0 0 359309 8.0500 NaN 1283 male NaN 0 0 C.A. 49867 7.5500 NaN 1284 male NaN 0 0 SOTON/OQ 3101316 8.0500 NaN 1285 male 32.50 0 0 345775 9.5000 NaN 1286 female 38.00 0 0 2688 7.2292 NaN 1287 male 51.00 0 0 347064 7.7500 NaN 1288 male 18.00 1 0 3101267 6.4958 NaN 1289 male 21.00 1 0 3101266 6.4958 NaN 1290 female 47.00 1 0 363272 7.0000 NaN 1291 male NaN 0 0 3410 8.7125 NaN 1292 male NaN 0 0 S.O./P.P. 751 7.5500 NaN 1293 male NaN 0 0 A/5 2466 8.0500 NaN 1294 male 28.50 0 0 54636 16.1000 NaN 1295 male 21.00 0 0 SOTON/OQ 3101317 7.2500 NaN 1296 male 27.00 0 0 315154 8.6625 NaN 1297 male NaN 0 0 A/4. 34244 7.2500 NaN 1298 male 36.00 0 0 345771 9.5000 NaN 1299 male 27.00 1 0 2659 14.4542 NaN 1300 female 15.00 1 0 2659 14.4542 NaN 1301 male 45.50 0 0 2628 7.2250 NaN 1302 male NaN 0 0 2647 7.2250 NaN 1303 male NaN 0 0 2627 14.4583 NaN 1304 female 14.50 1 0 2665 14.4542 NaN 1305 female NaN 1 0 2665 14.4542 NaN 1306 male 26.50 0 0 2656 7.2250 NaN 1307 male 27.00 0 0 2670 7.2250 NaN 1308 male 29.00 0 0 315082 7.8750 NaN embarked boat body home.dest 0 S 2 NaN St Louis, MO 1 S 11 NaN Montreal, PQ / Chesterville, ON 2 S NaN NaN Montreal, PQ / Chesterville, ON 3 S NaN 135 Montreal, PQ / Chesterville, ON 4 S NaN NaN Montreal, PQ / Chesterville, ON 5 S 3 NaN New York, NY 6 S 10 NaN Hudson, NY 7 S NaN NaN Belfast, NI 8 S D NaN Bayside, Queens, NY 9 C NaN 22 Montevideo, Uruguay 10 C NaN 124 New York, NY 11 C 4 NaN New York, NY 12 C 9 NaN Paris, France 13 S 6 NaN NaN 14 S B NaN Hessle, Yorks 15 S NaN NaN New York, NY 16 C NaN NaN Montreal, PQ 17 C 6 NaN Montreal, PQ 18 C 8 NaN NaN 19 C A NaN Winnipeg, MN 20 S 5 NaN New York, NY 21 S 5 NaN New York, NY 22 C 5 NaN New York, NY 23 C 4 NaN NaN 24 S 8 NaN NaN 25 C NaN 148 San Francisco, CA 26 C 7 NaN Dowagiac, MI 27 C 7 NaN Dowagiac, MI 28 S 8 NaN NaN 29 S D NaN Stockholm, Sweden / Washington, DC ... ... ... ... ... 1279 S NaN NaN NaN 1280 S NaN NaN NaN 1281 S NaN NaN Antwerp, Belgium / Stanton, OH 1282 S NaN NaN NaN 1283 S NaN NaN NaN 1284 S NaN NaN NaN 1285 S NaN 298 NaN 1286 C C NaN NaN 1287 S NaN NaN NaN 1288 S NaN 314 NaN 1289 S NaN NaN NaN 1290 S NaN NaN NaN 1291 S NaN NaN NaN 1292 S NaN NaN NaN 1293 S NaN NaN NaN 1294 S NaN 14 NaN 1295 S NaN NaN NaN 1296 S NaN 131 NaN 1297 S NaN NaN NaN 1298 S NaN NaN NaN 1299 C C NaN NaN 1300 C NaN NaN NaN 1301 C NaN 312 NaN 1302 C NaN NaN NaN 1303 C NaN NaN NaN 1304 C NaN 328 NaN 1305 C NaN NaN NaN 1306 C NaN 304 NaN 1307 C NaN NaN NaN 1308 S NaN NaN NaN [1309 rows x 14 columns]>
(len(data) - data.count()) / len(data)
pclass 0.000000 survived 0.000000 name 0.000000 sex 0.000000 age 0.200917 sibsp 0.000000 parch 0.000000 ticket 0.000000 fare 0.000764 cabin 0.774637 embarked 0.001528 boat 0.628724 body 0.907563 home.dest 0.430863 dtype: float64
欠損値の多い特徴量や,分析に有効でなさそうな特徴量を削除します.
※本来特徴量の選択は分析の試行錯誤のなかで行うべきですが,演習の都合上最初に行っています.
data2 = data.drop(['name', 'ticket', 'cabin', 'boat', 'body', 'home.dest'], 1)
カテゴリデータ(pclass, embarked, sex)は積み上げ棒グラフにして
変数の割合と,変数内の生死の割合を可視化
fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(15, 10))
pd.pivot_table(data2, index=['pclass'], columns=['survived'],
aggfunc='count')['sex'].plot(kind='barh', stacked=True,
ax=axes[0,0])
pd.pivot_table(data2, index=['embarked'], columns=['survived'],
aggfunc='count')['pclass'].plot(kind='barh', stacked=True,
ax=axes[0,1])
pd.pivot_table(data2, index=['sex'], columns=['survived'],
aggfunc='count')['pclass'].plot(kind='barh', stacked=True,
ax=axes[1,0])
plt.subplots_adjust(wspace=.5)
数値データ(age, sibsp, parch, fare)は生/死ごとにヒストグラムで可視化
fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(15, 10))
h = pd.concat([data2['age'][data2['survived']==0],
data2['age'][data2['survived']==1]], axis=1)
h.columns = ['survived: 0', 'survived: 1']
h.plot(kind='hist', bins=15, alpha=0.3, color=('r','b'),
ax=axes[0,0], title='age')
h = pd.concat([data2['sibsp'][data2['survived']==0],
data2['sibsp'][data2['survived']==1]], axis=1)
h.columns = ['survived: 0', 'survived: 1']
h.plot(kind='hist', bins=8, alpha=0.3, color=('r','b'),
ax=axes[0,1], title='sibsp')
h = pd.concat([data2['parch'][data2['survived']==0],
data2['parch'][data2['survived']==1]], axis=1)
h.columns = ['survived: 0', 'survived: 1']
h.plot(kind='hist', bins=9, alpha=0.3, color=('r','b'),
ax=axes[1,0], title='parch')
h = pd.concat([data2['fare'][data2['survived']==0],
data2['fare'][data2['survived']==1]], axis=1)
h.columns = ['survived: 0', 'survived: 1']
h.plot(kind='hist', bins=15, alpha=0.3, color=('r','b'),
ax=axes[1,1], title='fare')
plt.subplots_adjust(wspace=.5)
カテゴリデータ同士の関係はクロス集計(Cross tabulation)で可視化
※parch, sibspは数値データですが,取りうる値が限られているためクロス集計を利用
pd.crosstab(data2['embarked'], data2['pclass'])
pclass | 1 | 2 | 3 |
---|---|---|---|
embarked | |||
C | 141 | 28 | 101 |
Q | 3 | 7 | 113 |
S | 177 | 242 | 495 |
pd.crosstab(data2['embarked'], data2['sex'])
sex | female | male |
---|---|---|
embarked | ||
C | 113 | 157 |
Q | 60 | 63 |
S | 291 | 623 |
pd.crosstab(data2['embarked'], data2['sibsp'])
sibsp | 0 | 1 | 2 | 3 | 4 | 5 | 8 |
---|---|---|---|---|---|---|---|
embarked | |||||||
C | 171 | 90 | 9 | 0 | 0 | 0 | 0 |
Q | 100 | 14 | 4 | 0 | 5 | 0 | 0 |
S | 618 | 215 | 29 | 20 | 17 | 6 | 9 |
pd.crosstab(data2['embarked'], data2['parch'])
parch | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 9 |
---|---|---|---|---|---|---|---|---|
embarked | ||||||||
C | 196 | 51 | 20 | 3 | 0 | 0 | 0 | 0 |
Q | 114 | 7 | 1 | 0 | 0 | 1 | 0 | 0 |
S | 690 | 112 | 92 | 5 | 6 | 5 | 2 | 2 |
pd.crosstab(data2['pclass'], data2['sex'])
sex | female | male |
---|---|---|
pclass | ||
1 | 144 | 179 |
2 | 106 | 171 |
3 | 216 | 493 |
pd.crosstab(data2['pclass'], data2['sibsp'])
sibsp | 0 | 1 | 2 | 3 | 4 | 5 | 8 |
---|---|---|---|---|---|---|---|
pclass | |||||||
1 | 198 | 113 | 8 | 4 | 0 | 0 | 0 |
2 | 182 | 82 | 12 | 1 | 0 | 0 | 0 |
3 | 511 | 124 | 22 | 15 | 22 | 6 | 9 |
pd.crosstab(data2['pclass'], data2['parch'])
parch | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 9 |
---|---|---|---|---|---|---|---|---|
pclass | ||||||||
1 | 242 | 50 | 27 | 2 | 2 | 0 | 0 | 0 |
2 | 206 | 43 | 25 | 3 | 0 | 0 | 0 | 0 |
3 | 554 | 77 | 61 | 3 | 4 | 6 | 2 | 2 |
pd.crosstab(data2['sex'], data2['sibsp'])
sibsp | 0 | 1 | 2 | 3 | 4 | 5 | 8 |
---|---|---|---|---|---|---|---|
sex | |||||||
female | 262 | 160 | 19 | 12 | 7 | 2 | 4 |
male | 629 | 159 | 23 | 8 | 15 | 4 | 5 |
pd.crosstab(data2['sex'], data2['parch'])
parch | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 9 |
---|---|---|---|---|---|---|---|---|
sex | ||||||||
female | 293 | 88 | 69 | 6 | 4 | 4 | 1 | 1 |
male | 709 | 82 | 44 | 2 | 2 | 2 | 1 | 1 |
pd.crosstab(data2['sibsp'], data2['parch'])
parch | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 9 |
---|---|---|---|---|---|---|---|---|
sibsp | ||||||||
0 | 790 | 52 | 43 | 2 | 2 | 2 | 0 | 0 |
1 | 183 | 90 | 29 | 5 | 4 | 4 | 2 | 2 |
2 | 26 | 9 | 6 | 1 | 0 | 0 | 0 | 0 |
3 | 3 | 9 | 8 | 0 | 0 | 0 | 0 | 0 |
4 | 0 | 10 | 12 | 0 | 0 | 0 | 0 | 0 |
5 | 0 | 0 | 6 | 0 | 0 | 0 | 0 | 0 |
8 | 0 | 0 | 9 | 0 | 0 | 0 | 0 | 0 |
数値データとカテゴリデータの関係は箱ひげ図(boxplot)で可視化
fig, axes = plt.subplots(nrows=5, ncols=2, figsize=(15, 30))
data2.boxplot(column='age', by='embarked', ax=axes[0,0])
data2.boxplot(column='age', by='pclass', ax=axes[0,1])
data2.boxplot(column='age', by='sex', ax=axes[1,0])
data2.boxplot(column='age', by='sibsp', ax=axes[1,1])
data2.boxplot(column='age', by='parch', ax=axes[2,0])
data2.boxplot(column='fare', by='embarked', ax=axes[2,1])
data2.boxplot(column='fare', by='pclass', ax=axes[3,0])
data2.boxplot(column='fare', by='sex', ax=axes[3,1])
data2.boxplot(column='fare', by='sibsp', ax=axes[4,0])
data2.boxplot(column='fare', by='parch', ax=axes[4,1])
plt.suptitle('')
plt.subplots_adjust(wspace=.4, hspace=.2)
数値データ同士の関係は散布図(Scatter plot)で可視化
plt.plot(data2['age'], data2['fare'], 'b+')
plt.xlabel('age')
plt.ylabel('fare')
plt.suptitle('')
plt.subplots_adjust(wspace=.4, hspace=.2)
今回は,数値データは中央値で,カテゴリデータは最頻値で補間します.
ただし,運賃(fare)は社会経済的地位(pclass)と相関があるため※
等級ごとの中央値で補間します.
※pclassとfareの可視化結果(boxplot)参照
age_median = data2['age'].dropna().median()
embarked_mode = data2['embarked'].dropna().mode().values
fare_median_c1 = data2['fare'][data2['pclass']=='1'].dropna().median()
fare_median_c2 = data2['fare'][data2['pclass']=='2'].dropna().median()
fare_median_c3 = data2['fare'][data2['pclass']=='3'].dropna().median()
data2.loc[data2['age'].isnull(), 'age'] = age_median
data2.loc[data2['embarked'].isnull(), 'embarked'] = embarked_mode
data2.loc[(data2['fare'].isnull()) & (data2['pclass']=='1'), 'fare'] = fare_median_c1
data2.loc[(data2['fare'].isnull()) & (data2['pclass']=='2'), 'fare'] = fare_median_c2
data2.loc[(data2['fare'].isnull()) & (data2['pclass']=='3'), 'fare'] = fare_median_c3
(len(data2) - data2.count()) / len(data2)
pclass 0 survived 0 sex 0 age 0 sibsp 0 parch 0 fare 0 embarked 0 dtype: float64
DictVectorizerはN種類の変数をN個の数値特徴量に変換しますが
これは冗長なので,1つ削除します
from sklearn.feature_extraction import DictVectorizer as DV
vec = DV()
data2 = pd.DataFrame(vec.fit_transform(data2.T.to_dict().values()).toarray(),
columns=vec.get_feature_names(), index=data2.index)
del data2['embarked=S']
del data2['pclass=3']
del data2['sex=male']
scikit-learnの入力に合わせ,特徴量と目的変数を分けます
data2_y = data2.pop('survived')
数値データを平均0,分散1に標準化します.
from sklearn.preprocessing import StandardScaler
standardizer = StandardScaler().fit(data2.loc[:, ['age', 'sibsp', 'parch', 'fare']])
data2.loc[:, ['age', 'sibsp', 'parch', 'fare']] =\
standardizer.transform(data2.loc[:, ['age', 'sibsp', 'parch', 'fare']])
決定木(Decision tree)を使って生死を予測してみます.
予測結果は精度(accuracy)で評価します.
from sklearn.tree import DecisionTreeClassifier
clf0 = DecisionTreeClassifier(random_state=1).fit(data2, data2_y)
from sklearn.metrics import accuracy_score
accuracy_score(data2_y, clf0.predict(data2))
0.96562261268143623
from sklearn.grid_search import GridSearchCV
from sklearn.cross_validation import KFold
cv = KFold(data2.shape[0], n_folds=10, shuffle=True, random_state=1)
clf1 = GridSearchCV(DecisionTreeClassifier(random_state=1),
[{'max_depth': [3, 5, 7]}],
cv=cv, scoring='accuracy', n_jobs=-1).fit(data2, data2_y)
clf1.best_params_
{'max_depth': 5}
clf1.best_score_
0.799083269671505
from sklearn.learning_curve import learning_curve
import numpy as np
cv = KFold(data2.shape[0], n_folds=10, shuffle=True, random_state=1)
train_sizes, train_scores, test_scores = learning_curve(
clf1.best_estimator_, data2, data2_y, cv=cv, scoring='accuracy',
train_sizes=[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0], n_jobs=-1)
train_scores_mean = np.mean(train_scores, axis=1)
test_scores_mean = np.mean(test_scores, axis=1)
plt.figure()
plt.xlabel("Training examples")
plt.ylabel("Score")
plt.grid()
plt.plot(train_sizes, train_scores_mean, 'o-', color="r", label="Training score")
plt.plot(train_sizes, test_scores_mean, 'o-', color="g", label="Cross-validation score")
plt.ylim((0.4, 1.))
plt.legend(loc="lower right")
<matplotlib.legend.Legend at 0x10a0a4f90>
学習曲線をみると,
ことから,ハイバイアスな状態であると分かります.
そこで,性能改善のために,
を検討してみます.
モデルをより柔軟性の高い,アンサンブル学習(ブースティング)を利用したGradientBoostingClassifierに変更してみます.
from sklearn.ensemble import GradientBoostingClassifier
cv = KFold(data2.shape[0], n_folds=10, shuffle=True, random_state=1)
clf2 = GridSearchCV(GradientBoostingClassifier(n_estimators=100, random_state=1),
[{'max_leaf_nodes': [3, 5, 7, 9],
'min_samples_leaf': [9, 13, 17, 21],
'learning_rate': [0.3, 0.4, 0.5]}],
cv=cv, scoring='accuracy', n_jobs=-1).fit(data2, data2_y)
clf2.best_params_
{'learning_rate': 0.4, 'max_leaf_nodes': 5, 'min_samples_leaf': 17}
clf2.best_score_
0.82352941176470584
グリッドサーチ+交差検証 | モデル変更 |
---|---|
0.799 | 0.824 |
欠損率が高く利用を見送っていたcabinの情報を利用してみます.
cabinは文字+数値という形なので,文字と数値に分離して利用します.
data3 = data2.copy(deep=True)
import re
data3['cabin'] = data['cabin']
data3.loc[data3['cabin'].isnull(), 'cabin'] = 'unknown'
cabin_room = [re.sub('^[^\d]+(\d+).*$', '\\1', c) for c in data3['cabin']]
data3['cabin_room'] = [int(c) if c.isdigit() else 0 for c in cabin_room]
data3['cabin'] = [re.sub('^([A-Z]).*$', '\\1', c) for c in data3['cabin']]
vec = DV()
data3 = pd.DataFrame(vec.fit_transform(data3.T.to_dict().values()).toarray(),
columns=vec.get_feature_names(), index=data3.index)
del data3['cabin=unknown']
cv = KFold(data3.shape[0], n_folds=10, shuffle=True, random_state=1)
clf3 = GridSearchCV(GradientBoostingClassifier(n_estimators=100, random_state=1),
[{'max_leaf_nodes': [3, 5, 7, 9],
'min_samples_leaf': [5, 9, 13, 17],
'learning_rate': [0.1, 0.2, 0.3, 0.4]}],
cv=cv, scoring='accuracy', n_jobs=-1).fit(data3, data2_y)
clf3.best_params_
{'learning_rate': 0.3, 'max_leaf_nodes': 5, 'min_samples_leaf': 9}
clf3.best_score_
0.82887700534759357
グリッドサーチ+交差検証 | モデル変更 | cabin利用 |
---|---|---|
0.799 | 0.824 | 0.829 |
単純に中央値で補間していた年齢の欠損値を,その他の特徴量を使って予測してみます.
data4 = data3.copy(deep=True)
title = [re.split('\.|,', n)[1].strip() for n in data['name']]
data4['immature'] = [1 if t in ['Master', 'Mlle', 'Miss'] else 0 for t in title]
data4 = data4[['sibsp', 'parch', 'pclass=1', 'pclass=2', 'fare', 'immature']]
cv = KFold(data4[data['age'].notnull()].shape[0], n_folds=10, shuffle=True,
random_state=1)
from sklearn.tree import DecisionTreeRegressor
clf_age = GridSearchCV(DecisionTreeRegressor(random_state=1),
[{'max_depth': [2, 3, 4]}],
cv=cv,
n_jobs=-1).fit(data4[data['age'].notnull()],
data['age'][data['age'].notnull()])
clf_age.best_params_
{'max_depth': 3}
data3.loc[data['age'].isnull(), 'age'] = clf_age.predict(data4[data['age'].isnull()])
cv = KFold(data3.shape[0], n_folds=10, shuffle=True, random_state=1)
clf4 = GridSearchCV(GradientBoostingClassifier(n_estimators=100, random_state=1),
[{'max_leaf_nodes': [5, 7, 9, 11],
'min_samples_leaf': [13, 17, 21, 25],
'learning_rate': [0.1, 0.2, 0.3, 0.4]}],
cv=cv, scoring='accuracy', n_jobs=-1).fit(data3, data2_y)
clf4.best_params_
{'learning_rate': 0.2, 'max_leaf_nodes': 7, 'min_samples_leaf': 17}
clf4.best_score_
0.8304048892284186
グリッドサーチ+交差検証 | モデル変更 | cabin利用 | age予測 |
---|---|---|---|
0.799 | 0.824 | 0.829 | 0.830 |