使用网格搜索优化钻石价格模型
主要步骤流程：
- 1. 导入包
- 2. 导入数据集
- 3. 数据预处理
- 3.1 处理缺失数据
- 3.2 处理类别型字段
- 3.2.1 统计类别型字段
- 3.2.2 字符编码和独热编码
- 3.3 生成自变量和因变量
- 3.4 拆分数据集
- 3.5 特征缩放
- 4. 构建最优模型
- 5. 应用网格搜索优化模型
- 6. 比较模型优化前和优化后的性能
数据集链接：
1. 导入包
In [1]:
# 导入包
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
2. 导入数据集
In [2]:
# 导入数据集
dataset = pd.read_csv('diamonds.csv')
dataset
Out[2]:
| | carat | cut | color | clarity | depth | table | x | y | z | price |
|---|
| 0 |
0.23 |
Ideal |
E |
SI2 |
61.5 |
55.0 |
3.95 |
3.98 |
2.43 |
326 |
|---|
| 1 |
0.21 |
Premium |
E |
SI1 |
59.8 |
61.0 |
3.89 |
3.84 |
2.31 |
326 |
|---|
| 2 |
0.23 |
Good |
E |
VS1 |
56.9 |
65.0 |
4.05 |
4.07 |
2.31 |
327 |
|---|
| 3 |
0.29 |
Premium |
I |
VS2 |
62.4 |
58.0 |
4.20 |
4.23 |
2.63 |
334 |
|---|
| 4 |
0.31 |
Good |
J |
SI2 |
63.3 |
58.0 |
4.34 |
4.35 |
2.75 |
335 |
|---|
| ... |
... |
... |
... |
... |
... |
... |
... |
... |
... |
... |
|---|
| 53935 |
0.72 |
Ideal |
D |
SI1 |
60.8 |
57.0 |
5.75 |
5.76 |
3.50 |
2757 |
|---|
| 53936 |
0.72 |
Good |
D |
SI1 |
63.1 |
55.0 |
5.69 |
5.75 |
3.61 |
2757 |
|---|
| 53937 |
0.70 |
Very Good |
D |
SI1 |
62.8 |
60.0 |
5.66 |
5.68 |
3.56 |
2757 |
|---|
| 53938 |
0.86 |
Premium |
H |
SI2 |
61.0 |
58.0 |
6.15 |
6.12 |
3.74 |
2757 |
|---|
| 53939 |
0.75 |
Ideal |
D |
SI2 |
62.2 |
55.0 |
5.83 |
5.87 |
3.64 |
2757 |
|---|
53940 rows × 10 columns
3. 数据预处理
3.1 处理缺失数据
In [3]:
# 统计缺失数据
df_null = dataset.isnull().sum()
df_null
Out[3]:
carat 0
cut 0
color 0
clarity 0
depth 0
table 0
x 0
y 0
z 0
price 0
dtype: int64
没有任何一个字段有缺失值
In [4]:
# 找出业务不合理的数据
xyz_zero_df = dataset.query('x < 0.01 or y < 0.01 or z < 0.01')
xyz_zero_df
Out[4]:
| | carat | cut | color | clarity | depth | table | x | y | z | price |
|---|
| 2207 |
1.00 |
Premium |
G |
SI2 |
59.1 |
59.0 |
6.55 |
6.48 |
0.0 |
3142 |
|---|
| 2314 |
1.01 |
Premium |
H |
I1 |
58.1 |
59.0 |
6.66 |
6.60 |
0.0 |
3167 |
|---|
| 4791 |
1.10 |
Premium |
G |
SI2 |
63.0 |
59.0 |
6.50 |
6.47 |
0.0 |
3696 |
|---|
| 5471 |
1.01 |
Premium |
F |
SI2 |
59.2 |
58.0 |
6.50 |
6.47 |
0.0 |
3837 |
|---|
| 10167 |
1.50 |
Good |
G |
I1 |
64.0 |
61.0 |
7.15 |
7.04 |
0.0 |
4731 |
|---|
| 11182 |
1.07 |
Ideal |
F |
SI2 |
61.6 |
56.0 |
0.00 |
6.62 |
0.0 |
4954 |
|---|
| 11963 |
1.00 |
Very Good |
H |
VS2 |
63.3 |
53.0 |
0.00 |
0.00 |
0.0 |
5139 |
|---|
| 13601 |
1.15 |
Ideal |
G |
VS2 |
59.2 |
56.0 |
6.88 |
6.83 |
0.0 |
5564 |
|---|
| 15951 |
1.14 |
Fair |
G |
VS1 |
57.5 |
67.0 |
0.00 |
0.00 |
0.0 |
6381 |
|---|
| 24394 |
2.18 |
Premium |
H |
SI2 |
59.4 |
61.0 |
8.49 |
8.45 |
0.0 |
12631 |
|---|
| 24520 |
1.56 |
Ideal |
G |
VS2 |
62.2 |
54.0 |
0.00 |
0.00 |
0.0 |
12800 |
|---|
| 26123 |
2.25 |
Premium |
I |
SI1 |
61.3 |
58.0 |
8.52 |
8.42 |
0.0 |
15397 |
|---|
| 26243 |
1.20 |
Premium |
D |
VVS1 |
62.1 |
59.0 |
0.00 |
0.00 |
0.0 |
15686 |
|---|
| 27112 |
2.20 |
Premium |
H |
SI1 |
61.2 |
59.0 |
8.42 |
8.37 |
0.0 |
17265 |
|---|
| 27429 |
2.25 |
Premium |
H |
SI2 |
62.8 |
59.0 |
0.00 |
0.00 |
0.0 |
18034 |
|---|
| 27503 |
2.02 |
Premium |
H |
VS2 |
62.7 |
53.0 |
8.02 |
7.95 |
0.0 |
18207 |
|---|
| 27739 |
2.80 |
Good |
G |
SI2 |
63.8 |
58.0 |
8.90 |
8.85 |
0.0 |
18788 |
|---|
| 49556 |
0.71 |
Good |
F |
SI2 |
64.1 |
60.0 |
0.00 |
0.00 |
0.0 |
2130 |
|---|
| 49557 |
0.71 |
Good |
F |
SI2 |
64.1 |
60.0 |
0.00 |
0.00 |
0.0 |
2130 |
|---|
| 51506 |
1.12 |
Premium |
G |
I1 |
60.4 |
59.0 |
6.71 |
6.67 |
0.0 |
2383 |
|---|
x < 0.01 或者 y < 0.01 或者 z < 0.01，这些数据不符合业务场景，且数据量比较少，故直接删除。
In [5]:
dataset = dataset.drop(xyz_zero_df.index)
In [6]:
xyz_zero_df = dataset.query('x < 0.01 or y < 0.01 or z < 0.01')
xyz_zero_df
Out[6]:
| | carat | cut | color | clarity | depth | table | x | y | z | price |
|---|
3.2 处理类别型字段
3.2.1 统计类别型字段
In [7]:
# take care of categorical data
cols = dataset.columns
num_cols = dataset._get_numeric_data().columns
cat_cols = set(cols) - set(num_cols)
print('类别型字段是：' + str(cat_cols))
类别型字段是：{'color', 'cut', 'clarity'}
由数据集字段说明可知，cut、clarity、color这3个字段都是类别型字段。对他们做字符编码和独热编码。
3.2.2 字符编码和独热编码
In [8]:
dataset = pd.get_dummies(dataset, drop_first=True)
dataset.shape
Out[8]:
(53920, 24)
数据集由10列变为24列。
3.3 生成自变量和因变量
In [9]:
# 生成自变量和因变量
y = dataset['price'].values
dataset = dataset.drop(['price'], axis = 1)
X = dataset.values
In [10]:
X[:5,:]
Out[10]:
array([[ 0.23, 61.5 , 55. , 3.95, 3.98, 2.43, 0. , 1. , 0. ,
0. , 1. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
1. , 0. , 0. , 0. , 0. ],
[ 0.21, 59.8 , 61. , 3.89, 3.84, 2.31, 0. , 0. , 1. ,
0. , 1. , 0. , 0. , 0. , 0. , 0. , 0. , 1. ,
0. , 0. , 0. , 0. , 0. ],
[ 0.23, 56.9 , 65. , 4.05, 4.07, 2.31, 1. , 0. , 0. ,
0. , 1. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 1. , 0. , 0. , 0. ],
[ 0.29, 62.4 , 58. , 4.2 , 4.23, 2.63, 0. , 0. , 1. ,
0. , 0. , 0. , 0. , 0. , 1. , 0. , 0. , 0. ,
0. , 0. , 1. , 0. , 0. ],
[ 0.31, 63.3 , 58. , 4.34, 4.35, 2.75, 1. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. , 0. , 1. , 0. , 0. ,
1. , 0. , 0. , 0. , 0. ]])
In [11]:
y
Out[11]:
array([ 326, 326, 327, ..., 2757, 2757, 2757], dtype=int64)
3.4 拆分数据集
In [12]:
# 拆分数据集
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
3.5 特征缩放
In [13]:
# 特征缩放
from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X_train = sc_X.fit_transform(X_train)
X_test = sc_X.transform(X_test)
In [14]:
sc_y = StandardScaler()
y_train = np.ravel(sc_y.fit_transform(y_train.reshape(-1, 1)))
y_test = np.ravel(sc_y.transform(y_test.reshape(-1, 1)))
4. 构建最优模型
通过k折交叉验证法，从3个模型（随机森林、多元线性回归、支持向量机）中选择出了支持向量机模型。
In [15]:
# 构建支持向量机模型
from sklearn.svm import SVR
svr_regressor = SVR(kernel = 'rbf', C=1.0, verbose = 1)
5. 应用网格搜索优化模型
为了使案例简单化，gamma、epsilon等超参数暂不考虑。只考虑 C 和 kernel。
In [16]:
# 应用网格搜索算法找到最好的超参数
from sklearn.model_selection import GridSearchCV
?
parameters = [{'C': [0.5, 1, 1.5, 2], 'kernel': ['linear', 'rbf']}]
?
grid_search = GridSearchCV(estimator = svr_regressor,
param_grid = parameters,
scoring = 'r2',
cv = 5,
verbose=1,
n_jobs=6)
在4核8GB机器上，大约训练 40~50 分钟。
In [17]:
# 开始执行
grid_search.fit(X_train, y_train)
Fitting 5 folds for each of 8 candidates, totalling 40 fits
[Parallel(n_jobs=6)]: Using backend LokyBackend with 6 concurrent workers.
[Parallel(n_jobs=6)]: Done 40 out of 40 | elapsed: 53.2min finished
[LibSVM]
Out[17]:
GridSearchCV(cv=5, estimator=SVR(verbose=1), n_jobs=6,
param_grid=[{'C': [0.5, 1, 1.5, 2], 'kernel': ['linear', 'rbf']}],
scoring='r2', verbose=1)
In [18]:
best_score = grid_search.best_score_
best_score
Out[18]:
0.9766113677511867
In [19]:
best_parameters = grid_search.best_params_
best_parameters
Out[19]:
{'C': 2, 'kernel': 'rbf'}
最好的参数组合是：{'C': 2, 'kernel': 'rbf'}
6. 比较模型优化前和优化后的性能
6.1 优化前
In [20]:
# 优化前
from sklearn.svm import SVR
before_svr_regressor = SVR(kernel = 'rbf', C=1.0, verbose = 1)
before_svr_regressor.fit(X_train, y_train)
before_y_pred = before_svr_regressor.predict(X_test)
[LibSVM]
In [21]:
from sklearn.metrics import r2_score
before_r2_score = r2_score(y_test, before_y_pred)
In [22]:
before_r2_score
Out[22]:
0.9776441352140839
6.2 优化后
In [23]:
# 优化后
after_svr_regressor = SVR(kernel = 'rbf', C=2.0, verbose = 1)
after_svr_regressor.fit(X_train, y_train)
after_y_pred = after_svr_regressor.predict(X_test)
[LibSVM]
In [24]:
after_r2_score = r2_score(y_test, after_y_pred)
In [25]:
after_r2_score
Out[25]:
0.9788539228622857
结论：
- 优化后性能有所提升。
?