Model-based learning 简单实践

从样本集进行归纳的方法是建立这些样本的模型，然后使用这个模型进行预测，这叫作基于模型学习（Model-based learning）。

例如，你想知道钱是否能让人快乐？下面是一个简单的基于线性模型的案例。

数据来源：https://github.com/ageron/handson-ml

# Python ≥3.5
import sys
assert sys.version_info >= (3, 5)

# Scikit-Learn ≥0.20
import sklearn
assert sklearn.__version__ >= "0.20"

加载数据

# 数据所在路径设置
import os
datapath = os.path.join("datasets", "lifesat", "")
print(datapath)

datasets/lifesat/

从 OECD 网站下载了 Better Life Index 指数数据，如下：

import numpy as np
import pandas as pd

oecd_bli = pd.read_csv(datapath + "oecd_bli_2015.csv", thousands=',') # thousands 设置千位分隔符；
oecd_bli.head()

从 IMF 下载了人均 GDP 数据，如下：

gdp_per_capita = pd.read_csv(datapath + "gdp_per_capita.csv", thousands=',',  # per capita 人均
                             delimiter='\t', encoding='latin1', na_values="n/a")
gdp_per_capita.head()

准备数据

This function just merges the OECD's life satisfaction data and the IMF's GDP per capita data. It's a bit too long and boring and it's not specific to Machine Learning, which is why I left it out of the book.

def prepare_country_stats(oecd_bli, gdp_per_capita):
    oecd_bli = oecd_bli[oecd_bli["INEQUALITY"]=="TOT"]
    oecd_bli = oecd_bli.pivot(index="Country", columns="Indicator", values="Value")
    gdp_per_capita.rename(columns={"2015": "GDP per capita"}, inplace=True)
    gdp_per_capita.set_index("Country", inplace=True)
    full_country_stats = pd.merge(left=oecd_bli, right=gdp_per_capita,
                                  left_index=True, right_index=True)
    full_country_stats.sort_values(by="GDP per capita", inplace=True)
    remove_indices = [0, 1, 6, 8, 33, 34, 35]
    keep_indices = list(set(range(36)) - set(remove_indices))
    return full_country_stats[["GDP per capita", 'Life satisfaction']].iloc[keep_indices]

country_stats = prepare_country_stats(oecd_bli, gdp_per_capita)
country_stats.head()

可视化数据

import matplotlib.pyplot as plt
country_stats.plot(kind='scatter', x="GDP per capita", y='Life satisfaction')
plt.show()

线性回归

import sklearn.linear_model
model = sklearn.linear_model.LinearRegression()

训练模型

X = np.c_[country_stats["GDP per capita"]]
y = np.c_[country_stats["Life satisfaction"]]
model.fit(X, y)

LinearRegression()

根据模型进行预测

X_new = [[22587]]  # Cyprus' GDP per capita
print(model.predict(X_new)) # outputs [[ 5.96242338]]

[[5.96242338]]

总结

read_csv参数

• thousands=',' : 千位分隔符；可以将"1,000"转换为 int 型的1000；
• delimiter='\t' : sep的替代参数，csv文件分隔符可能为"," or "\t"，可用sublime查看；
• encoding='latin1' : 确定正确的编码方式才能正确解码；vim this file and set fileencoding即可显示编码格式；
• na_values="n/a" : 缺少值处理，可参考 https://blog.csdn.net/weixin_44520259/article/details/106053987 ；

学习重点是机器学习原理，对于numpy，pandas之类的不熟悉的遇到了就学一下，不需要系统的学习，抓住重点！