spark整合hive

1.安装MySQL（hive的元数据库）并创建一个普通用户，并且授权
CREATE USER 'xiaoniu'@'%' IDENTIFIED BY '123568';
GRANT ALL PRIVILEGES ON hivedb.* TO 'xiaoniu'@'%' IDENTIFIED BY '123568' WITH GRANT OPTION;
FLUSH PRIVILEGES;

#在spark的conf目录下创建一个hive的配置文件
2.添加一个hive-site.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
<property>
<name>javax.jdo.option.ConnectionURLname>
<value>jdbc:mysql://node-6:3306/hivedb?createDatabaseIfNotExist=truevalue>
<description>JDBC connect string for a JDBC metastoredescription>
property>

<property>
<name>javax.jdo.option.ConnectionDriverNamename>
<value>com.mysql.jdbc.Drivervalue>
<description>Driver class name for a JDBC metastoredescription>
property>

<property>
<name>javax.jdo.option.ConnectionUserNamename>
<value>bigdatavalue>
<description>username to use against metastore databasedescription>
property>

<property>
<name>javax.jdo.option.ConnectionPasswordname>
<value>123568value>
<description>password to use against metastore databasedescription>
property>

configuration>

3.上传一个mysql连接驱动（sparkSubmit也要连接MySQL，获取元数据信息）
./spark-sql --master spark://node-4:7077,node-5:7077 --driver-class-path /home/xiaoniu/mysql-connector-java-5.1.7-bin.jar

4.sparkSQL会在mysql上创建一个database，需要手动改一下DBS表中的DB_LOCATION_UIR改成hdfs的地址

5.要在/etc/profile中配置一个环节变量(让sparkSQL知道hdfs在哪里，其实就是namenode在哪里)
exprot HADOOP_CONF_DIR

6.重新启动SparkSQL的命令行