Centos7 install Hive2.0.1 cluster

1. Preparation
1.1, install jdk1.8 and mysql5.7.21, omit
1.2, install Hadoop 2.6.0, omit
1.3, machine introduction

192.168.1.89 node1192.168.1.149 node2192.168.1.180 node3

Hadoop 2.6.0 has been installed on node1, node2, and node3
2. Download and upload (all three nodes are operated)
http://archive.apache.org/dist/hive/hive-2.0.1/apache-hive-2.0 .1-bin.tar.gz
uploaded to the /data/server directory of the three machines, and decompressed:
tar -zxvf apache-hive-2.0.1-bin.tar.gz
3. Modify the configuration file (all three nodes operating)

cd apache-hive-2.0.1-bin/confcp hive-default.xml.template hive-site.xmlvi hive-site.xml,新增:<configuration><property>    <name>datanucleus.schema.autoCreateAll</name>    <value>true</value></property><property>    <name>javax.jdo.option.ConnectionURL</name>    <value>jdbc:mysql://192.168.1.166:3306/hive_db?createDatabaseIfNotExist=true</value></property><property>    <name>javax.jdo.option.ConnectionDriverName</name>    <value>com.mysql.jdbc.Driver</value></property><property>    <name>javax.jdo.option.ConnectionUserName</name>    <value>root</value></property><property>    <name>javax.jdo.option.ConnectionPassword</name>    <value>123456</value></property><property>    <name>hive.support.concurrency</name>    <value>true</value></property><property>    <name>hive.enforce.bucketing</name>    <value>true</value></property>    <property>    <name>hive.exec.dynamic.partition.mode</name>    <value>nonstrict</value></property><property>    <name>hive.txn.manager</name>    <value>org.apache.hadoop.hive.ql.lockmgr.DbTxnManager</value></property><property>    <name>hive.compactor.initiator.on</name>    <value>true</value></property><property>    <name>hive.compactor.worker.threads</name>    <value>1</value></property><property>    <name>hive.in.test</name>    <value>true</value></property><property>    <name>hive.server2.transport.mode</name>    <value>binary</value>    <description>      Expects one of [binary, http].      Transport mode of HiveServer2.    </description></property><property>    <name>hive.server2.thrift.bind.host</name>    <value>机器IP</value>    <description>       Bind host on which to run the HiveServer2 Thrift interface.Can       be overridden by setting $HIVE_SERVER2_THRIFT_BIND_HOST    </description></property><property>    <name>hive.server2.thrift.port</name>    <value>10000</value>    <description>Port number of HiveServer2 Thrift interface when hive.server2.transport.mode is 'binary'.</description> </property></configuration>

4. Modify environment variables (all three nodes operate)

vim /etc/profileexport HIVE_HOME=/data/server/apache-hive-2.0.1-binexport HIVE_CONF_DIR=$HIVE_HOME/confexport PATH=$HIVE_HOME/bin:$PATH激活配置:source /etc/profile

5. Upload mysql driver (all three nodes operate)

cd /data/server/apache-hive-2.0.1-bin/lib/上传mysql-connector-java-5.1.30.jar

6. Whether the test is successful (all three nodes are operated)
(1) hive //Log in to hive
(2) show databases; //Display the hive table name
7. Start the service in the background (all three nodes are operated)
//Nohup needs to be started hive --service metastore &
nohup hive --service hiveserver2 &
8. Simple use 1 (operation on node1, no row data can be modified or deleted)
Use beeline command to open the client, use !connect jdbc:hive2://node1:10000/ default connect to the hive default library and press Enter
8.1, create library

create database test_db;

8.2, create a table

use test_db;CREATE TABLE t_test1 (a int, b int, c int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';

Keyword definition:
ROW FORMAT DELIMITED is
FIELDS indicating that the following keywords are column and element separators. TERMINATED BY is the field separator.
8.3. Import data

vi /data/server/apache-hive-2.0.1-bin/t_test1.txt,字段之间以tab隔开:

16 2 3
61 12 13
41 2 31
17 21 3
71 2 31
1 12 34
11 2 34
Import script:

LOAD DATA LOCAL INPATH '/data/server/apache-hive-2.0.1-bin/t_test1.txt' OVERWRITE INTO TABLE t_test1;

8.4. New data

insert into t_test1(a,b,c) values(12,13,14);

8.5. View data (you can check whether the data is synchronized to other nodes to verify the cluster)

select * from t_test1;

8.6, view the table structure

desc t_test1;

8.7, delete table

drop table t_test1;

9. Simple use 2 (operation on node1, you can modify or delete the row data)
use the beeline command to open the client, use !connect jdbc:hive2://node1:10000/default to connect to the hive default library and press Enter
9.1, create a table

use test_db;create table t_test2(id int ,name string ) clustered by (id) into 2 buckets stored as orc TBLPROPERTIES('transactional'='true');

Keyword definition:
clustered by (id) into 2 buckets The table is divided into 2 buckets according to id
stored as orc The format of the table storage file is orc
TBLPROPERTIES('transactional'='true') Enable transaction support of the table
9.2, import data
File import
9.3, new data is not supported

insert into t_test2(id,name) values(1,'你好');

9.4, modify data

update t_test2 set name = '你不好' where id=1;

9.5. View the data (you can check whether the data is synchronized to other nodes to verify the cluster)

select * from t_test2;

9.6, delete data

delete from t_test2 where id=1;

9.7, view table structure

desc t_test1;

9.8, delete table

drop table t_test1;

9.9. Note
If a lock error message appears when creating an orc table, you can add Hive metadata (stored using mysql) INSERT INTO NEXT_LOCK_ID VALUES(1);
10. Java calls hive instance
10.1, import maven dependencies

<dependency>    <groupId>org.apache.hive</groupId>    <artifactId>hive-jdbc</artifactId>    <version>2.1.0</version></dependency>

10.2 Code implementation

package com.x.y.test;import java.sql.Connection;import java.sql.DriverManager;import java.sql.ResultSet;import java.sql.Statement;import org.junit.After;import org.junit.Before;import org.junit.Test; public class HiveJDBC {        private static String driverName="org.apache.hive.jdbc.HiveDriver";    private static String url = "jdbc:hive2://192.168.1.89:10000/test_db";    private static Connection conn = null;    private static Statement stmt = null;    private static ResultSet rs = null;     public static void main(String[] args) throws Exception {        Class.forName(driverName);        conn = DriverManager.getConnection(url, null, null);        stmt = conn.createStatement();         String sql = "select * from t_xxl";        System.out.println("Running: " + sql);        rs = stmt.executeQuery(sql);        System.out.println("a" + "\t" + "b" +"\t" + "c" );        while (rs.next()) {            System.out.println(rs.getInt(1) + "\t" + rs.getString(2)+ "\t"+ rs.getString(3));        }         if (rs != null) {            rs.close();        }        if (stmt != null) {            stmt.close();        }        if (conn != null) {            conn.close();        }     } }