Hive build and connect to mysql

Prerequisite: hadoop needs to be running

My hadoop is hadoop-2.7.1.tar.gz

First download hive  http://archive.apache.org/dist/hive/

Ache : apache-hive-2.1.1-bin.tar.gz

-----------------------------------------------------------------

Download completed

1. 解压 tar -zxvf apache-hive-2.1.1-bin.tar.gz

2.mv apache-hive-2.1.1-bin hive

3.vim /etc/profile

export HIVE_HOME=/usr/local/hiveexport PATH=$PATH:$HIVE_HOME/bin

Execute source /etc/profile

Execute hive --version

---------------------------------------------------------------------

1. cd /usr/local/hive/conf   cp hive-default.xml.template hive-site.xml2. Configure specific confidence in hive-siteNote: ip configuration involves schema initializationIt is best to test the MySQL library on the same machineOfficial: Remote initialization needs to pay attention to mysql permissions<!-- Insert the code-->    <property>        <name>javax.jdo.option.ConnectionUserName</name>        <value>root</value>    </property>    <property>        <name>javax.jdo.option.ConnectionPassword</name>        <value>123456</value>    </property>   <property>        <name>javax.jdo.option.ConnectionURL</name>        <value>jdbc:mysql://ip:3306/hive?createDatabaseIfNotExist=true</value>    </property>    <property>        <name>javax.jdo.option.ConnectionDriverName</name>        <value>com.mysql.jdbc.Driver</value>    </property>  <property>    <name>Hive.exec.local.scratchdir</name>    <value>/usr/local/hadoop/tmp/${user.name}</value>    <description>Local scratch space for Hive jobs</description>  </property>   <property>    <name>hive.downloaded.resources.dir</name>    <value>/usr/local/hadoop/tmp/${hive.session.id}_resources</value>    <description>Temporary local directory for added resources in the remote file system.</description>  </property> <property>    <name>hive.server2.logging.operation.log.location</name>    <value>/usr/local/hadoop/tmp/operation_logs</value>    <description>Top level directory where operation logs are stored if logging functionality is enabled</description>  </property>   cp hive-default.xml.template hive-site.xml

3 Copy the mysql driver to hive/lib

4. Hive schema in mysql (you need to create hive database under mysql before this)

[[email protected] conf]# schematool -initSchema -dbType mysql

It is easy to make mistakes here, and specific problems are analyzed in detail

mysql mapping

Successfully execute hive and enter hive

-------------------------------------------------

hive test:

hadoop HDFS storage

bin/hadoop fs -lsr /

[[email protected] hadoop]# bin/hadoop fs -lsr /
lsr: DEPRECATED: Please use 'ls -R' instead.
drwxr-xr-x   - root supergroup          0 2021-05-07 12:08 /input
drwxr-xr-x   - root supergroup          0 2021-05-07 12:08 /input/out
-rw-r--r--   1 root supergroup          0 2021-05-07 12:08 /input/out/_SUCCESS
-rw-r--r--   1 root supergroup        131 2021-05-07 12:08 /input/out/part-r-00000
-rw-r--r--   1 root supergroup        179 2021-05-07 11:29 /input/word.txt
drwxr-xr-x   - root supergroup          0 2021-05-07 11:24 /test
-rw-r--r--   1 root supergroup        179 2021-05-07 11:24 /test/word.txt
drwxr-xr-x   - root supergroup          0 2021-06-02 17:17 /test2
drwx-wx-wx   - root supergroup          0 2021-06-02 21:24 /tmp
drwx-wx-wx   - root supergroup          0 2021-06-02 21:24 /tmp/hive
drwx------   - root supergroup          0 2021-06-02 21:30 /tmp/hive/root
drwx------   - root supergroup          0 2021-06-02 21:30 /tmp/hive/root/658a2a46-ea99-4967-a6a8-8954821f33c8
drwx------   - root supergroup          0 2021-06-02 21:30 /tmp/hive/root/658a2a46-ea99-4967-a6a8-8954821f33c8/_tmp_space.db
drwxr-xr-x   - root supergroup          0 2021-06-02 21:30 /user
drwxr-xr-x   - root supergroup          0 2021-06-02 21:30 /user/hive
drwxr-xr-x   - root supergroup          0 2021-06-02 21:30 /user/hive/warehouse
drwxr-xr-x   - root supergroup          0 2021-06-02 21:30 /user/hive/warehouse/hive_1.db


1. Enter mysql view