Hadoop集群搭建之 Sqoop数据迁移

2. 连接到hadoop01的mysql数据库

打开Navicat

连接到hadoop01成功

3.创建hive维度表关联的mysql表

执行以下语句：

CREATE DATABASE JobData CHARACTER SET utf8 COLLATE utf8_general_ci;

use JobData;

CREATE TABLE t_city_count(

city varchar(30) DEFAULT NULL,

count int(5) DEFAULT NULL

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE t_salary_dist(

salary varchar(30) DEFAULT NULL,

count int(5) DEFAULT NULL

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE t_company_count(

company varchar(30) DEFAULT NULL,

count int(5) DEFAULT NULL

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE t_kill_count(

kills varchar(30) DEFAULT NULL,

count int(5) DEFAULT NULL

) ENGINE=InnoDB DEFAULT CHARSET=utf8;

4. 查询表是否成功创建

创建成功

5. 启动集群

start-dfs.sh

start-yarn.sh

6.完成数据迁移

sqoop export \

--connect jdbc:mysql://localhost:3306/JobData?characterEncoding=UTF-8 \

--username root \

--password 123456 \

--table t_city_count \

--columns "city,count" \

--fields-terminated-by ',' \

--export-dir /user/hive/warehouse/jobdata.db/t_ods_city

sqoop export \

--connect jdbc:mysql://localhost:3306/JobData?characterEncoding=UTF-8 \

--username root \

--password 123456 \

--table t_salary_dist \

--columns "salary,count" \

--fields-terminated-by ',' \

--export-dir /user/hive/warehouse/jobdata.db/t_ods_salary

sqoop export \

--connect jdbc:mysql://localhost:3306/JobData?characterEncoding=UTF-8 \

--username root \

--password 123456 \

--table t_company_count \

--columns "company,count" \

--fields-terminated-by ',' \

--export-dir /user/hive/warehouse/jobdata.db/t_ods_company

sqoop export \

--connect jdbc:mysql://localhost:3306/JobData?characterEncoding=UTF-8 \

--username root \

--password 123456 \

--table t_kill_count \

--columns "kills,count" \

--fields-terminated-by ',' \

--export-dir /user/hive/warehouse/jobdata.db/t_ods_kill

7.查看mysql表是否有数据

成功，sqoop数据迁移成功