What Is Sqoop ? How It Is Used And Its Installation?

Reading Time: 2 minutes

The traditional application management system, that is, the interaction of applications with relational database using RDBMS, is one of the sources that generate Big Data.

Such Big Data, generated by RDBMS, is stored in Relational Database Servers in the relational database structure.

Sqoop − “SQL to Hadoop and Hadoop to SQL”

Sqoop is a tool designed to transfer data between Hadoop and relational database servers.

It is used to import data from relational databases such as MySQL, Oracle to Hadoop HDFS, and export from Hadoop file system to relational databases. It is provided by the Apache Software Foundation.

How Sqoop Works?

The following image describes the workflow of Sqoop.

Sqoop Tutorial for Beginners - Sqoop Introduction and Features - TechVidvan

Sqoop installation on Linux

Before installing Sqoop, make sure Hadoop is installed.

To install Sqoop follow the below steps:

Open the terminal and download the sqoop archive:

$ wget http://mirrors.estointernet.in/apache/sqoop/1.4.7/sqoop-1.4.7.tar.gz

This will download the Sqoop archive, Now extract the Sqoop files:

$ tar -xvf sqoop-1.4.7.tar.gz

Now you have to set Sqoop Environment Variables in .bashrc file.

$ export SQOOP_HOME=/home/edureka/sqoop-1.4.7 export PATH=$PATH:$SQOOP_HOME/bin

Now you have to configure Sqoop.

Follow the below steps to do this:

$ cd $SQOOP_HOME/conf

$ mv sqoop-env-template.sh sqoop-env.sh

Now open the sqoop-env.sh file with a text editor

$ nano sqoop-env.sh

and add the path to Hadoop:

The path might change depending on where you have stored Hadoop. Use the path applicable to you.

Save (Ctrl+o) and exit (Ctrl+x)

Next step is to configure the mysql connector.

Run the below commands to do it:

$ wget http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-J/mysql-connector-java-8.0.15.tar.gz

$ tar -xvf mysql-connector-java-8.0.15.tar.gz
$ mv mysql-connector-java-8.0.15/mysql-connector-java-8.0.15.jar /$SQOOP_HOME/lib
Finally, to verify the installation, run:

$ sqoop -version

This will output version about sqoop

25/2/19 18:52:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
Sqoop 1.4.5 git commit id 5b34accaca7de251fc91161733f906af2eddbe83

Syntax

The following syntax is used to import data into HDFS.

$ sqoop import (generic-args) (import-args) 

$ sqoop import (generic-args) (import-args) 

Example

Let us take an example of three tables named as empemp_add, and emp_contact, which are in a database called userdb in a MySQL database server.

For More Information:https://www.tutorialspoint.com/sqoop/sqoop_introduction.html

Written by 

I am an enthusiastic , hard-working and determine girl with strong attention to detail and eager to learn about new technologies.

Leave a Reply