Neo4j With Scala : Migrate Data From Other Database to Neo4j

Hello Folks;

Lets continue the Neo4j with Scala. We have earlier discuss about the use of Neo4j with Scala and Neo4j APOC with Scala. In this blog we are going to discuss about how we can migrate data from the other database like MYSQL, PostgreSQL, Oracle and Cassandra.

But before starting the journey, To those who have caught the train late 😉 , this is what has happened till now:

  1. Getting Started Neo4j with Scala : An Introduction
  2. Neo4j with Scala: Defining User Defined Procedures and APOC

So till now we have some basic understanding of Neo4j and APOC with Scala. Now we will start blog for the series.

We use many databases for storing the data. But when we have a large amount of data and tables that time it becomes so hard to make query and execute them on the database. We have to be extra cautious to perform the task and we get bored to see same screen without any fun 😉 .

We have solution for this. We can use Neo4j for this, where we find more fun to do work and its not difficult as we have discuss it in first blog. Before starting the discussion I want to clear one think that when we want to migrate data then we have to keep Neo4j APOC Kit (Download and Install) in the $Neo4j_Home/plugins. Now we start discussing about every database which I have mention before that how we can connect and transfer data from that database to Neo4j :

PostgreSQL :

When we use PostgreSQL, we have to download JDBC .jar file (Download) and keep it in the $Neo4j_Home/plugins and restart the Neo4j. After restarting the Neo4j server we are set for migrating the data from the PostgreSQL to Neo4j.

  • Now we load the driver with the APOC.

  • Now we create the call for fetching the data from the PostgreSQL where we have a table with name employee_details to Neo4j.
  • If we don’t want to use these step than we can provide URL in the $Neo4j_Home/conf/neo4j.conf and restart the server :

We can now fetch data direct. We don’t need to load driver also.

  • Create Nodes and Relation in the data.

Screenshot from 2016-09-09 11:09:37.png

screenshot-from-2016-09-09-010005

MYSQL :

We want to migrate data from the MYSQL as before we have to download JDBC .jar file (Download) and keep it in the $Neo4j_Home/plugins and update $Neo4j_Home/conf/neo4j.conf as:

Restart the Neo4j server and we are set for migrating the data from the Cassandra to Neo4j.

  • We hit the MySQL and start fetching data and perform count operation.

Screenshot from 2016-09-10 12:36:34 (copy).png

Cassandra :

Now we migrate data from the Cassandra to Neo4j. Now we first import data into the cassandra if we don’t have data in the cassandra or we can use it for test also.

  • We have to run following command for setting up initial data in the cassandra :
  • We have set our cassandra database with the data. We have to download JDBC .jar file (Download) and keep it in the $Neo4j_Home/plugins. We can provide URL in the $Neo4j_Home/conf/neo4j.conf as :

Restart the Neo4j server and we are set for migrating the data from the Cassandra to Neo4j.

  • We hit the cassandra and start fetching data and perform count operation.

screenshot-from-2016-09-09-130005

  • Let’s create Index, Constraints and Relation the data.

Screenshot from 2016-09-09 13:12:25.png

  • Now we will load data and perform Merge and Create operation so that we can create the node and relationship between the node.

Screenshot from 2016-09-09 13:13:28.png

  • We can see Relation Graph and it will look something like this :

Performed.png

Screenshot from 2016-09-09 14:11:41.png

Oracle :

We are in the last database to migrate data to Neo4j but as obvious not least. We can download JDBC .jar file (Download) and keep it in the $Neo4j_Home/plugins and restart the Neo4j. We can provide URL in the $Neo4j_Home/conf/neo4j.conf as :

  • After restarting the Neo4j server we are set for migrating the data from the Oracle to Neo4j. We fetch the data from the Oracle where we have a table with name employee_details to Neo4j.Now we load the driver with the APOC.

screenshot-from-2016-09-10-123634

  • Let’s create Index, Constraints and Relation the data.

Screenshot from 2016-09-10 14:53:00.png

  • Now we will load data and perform Merge and Create operation so that we can create the node and relationship between the node.

screenshot-from-2016-09-10-145400

  • We can see Relation Graph and it will look something like this :

screenshot-from-2016-09-10-145506

Now we can see that it is so easy to migrate data from other database to Neo4j.

After importing the data in Neo4j, we have to thing about the sync of data. We can use schedule process which can be timebase and automatically sync data between the databases. We can also used event based integration where we will defined the event at which we want to update the database.

Note : As we discuss I want to notify again if you do not update driver name into $Neo4j_Home/conf/neo4j.conf then you have to load driver in Neo4j otherwise you have to provide only driver name into the query.

I hope it will help for migrating data into the Neo4j. After fetching data we can write simple Scala code for persisting data in the Neo4j as we discuss in the first blog.

Reference:

  1. Neo4j Apoc Procedures

 


KNOLDUS-advt-sticker

Written by 

Anurag is the Sr. Software Consultant @ Knoldus Software LLP. In his 3 years of experience, he has become the developer with proven experience in architecting and developing web applications.

6 thoughts on “Neo4j With Scala : Migrate Data From Other Database to Neo4j

Leave a Reply

%d bloggers like this: