How to query external hive Metastore From Spark


In this Blog we will learn how can we access tables from hive metastore in spark,so now just lets get started

  • start your hive metastore as  as service with following command
  • hive –service metastore by default it will start metastore on port 9083

Screenshot from 2017-10-24 20-47-58.png

  • go to your spark client and tell driver to conncet to metastore listening on port 9083  when starting the hive thrift server with following command
  • $SPARK_HOME/sbin/start-thriftserver.sh –driver-java-options “-Dhive.metastore.uris=thrift://localhost:9083”

Screenshot from 2017-10-24 20-53-28.png
as you can see show tables show all the tables that are in hive metastore now they are available to spark-client

i hope this blog will help


knoldus-advt-sticker


 

Advertisements
This entry was posted in Scala. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s