Databricks Command Line Interface is one of the handiest CLI which can come to the rescue whenever you are working with Databricks notebook. It is an easy-to-use interface for the Databricks platform which is built upon the Rest API 2.0.
To Get Started With Databricks CLI
Installing the DBCLI
For installing the CLI you must have these prerequisite in your local machine:
- Python2(version 2.79 & above) or Python3(version 3.6 & above)
- PIP(Preferred Installer Program).
# For installing the CLI pip install databricks-cli #For Updating the CLi pip install databricks-cli --upgrade #In order to get the version of your DBCLI databricks -v
Setting Up the Authentication
Now in order to access your resource from the cloud, you need to set up your databricks cloud credentials details in your local machines in a filename ~/.databrickscfg. You can store your credentials using the command
#This command will let you add the host name databricks configure --token #Output of the above command Databricks Host (should begin with https://):
Instead of the token, you can also use your username and password of your databricks but it’s not recommended by databricks. CLI with the version above 0.8.0 supports the following environment variables:
Following is the example for using the CLI.
databricks --profile AZURE fs cp -r <src> <desitnation>
Useful Databricks CLI Commands
There are many databricks command which a user can use like Cluster Policies Cli, Cluster Cli, Tokens, DBFS Cli and many more but I’ll talk about the mostly used CLI commands which can be really helpful, so first I wanna talk about:
Databricks File System CLI is specially used for performing basic performation like move, delete, copy etc.
But you need to be careful while using the DBFS CLI for the operation i.e containing the files more than 10k which can leads to timeout situation.
The alternative in this type of situation will be File system utility (dbutils.fs).
- Listing the Contents of A File
databricks --profile AWS fs cat dbfs:/my-temp/aTextFile.txt // Output This is the DB CLI Blog
- Copy a File
databricks --profile AZURE dbfs:/my-temp/aTextFile.txt /home/ravi/myNas
- Creating a Directory
databricks fs mkdirs dbfs:/tmp/new-dir
- Deleting a File
databricks fs rm dbfs:/my-temp/aTextFile.txt
Thanks for making to the end of the blog, if you still want to learn more about Databricks CLI you can refer to their official documentation by clicking the link mentioned below.
If you like my writing sense then feel free to check my more blogs by clicking here.