Transfering Data from Couchbase to the ElasticSearch (Transport-Couchbase Plugin)

Reading Time: 3 minutes

If we want to transfer the data persisted in your Couchbase to ElasticSearch and use the power of inverted indexing of Elastic Search along with Couchbase. Then we can do this in easy steps.

Couchbase provides us a plugin for the Elastic search that makes your ElasticSearch node appear like a Couchbase Server node. After installation you can use the Cross-Datacenter Replication (XDCR) feature of Couchbase Server to transfer data continuously.

Installing Couchbase-Transport plugin:

Steps for installing the Couchbase transport plugin are:

  1. Go to your installation directory for Elasticsearch:
    cd elasticsearch-<version>
  2. Install the Couchbase Plug-in. Replace the version number with the appropriate one. You can check your own required version here.
    bin/plugin -install transport-couchbase -url \ \

    After a successful installation, the plug-in installer returns:

    DONE Installed transport-couchbase
  3. Set the user name and password for the plug-in:
    echo "couchbase.password: password" >> config/elasticsearch.yml; 
    echo "couchbase.username: Administrator" >> config/elasticsearch.yml    You can add it permanently inside your elasticsearch.yml for easy access. 
  4. Install kopf, a third-party plug-in for Elasticsearch that provides a simple web-based interface you can use to interact with Elasticsearch:
    ./elasticsearch/bin/plugin install lmenezes/elasticsearch-kopf/{branch|version}
  5. Start Elasticsearch:

    Elasticsearch starts and runs on your machine in the background.

Updating configurations:

  1. For making index into the elastic search you have to set a default template for your index.
    This default template json can be found here. You can edit it as per your need and change the mapping of your index accordingly as per the need.

The command for making the default template is :

curl -XPUT http://localhost:9200/_template/couchbase \ -d @plugins/transport-couchbase/couchbase_template.json

You can also set it by copying the content of  couchbase_template.json and making the  template from kopf UI like this:

Screenshot from 2016-01-15 23-17-59

2. Now make the index for the bucket whose data is to be copied.
the command for making the index is:

curl -XPUT http://localhost:9200/<index-name&gt;

For more insight in the ElasticSearch you can refer here.

Making XDCR replication from Couchbase to the ES.

Now you can make an XDCR replication on the ElasticSearch server.This can be done in small and easy steps.

  1. Make connection to the ElasticSearch server from the XDCR tab in the Couchbase UI.ElasticSearch server
  2. Add an ongoing replication to the server given above.
    1. If you are using Couchbase Server 2.2 or later, click Advanced settings and change the XDCR Protocol setting to Version 1.


  3. You can use the advance filtering option for copying a particular type of document and can easily specify a regex for the document Id that you want to copy.RegexAdvance
  4. Start the XDCR and you are good to go, and it starts getting copied inside your ES. Which will have the whole document as well as the couchbase meta-data for each and every document.
  5. For more tuning of the elastic search you can use the following settings for your elasticSearch which can be feed up into the elasticsearch.ymlsettings

Now you are good to go to see your data inside your ElasticSearch by making a Query to the ElasticSearch. For more insight in the CRUD operations on ES and Querying ES you can refer this link. 

If you have any doubts. We’d be happy to solve them. Ping us!

6 thoughts on “Transfering Data from Couchbase to the ElasticSearch (Transport-Couchbase Plugin)3 min read

    1. Yes, updating documents in ElasticSearch is a lot simpler in CouchBase than in ElasticSearch. Additionally, it’s easy to do something bad in ElasticSearch. Having a firewall between ES and CB gives another layer of security.

      1. Can you please elaborate. Updating a elastic search document is quite trivial and I want want to define the indexes as I please. More control is always better, don’t you think?

  1. Can someone explain this? Why would I not use Elastic search directly and use it via Couchbase instead. Seems like an unnecessary hard coupling.

Comments are closed.