Simple Things You Can Learn From Cassandra Nodetool (Monitor/Manage) For DC/OS


As we know, DC/OS gave us minimal packages of services for running on apache mesos like cassandra. Cassandra services only give us the functionality for storing data , not other things like native tools as `nodetool`, `cqlsh`, `cassandra-stress` and more. Today, we are looking into the way for using `nodetool`, for monitoring DC/OS cassandra services nodes or cluster.

Brief:

Nodetool is a native command line interface for managing and monitoring cassandra cluster. While our cassandra services are running on DC/OS,  we can manage and monitor cluster using nodetool.  We need to execute nodetool on DC/OS using cassandra docker image on any of the DC/OS cassandra nodes by performing below steps:

  1. First, need to ssh any of  DC/OS cassandra node: ssh @
  2. Second, execute the docker command for download cassandra docker image and pass nodetool command for execution as below:
$ sudo docker run -t --net=host cassandra:3.0 nodetool tablestats -H .

Note: Sometime nodetool commands may not work if Cassandra version and Nodetool version is incompatible e.g: Cassandra Version 3.0 and Nodetool version 3.10, nodetool will not be able to run all commands.

Below, we are looking into some of the important nodetool commands for monitoring perspective of cassandra cluster. But still, nodetool have lots of commands, which we are not discussing now. For reference please click on this link.

1. CASSANDRA CLUSTER STATUS

For looking into cassandra cluster health or status using node tool, we need to execute below command:

Continue reading

Advertisements
Posted in Best Practices, big data, Cassandra, cluster, NoSql | Tagged , , , , , , , , , , , , , | Leave a comment

Knolx: Getting started with Presto


Hi all,

Knoldus has organized a 1-hour session on 8th September 2017. The topic was “Getting started with Presto”. Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides or video.

The slides of the Knolx are here:

And Here’s the video of the session:

For any queries, feel free to comment.


knoldus-advt-sticker


 

Posted in big data, Scala, sql | Leave a comment

Knolx: Introduction to ReactJs


Hi all,

Knoldus has organized a 30 min session on 1st September 2017. The topic was “Introduction to React JS”. Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides or video.

The slides of the KnolX are here:

And Here’s the video of the session:

I hope this helps.
For any queries, feel free to comment.


KNOLDUS-advt-sticker

Posted in JavaScript, React, ReactJS | Leave a comment

Getting Started with Ansible


What is Configuration Management?

Before starting with Ansible let’s discuss what configuration management is. Configuration Management (CM) is the process of handling changes in any system systematically, and it maintains the consistency of the product. It retains its consistency because it is applied over the entire lifecycle of the system. Configuration Management provides the capability of controlling and monitoring the performance of the system. Using this capability of monitoring we can prevent errors by notifying and controlling the capability to change anything in the system. If any of node in cluster gets failed we can reconfigure it. Also, configuration management keeps the snapshot of all the version of infrastructure.

Why Configuration Management?

The reason why we should use configuration management is to overcome the difficult situation that we face while setting up the cluster. Few of these are:

  • Managing multiple servers
  • Scale Up and Scale down
  • Sync up with development team and infrastructure team

What is Ansible? Continue reading

Posted in Devops | Tagged , , , | 2 Comments

Reactors IO: Actors Done Right


Hi all,

Knoldus has organized a 30 min session on 1st  September 2017 at 4:15 PM. The topic was Reactors IO. Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides or video.

The slides of the KnolX are here:

 

 

And Here’s the video of the session:

 

 

I hope this helps. 🙂
For any queries, feel free to comment.


knoldus-advt-sticker


 

Posted in Functional Programming, Java, Reactive, Scala | Tagged , , , , | Leave a comment

Join Semantics in Kafka Streams


Image result for apache kafkaIntroduction to core concepts:

 

Apache Kafka is a distributed streaming platform which enables you to publish and subscribe to a stream of records also letting you process this stream of records as it occurs.

Kafka Streams is a client library used for building applications and microservices, where the input and output data are stored in Kafka clusters.

Interface KStream<K, V> is an abstraction of record stream of key-value pairs. It is defined from one or more Kafka topics that are consumed message by message or as a result of KStream transformation.

Interface KTable<K, V> is an abstraction of changelog stream from a  primary-keyed table. Each record in this stream is an update on the primary keyed table with the record key as the primary key. Like KStreams, it is defined from one or more Kafka topics that are consumed message by message or as a result of a KTable transformation.

 

Joins in Kafka Streams:

Kafka Streams offer three types of joins,

  • KStream-KStream join
  • KTable-KTable join
  • KStream-KTable join

Continue reading

Posted in Apache Kafka, Scala | Tagged , , , , , , | 2 Comments

ScalaTest – How to Disable Test Cases using Tagging


Scala Test – How to disable Test Cases using Tagging

Tagging your tests

ScalaTest allows you to define some test categories, to “tag” tests as belonging to those categories, and filter tests to run based on their tags. For example, you could tag some tests as being slow and chose to exclude the slow tests during some runs, or say you have some intergration test that does not runs perfectly on your jenkins(or some other CI tool). ScalaTest supports one tag by default: ignore. You can tag a test as ignored to “switch it off” temporarily.

Tagging tests as ignored

For “temporarily” disabling a test(ofcourse you can restore it later), each style trait provides a way to tag tests as ignored. For example, in a FlatSpec you can change an it or an in to ignore.

Lets take an example –

Suppose you have a sample object named calculator that does the common calculation jobs.

/**
* Created by knoldus on 13/10/17.
*/
object Calculator{

def add(value1:Int, value2:Int): Int ={
value1 + value2
}

def sub(value1:Int, value2:Int): Int ={
value1 – value2
}

def mul(value1:Int, value2:Int): Int ={
value1 * value2
}

def div(value1:Int, value2:Int): Int = {
value2 match {
case value2 if value2 > 0 => value1 / value2
case _ => 0
}
}
}

and then comes its test cases –

Continue reading

Posted in Scala, scalatest | Leave a comment

Knolx – Introduction to D3.JS


Hi all,

Knoldus has organized a 30 min session on 4th August 2017 at 3:30 PM. The topic was Introduction to D3.JS. Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides.

The video of the session is here:

The slides of the KnolX are here:

For any queries, ask in the comments.

knoldus-advt-sticker


 

Posted in Scala | Leave a comment

Akka Split Brain Resolver


Hello all,

Knoldus organized a knolx session on the topic “Akka Split Brain Resolver” on Friday, 1st September 2017

In this session we have discussed about When operating an Akka cluster you must consider how to handle network partitions (a.k.a. Split Brain Scenarios) and machine crashes (including JVM and hardware failures). This is crucial for correct behavior if you use Cluster Singleton or Cluster Sharding, especially together with Akka Persistence.

Also The session takes you to the concept of SBR(split brain resolver) and various ways and techniques you can use to handle as per situations .

The slides for the session are as follows:

[]
You can also watch the video on you-tube :

KNOLDUS-advt-sticker

Posted in Scala | Leave a comment

Spark On Mesos(Installation)


In this Article We Will Learn How to Use Mesos On spark,so lets get started all you required is spark on your machine as a prerequisite,here are the steps to configure

1.Download Latest Mesos Version from here

2.extract the jar

3.Install Mesos dependencies.
$ sudo apt-get -y install build-essential python-dev python-six python-virtualenv libcurl4-nss-dev libsasl2-dev libsasl2-modules maven libapr1-dev libsvn-devInstall other Mesos dependencies.

4.Install libz which is required to build mesos

$sudo apt install zlib1g-dev

5.Configure and build mesos
$ mkdir build
$ cd build
build$../configure
$ make install

it will create libmesos.so file in /usr/local/lib folder

6.start mesos slave and master

$ cd build
#start master
./bin/mesos-master.sh –ip=127.0.0.1 –work_dir=/tmp/mesos
#start slave
./bin/mesos-slave.sh –master=127.0.0.1:5050 –work_dir=/tmp/mesos

navigate to mesos uri localhost:5050 you can see its working or not on worker tab there will be 1 active worker

Screenshot from 2017-10-09 11-16-24.png

7.now got to $SPARK_HOME/CONF

inside your spark-env.sh add following parameters

export MESOS_NATIVE_JAVA_LIBRARY= /usr/local/lib/libmesos.so
export SPARK_EXECUTOR_URI=/path/to/spark-2.2.0-bin-hadoop2.7.tgz

8. start spark shell with mesos as master

./bin/spark-shell –master mesos://127.0.0.1:5050

Screenshot from 2017-10-09 11-23-15.png

go to your mesos ui

Screenshot from 2017-10-09 11-25-06.pnghere is my active task now you can go to sandbox you can find the complete logs of your slave

Screenshot from 2017-10-09 11-26-22.pngi hope this blog will help


knoldus-advt-sticker


 

Posted in Scala | 1 Comment