Error handling in Scala: What, where and how?


The honest way of Handling an errors is to making it to the end user and telling the exact thing that has been happened over failures. Sometimes we can’t afford longer debugging sessions in case of mission critical things. The following phrases won’t help much in debugging when seen by users:

“Blah” went wrong!! (“What?
Your Transaction can’t be complete !! (“why ?
Not possible … (“why?”
Engine did not start… (“why?
Blah.. Blah..

Should not the above phrases have been informative. E.g. “Engine did not start because the Pressure tolerance is 0.05 exceeding to safe pressure tolerance 0.0005”

In this blog, we will talk about some monadic constructs in Scala and other scala based library which saves a lot of debugging sessions when it comes to failure. In this blog,  how we would choose between them according to our need.

An Easy Example?

Let us take the example of validation. We always require at some point of our computation to validate input data. Suppose we have to take an address and pass it further if it is valid. Consider the following example:

def validateStreetAdd(street:String): Option[String] =
    if(street.nonEmpty && street.matches("some valid address")) Some(street) else None

Is “Option” a real option for the use case?

It is designed for a super simple use case. Sometimes you have some value and sometimes you have nothing. Suppose the above method is called with a value “some invalid address”

Here we’ve got following things not correct with our Code:
Unfortunately we can’t use None to process it further until we have a valid address.

When there is no error or backward communication, the users assumed that they had their “thing” done by typing an invalid address: A miscommunication.

The Ideal improvement at the moment would be the following modification of the current version:

def validateStreetAddressWithDefault(street:String): Option[String] =
    if(street.nonEmpty && street.matches("some valid address")) Some(street) else Some("Some Default Address")

Or If you don’t want to do this, you can call the previous version with a .getOrElse(“Default Value”). In fact, latter is the better option, because you have the flexibility to choose the Default value rather than hardcoding it as part of method. Now we are somewhat removed the first problem here but passing a default address for all of the customers would be the last option the users will agree upon. They still want to know what is the exact reason for the failure or What is not correct from the input.

Let’s Try to return the error.

Try[T] is another construct to capture the success or a failure scenarios. It returns a value in both cases. Put any expression in Try and it will return Success[T] if the expression is successfully evaluated and will return Failure[T] in the other case meaning you are allowed to return the exception as a value. However with one restriction that it in case of failures it will only return Throwable types:

def validateZipCode(zipCode:String): Try[Int] = Try(zipCode.toInt)

But Throwing an exception doesn’t make much sense here since it is not much of a calculation. Although we can take this example to understand the use case. If the given string is not a number, it will be a failure. The value from the Try can be extracted in same as Option. It can be matched

validateZipCode("123456") match { case Success(zip) => zip ; case Failure(e) => 100001 }

The above can be replaced with below:
validateZipCode(“123456”).getOrElse(100001)
Try can be mapped over its values:

validateZipCode("123456").map( _ .someOperation)

What if we don’t have to throw an Exception but return something else?

If you don’t want always to throw an exception in failure cases and make an exception/Error of your own, you can use scala.util.Either[A,B]. It gives you two type options A and B to return any information you’ve computed. Do not confuse it with tuple. It will only let use one at a time i.e. either scala.util.Left[A] or scala.util.Right[B]. Look at the enhanced version of zip code validation method:

case class ValidationError(msg:String)

def validateZipCode(zipCode:String): Either[ValidationError, String] =
 (zipCode.matches("^[0-9]+$"), zipCode.length == 6) match {
   case (true,true) => Right(zipCode)
   case (false,true) => Left(ValidationError("Zip Code should be Number"))
   case (true,false) => Left(ValidationError("Zip Code should be exactly 6 digits"))
   case _ => Left(ValidationError("Zip Code should be exactly 6 digits"))
 }

The Output:

scala> validateZipCode("1123")
res0: Either[ValidationError,String] = Left(ValidationError(Zip Code should be exactly 6 digits))
scala> validateZipCode("invalid")
res1: Either[ValidationError,String] = Left(ValidationError(Zip Code should be exactly 6 digits))
scala> validateZipCode("123456")
res2: Either[ValidationError,String] = Right(123456)

By semantics, If you’ve declared the return type Either[A,B] then you are bound to wrap type A in Left and type B in the Right. Going by the scala convention usually you should Project your “Good” values in the Right and “Failures/Errors/Exceptions” in the Left. But don’t worry if you are left handed person! You can always do

def validateZipCode(zipCode:String): Either[ String, ValidationError]

What if convention makes you efficient?

There is another construct called Or which is simpler and more powerful than Either in the Scala. The Or is from the Scalactic library. It uses infix Notation as opposed to Either.
Or has a very good values vs error projection. Either it will return a value wrapped in Good or anything which is not Good is Bad. Bad will wrap whatever has been declared as part of return type in Or. Look at the enhanced version of the validateZipCodeWithOr method below:

def validateZipCodeWithOr(zipCode:String): String Or ValidationError = {
 (zipCode.matches("^[0-9]+$"), zipCode.length == 6) match {
   case (true, true) => Good(zipCode)
   case (false, true) => Bad(ValidationError("Zip Code should be Number"))
   case (true, false) => Bad(ValidationError("Zip Code should be exactly 6 digits"))
   case _ => Bad(ValidationError("Zip Code should be exactly 6 digits"))
 }
}

Here is how I used it:

scala> validateZipCodeWithOr(“110020”)
res0: Good(110020)
scala> validateZipCodeWithOr(“Hello”)
scala: Bad(ValidationError(Zip Code should be exactly 6 digits))

Collecting Errors:

Sometimes giving a detailed error report after all inputs is very useful and saves a lot of efforts for users. Every in Scalactic library used to accumulate all the errors which can be summarized as a report:

Accumulating errors with Or

I could not resist the use of Copy and paste from Scalactic library to make you understand the use of Or for collecting Errors.

“Another difference between Or and Either is that Or enables you to accumulate errors if the Bad type is an Every. An Every is similar to a Seq in that it contains ordered elements, but different from Seq in that it cannot be empty. An Every is either a One, which contains one and only one element, or a Many, which contains two or more elements.

Here is a another variant for validating zip code with return type as Every.

def validateZipCodeWithOrEvery(zipCode:String): String Or One[ValidationError] = {
 if (zipCode.length == 6 ) Good(zipCode) else
   Bad(One(ValidationError("Zip code should be exactly 6 digits")))
}

Now let us Validate all addresses at Once. The below snippet uses Or Every for collecting all the errors while validating address. As you can see the return type it is: Address Or Every[ValidationError]

The Every[Validation] error will contain each error when validation fails of any address component e.g. street address or Zip Code.

def validateAddress(streetAddress:String, zipCode:String): Address Or Every[ValidationError] = {
   val stAddress = validateStAddressWithOrEvery(streetAddress)
   val zpCode = validateZipCodeWithOrEvery(zipCode)
 withGood(stAddress, zpCode){Address(_, _)}
}

The entire result will be wrapped in Good if all validations succeed while parsing address.

scala> validateAddress("Hello Street","123456")}
res0: Good(Address(Hello Street,123456))

The Error will be returned in the following way if Any validation fails:

scala> validateAddress("","ghb")}
res0: Bad(Many(ValidationError(Empty Adress!!), ValidationError(Zip code should be exactly 6 digits)))
scala> validateAddress("Non Empty","ghb")}
res1: Bad(One(ValidationError(Zip code should be exactly 6 digits)))<span 				data-mce-type="bookmark" 				id="mce_SELREST_start" 				data-mce-style="overflow:hidden;line-height:0" 				style="overflow:hidden;line-height:0" 			></span>

You can visit the library if you want to explore more about scalactic and other ways in which the errors can be collected.
Hope you liked reading the material. You are more than welcome for any suggestions!!

knoldus-advt-sticker

Advertisements
Posted in Scala, scalatest | 1 Comment

Gatling | Maven Setup.


Gatling is a highly capable load testing tool. It is designed for ease of use, maintainability and high performance. As this one-liner explanation doesn’t explain the whole concept, please visit this nice presentation: why should we use Gatling and what’s it has for us here. Well, I’ll take you through how we can implement Gatling in a project which uses maven builds.

So, We need to create a maven module/sub-project or a stand-alone maven project for Gatling with a meaningful name e.g. gatling-simulations or gatling-tests etc. So, you may have two cases here first you are implementing a gatling-simulator in a maven project where you’ll make gatling-test as a module of this project, second is when you can make a new Maven Gatling-Simulator standalone. Both cases are almost same just the directory structure may differ. For example:

The approach is exactly same, as both are maven projects. So, let’s start.

Continue reading

Posted in gatling, Scala | Leave a comment

Config Generator In Hyperledger Fabric


Every Ledger starts with a transaction kept inside the block but what will be the first block? Well, the answer is genesis block. Now, another question how to generate this genesis block? For this part we can use the configtxgen tool to generate the initial or genesis block.
The tool is primarily focused on generating the genesis block for bootstrapping the orderer, but it is intended to be enhanced in the future for generating new channel configurations as well as reconfiguring existing channels.

This tool takes the parameter in form of the configtx.yaml file. You can find the sample configtx.yaml file under the directory fabric/sampleconfig/configtx.yaml on github.

Lets explore the configtx.yaml file. This file mainly contains the following sections:-

  • The Profiles section.
  • The Organizations section.
  • The default sections.

The Profile Section :- Profiles can make a good starting point for construction a real deployment profile.Profiles may explicitly declare all configuration, but usually inherit configuration from the defaults Section.

Sample Profile Configuration:-
Continue reading

Posted in Scala | Leave a comment

Rules while working with stream in Java 8


First, let’s have a basic understanding of stream. Then we will have a look at the side effects that can occur while working with streams.

Stream represents a sequence of objects from a source, which supports aggregate operations. One thing to be notified while working with streams  is that, aggregate operation (intermediate operations) are lazy evaluated i.e. they do not start processing the contents of the stream until the terminal operation commences. This enables the Java compiler and runtime to optimise how they process streams.

With Java 8, Collection interface has two methods to generate a Stream.

  1. stream() − Returns a sequential stream considering collection as its source.

Given below is a very basic example in which array elements of type double which are less than 5 are converted into integer. The order in which the output received is fixed since the we are working with streams.


/**
* Basic example of streams
*/
Double[] doubleArray = {1.8, 2.0, 3.2, 4.5, 5.6, 6.0, 7.0, 8.9};
List<Double> listOfDouble =
new ArrayList<>(Arrays.asList(doubleArray));

listOfDouble
.stream()
.filter(element -> element < 5) .mapToInt(value -> value.intValue()).
forEach(System.out::println);

  1. parallelStream() − Returns a parallel Stream considering collection as its source.

Parallel computing involves dividing a problem into subproblems, solving those problems simultaneously (in parallel, with each subproblem running in a separate thread), and then combining the results of the solutions to the subproblems. When a stream executes in parallel, the Java runtime partitions the stream into multiple substreams. Aggregate operations iterate over and process these substreams in parallel and then combine the results.

Continue reading

Posted in Best Practices, Java, Streaming | Tagged , , , , , , , , | Leave a comment

A step-by-step guide for protecting sensitive data in docker


Managing the password, access tokens and private keys are being tedious in the application. Any small mistakes accidentally expose all the secret information. Even storing such thing in docker images can be easily accessible one should just run the image in the interactive mode container and all your application code is available in containers. Docker provides secrets to protect all secret data.

This blog explains the low-level of storage information as well as secured access to docker secret. so, let’s get started.

What is Docker Secret?

A secret is a blob of data may consist of password or any other sensitive information. Docker secret centrally manages this data and securely transmit to containers that need to access it. A secret is encrypted over transport and only accessible to granted containers. Docker secret only works in swarm services, not available to the standalone container. Let’s understand how docker secret works.

Architecture

swarm-architecture

Continue reading

Posted in cluster, Devops, Scala | Tagged | Leave a comment

Crypto Generator in HyperLedger Fabric


Security is the one of the major aspect of any network. Each of the node must have some identity and on the basis of this identity corresponding accesses are granted.

The same approach is follow by the Fabric network as well. Fabric CA generates the identity or artifacts file for each of the node that can be the part of the cluster but for generating these artifacts files we also need to specify some property . Well these properties can be specified in crypto-config.yaml file.

Crypto Configuration file:-
crypto-config file contains the following information:-

OrdererOrgs – Definition of organizations managing orderer nodes.
PeerOrgs – Definition of organizations managing peer nodes.

OrdererOrgs:-
OrdererOrgs contains the following Information about the Ordered Node in the Cluster.

Name:- Name of the Orderer
Domain:- Domain URL for Orderer
Hostname:- Host name for the Orderer. This came under the Specs section.

Sample Ordered Configuration:-
Continue reading

Posted in Scala | Leave a comment

Working With XML


Hi All,

In this blog, I will share some basic information that will help you more to work with an XML file.The first thing to know about XML in Scala is that Scala can process XML literals. so you don’t need to have any specific dependencies for working with XML. you can directly put the whole XML to Scala REPL and  Scala will automatically interpret them as XML elements.

so let’s do some basics example to get an understanding of how to play with XML
scala> val foo = <foo><bar type="greet">hi</bar><bar type="count">1</bar><bar type="color">yellow</bar></foo>
foo: scala.xml.Elem = <foo><bar type="greet">hi</bar><bar type="count">1</bar><bar type="color">yellow</bar></foo>

so if you want to retrieve the values of each tag then you can use –

scala> foo.text
res0: String = hi1yellow

so here you can see I just apply  .text on foo and it gave me the text values of each XML tag without any space.

if you want to retrieve the value of the “bar” tag then, in that case, you can use “\” Selector and tag name like –

scala> foo \ "bar"
res1: scala.xml.NodeSeq = NodeSeq(<bar type="greet">hi</bar>, <bar type="count">1</bar>, <bar type="color">yellow</bar>)

Continue reading

Posted in Scala | Leave a comment

Hyperledger Fabric Certificate Authority(CA)


Every operation in Hyperledger must be signed cryptographically with certificates. You can generate certificates yourself using OpenSSL or by using third party. Before moving further into details of CA lets first explore Hyperledger Fabric a little. 😉

Hyperledger Fabric

Hyperledger Fabric founded in 2015 which is an umbrella for open source projects some of which are Blockchain Distributed Ledger Frameworks such as Fabric, Sawtooth and Iroha. Hyperledger Fabric is a permissioned blockchain, means that parties that join the network are authenticated to participate on network. It reduces security risks and display records to only to the parties involved. It provides:

  • Data Privacy
  • Information Sharing
  • Immutability

That was a concise description about Hyperledger Fabric. Now, lets explore importance of Hyperledger Fabric CA.

Fabric Certificate Authority (CA)

Fabric CA is a tool through which you can generate certificates. Let say you have 10 users then, 10 certificates get generated for 10 users. You can add additional information called as attributes in certificates. So this information is propagated to the system. Chaincodes (Smart Contracts in Blockchain) can read this data and perform different operations As this information is within certificates you can’t modify it which makes process secure.

You can generate certificates by specifying the username, password and affiliations which is called as Enrollment. With these certificates you have to sign each and every request. Some data of the certificates will be stored inside the ledger through which you can know who actually execute the operation.

Continue reading

Posted in Scala | Tagged , , , , | Leave a comment

Java 8 Stream : Need with examples and Limitations


Before starting with java 8 stream firstly look at this example that will illustrates the need of java 8 stream and the power of stream as well.


private static int sumIterator(List list) {
	Iterator it = list.iterator();
	int sum = 0;
	while (it.hasNext()) {
		int num = it.next();
		if (num > 10) {
			sum += num;
		}
	}
	return sum;
}

There are some issues with this approach :-

  1. Here we just want the sum of integers but we would also have to provide how the iteration will take place, this is also called external iteration because this  program is handling the algorithm to iterate over the list.
  2. This program is sequential in nature, so here parallel execution can’t take place.
  3. We have to write too much code for just this simple task.
  4. Time taking approach.

To overcome all these shortcomings, Java 8 Stream API was introduced. We can use Java Stream API to implement internal iteration, that is better because java framework is in control of the iteration.

Mostly java 8 Stream API method arguments are functional interfaces, so that is why lambda expressions work very well with them. Not take a look how we can write above logic in a single line statement using Java Streams.


private static int sumStream(List list) {
	return list.stream().filter(i -> i > 10).mapToInt(i -> i).sum();
}

What is Stream ?
Stream represents a sequence of objects from a source, which supports aggregate operations. Most of people think streams is like collection but streams are different from collections.   

The classes StreamIntStreamLongStream, and DoubleStream are streams over objects and the primitive intlong and double types. Streams  are  much differ from the collections in several ways:

  • Stream doesn’t store data, it operates on the source data structure (collection and array) and produce pipelined data that we can use and perform specific operations. Such as we can create a stream from the list or array and filter it based on a condition.
  • Java 8 Stream support sequential as well as parallel processing, parallel processing can be very helpful in achieving high performance for large collections. That is why we can say it is Possibly unbounded.  While collections have a finite size, streams need not. Short-circuiting operations such as limit(n) or findFirst() can allow computations on infinite streams to complete in finite time.
  • Laziness-seeking. Many stream operations, such as filtering, mapping, or duplicate removal, can be implemented lazily, exposing opportunities for optimization. For example, “find the first String with three consecutive vowels” need not examine all the input strings. Stream operations are divided into intermediate (Stream-producing) operations and terminal (value- or side-effect-producing) operations. Intermediate operations are always lazy.In Java 8 Streams API, the intermediate operations are lazy and their internal  processing model is optimized to make it being capable of processing the large
    amount of data with high performance.
  • Consumable. This is not possible to reuse the same stream for multiple times .  The elements of a stream are only visited once during the life of a stream. Like an Iterator, a new stream must be generated to revisit the same elements of the source.
  • Functional in nature. Java 8 Stream operations use functional interfaces, that makes it good standard for functional progamming using lambda expression. An operation on a stream produces a result, but does not modify its source. For example, filtering a Stream obtained from a collection produces a new Stream without the filtered elements, instead of removing elements from the source collection.

Continue reading

Posted in Scala | Leave a comment

Artificial Intelligence vs Machine Learning vs Deep Learning


The world as we know it is moving towards machines big time. But we can not fully utilize the working of any machine without a lot of human interaction. So in order to do that, we needed some kind of intelligence for the machines. Here comes the place for Artificial Intelligence. It is the concept of machines being smart to carry out numerous tasks without any human intervention.

Artificial Intelligence and Machine Learning both terms often lead to the confusion and many of us don’t exactly know the difference between them. Hence we end up using these terms interchangeably. Machine Learning is basically the learning concepts of machines through which we are going to achieve Artificial Intelligence. Deep Learning is the latest thing in Artificial Intelligence field. It is one of the ways to implement Machine Learning to achieve AI.

Most of us have seen the AI based movies with machines having its own intelligence like Terminator series, I Robot. But in real life, the AI concept was not that much optimised to handle these real-life situations and act accordingly. Mostly AI implementations were only situation based codings. Where Machine Learning was introduced to handle that much amount of data and to make the machines learn using inputs/examples to process further problems.

Continue reading

Posted in artificial neural network, machine learning, Scala | Tagged , , | 1 Comment