Knoldus Blogs

Saving Spark DataFrames on Amazon S3 got Easier !!!

February 8, 2016July 23, 2018Amazon, Scala, Sparkamazon, Apache Spark, Big Data, S3, scala

Reading Time: < 1 minute In our previous blog post, Congregating Spark Files on S3, we explained that how we can Upload Files(saved in a Spark Cluster) on Amazon S3. Well, I agree that the method explained in that post was a little bit complex and hard to apply. Also, it adds a lot of boilerplate in our code. So, we started working on simplifying it & finding an easier way to provide a wrapper around Spark Continue Reading

S3Ninja an Introduction

December 31, 2015January 11, 2016ScalaAkka, Integration Testing, Integration Testing ScalaTest, Knoldus, Play framework, S3, s3ninja, scala, upload, uploading

Reading Time: 2 minutes S3Ninja is an emulator that emulates the S3API. S3Ninja provides an environment for your local system to support integration of upload a file, just as we do on S3. Currently it supports objects methods only like GET, PUT, HEAD, DELETE. S3Ninja can be used, to upload file on our local system instead of S3 to write integration tests that may integrate with upload of file Continue Reading

AWS Services: AWS SDK on the Scala with Play Framework

April 19, 2015April 19, 2015Akka, Amazon, Amazon EC2, AWS, AWS Services, Bootstrap, Future, Play FrameworkAkka, amazon, Amazon EC2, Amazon Services, AWS SDK, Bootstrap, Clean code, Code Quality, Play framework, S3

Reading Time: 3 minutes playing-aws-scala The following blog and attached code represent a simple example of Amazon Web Services in the Scala way with Play Framework using AWScala but in this blog I have implemented only Amazon Simple Storage Service (Amazon S3) functionalities. AWScala: AWS SDK on the Scala REPL AWScala enables Scala developers to easily work with Amazon Web Services in the Scala way. Though AWScala objects basically Continue Reading