upload

Congregating Spark files on S3

January 12, 2016July 20, 2018Amazon, Apache Spark, Big Data and Fast Data, Spark, Studio-ScalaAmazon Services, Apache Spark, Big Data, cluster computing, File upload, RDD, Spark, upload

Reading Time: 2 minutes We all know that Apache Spark is a fast and general engine for large-scale data processing and it is because of its speed that Spark was able to become one of the most popular frameworks in the world of big data. Working with Spark is a pleasant experience as it has a simple API for Scala, Java, Python and R. But, some tasks, in Spark, are still tough rows Continue Reading

S3Ninja an Introduction

December 31, 2015January 11, 2016Studio-ScalaAkka, Integration Testing, Integration Testing ScalaTest, Knoldus, Play framework, S3, s3ninja, scala, upload, uploading

Reading Time: 2 minutes S3Ninja is an emulator that emulates the S3API. S3Ninja provides an environment for your local system to support integration of upload a file, just as we do on S3. Currently it supports objects methods only like GET, PUT, HEAD, DELETE. S3Ninja can be used, to upload file on our local system instead of S3 to write integration tests that may integrate with upload of file Continue Reading