Knoldus Blogs

Apache Spark: Read Data from S3 Bucket

January 7, 2020March 12, 2020Amazon, Analytics, Apache Spark, Big Data and Fast Data, Cloud, Database, ML, AI and Data Engineering, Scala, Spark, SQL, Tech BlogsAmazon S3, AWS, Big Data, Big Data Analytics, Big Data Storage, data analysis, fast data analytics

Reading Time: < 1 minute Amazon S3 Accessing S3 Bucket through Spark Edit spark-default.conf file You need to add below 3 lines consists of your S3 access key, secret key & file system spark.hadoop.fs.s3a.access.key “s3keys” spark.hadoop.fs.s3a.secret.key “yourkey” spark.hadoop.fs.s3a.impl org.apache.hadoop.fs.s3a.S3AFileSystem

How to create a bucket on Amazon S3 and getting security credential keys?

February 11, 2019March 4, 2019AWS, AWS ServicesAmazon S3, AWS, AWS Services

Reading Time: 3 minutes Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. This blog describes : how you can create buckets on S3, getting credential keys, where should you keep your credential keys. CREATION OF BUCKET First of all, you need to sign up in aws S3 after that Continue Reading

How to upload any file on Amazon S3 using Rust?

February 11, 2019February 11, 2019AWS Services, Functional Programming, RustAmazon S3, Rust, rustfmt

Reading Time: 3 minutes Welcome everyone to the file upload on Amazon S3 using the Rust. Amazon S3 [Amazon Simple Storage Service] provides virtually limitless storage on the internet. For the bucket creations and security credentials please refer to my last blog. This blog explains following requests using Rust: sending a request to aws S3 bucket, list of objects in the bucket, putting an object in the bucket, deleting Continue Reading