AWS

Using CodeShip with CodeDeploy

Reading Time: 5 minutes What is CodeShip ? CodeShip is a hosted continuous delivery service that focuses on speed, simplicity and reliability. It is a fast and secure hosted CI service that scales as per user’s needs. CodeShip was founded in 2011, and pretty soon it became one amongst the Saas Continuous Integration and Continuous Delivery leader in the market. CodeShip is now backed by a larger company with Continue Reading

Loading JSON data into Snowflake

Reading Time: 4 minutes Have you ever faced any use case or scenario where you’ve to load JSON data into the Snowflake? We better know JSON data is one of the common data format to store and exchange information between systems. JSON is a relatively concise format. If we are implementing a database solution, it is very common that we will come across a system that provides data in Continue Reading

Amazon EMR

Reading Time: 3 minutes Businesses worldwide are discovering the power of new big data processing and analytics frameworks like Apache Hadoop and Apache Spark, but they are also discovering some of the challenges of operating these technologies in on-premises data lake environments. They may also have concerns about the future of their current distribution vendor. Common problems of on-premises big data environments include a lack of agility, excessive costs, Continue Reading

Data Lake – Build it in Phases

Reading Time: 3 minutes Data Lake – How to build a data lake and what are the phases involved in the same.

Apache Spark: Read Data from S3 Bucket

Reading Time: < 1 minute Amazon S3 Accessing S3 Bucket through Spark Edit spark-default.conf file You need to add below 3 lines consists of your S3 access key, secret key & file system

Getting started with Amazon SNS

Reading Time: 2 minutes Introduction The Simple Notification Service (SNS) is used as a publish and subscribe messaging service. But what does it mean? SNS is centered around topics and you can think of a topic as a group for collecting messages. Users or endpoints can then subscribe to this topic and messages or events are then published to that topic. When a message is published, all subscribers to Continue Reading

Getting started with Amazon SQS

Reading Time: 4 minutes With the continuing growth of microservices and a cloud best practice of designing decoupled systems, it’s important that developers have the ability to utilize a service or system that handles the delivery of messages between components and this is where SQS comes in. Amazon SQS (Simple Queue Service) is a fully managed service offered by AWS, that works seamlessly with server-less systems, microservices or any Continue Reading

Managing Terraform State

Reading Time: 4 minutes In this blog, We are going to learn how Terraform keeps tracks the state of your infrastructure and configuration. With the help of an example, we will learn how can we store state file to a remote location. We can create infrastructure on a cloud in various ways using CLI, directly using UI or any automation tool like terraform etc. Then how terraform would know Continue Reading

How to create a bucket on Amazon S3 and getting security credential keys?

Reading Time: 3 minutes Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. This blog describes : how you can create buckets on S3, getting credential keys, where should you keep your credential keys. CREATION OF BUCKET First of all, you need to sign up in aws S3 after that Continue Reading

OpsInit: Packer On AWS

Reading Time: 3 minutes What is Packer? Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration. Packer is lightweight, runs on every major operating system, and is highly performant, creating machine images for multiple platforms in parallel. Packer does not replace configuration management like chef or Puppet. In fact, when building images, Packer is able to use tools like Continue Reading