Studio-DevOps

KSnow: Load continuous data into Snowflake using Snowpipe

Reading Time: 5 minutes In this blog, we will discuss loading streaming data into Snowflake table using Snowpipe. But before that, if you haven’t read the previous part of this blog i.e., Loading Bulk Data into Snowflake then I would suggest you go through it. As now we have been set so let’s get started and see what Snowpipe is all about. Introduction Snowpipe is a mechanism provided by Continue Reading

Linking Logs to Grafana Panels using Loki

Reading Time: 4 minutes Hi folks, in this blog, we will be discussing what challenges we face when we want to view logs with respect to metrics and how we can achieve this using Grafana, Loki, and Promtail. Problem Statement When using monitoring solutions like Grafana, Prometheus, Elatic Stack etc for our infrastructure, we have metrics and logs decoupled to an extent that it is very difficult to look Continue Reading

How to use external kafka from kubernetes ?

Reading Time: 3 minutes Kafka is very widely used messaging infrastructure. If you are building software that require using kafka from your kubernetes cluster, you can either use strimzi.io kafka operator or use your local or aws based kafka cluster. While strimzi installation is a breeze and easy to use, one particular challenge is that it is cumbersome to access the kafka topics from local. In this simple blog, Continue Reading

KSnow: Loading Data Into Snowflake

Reading Time: 5 minutes This blog pertains to Loading Data into Snowflake, and I will explain you about the various step involved in this process. So let’s get started. Before moving ahead, you can visit the blog on understanding the basic of Snowflake Data Warehouse in case you want to refresh your concepts. Now let’s talk about the actual topic for which you have click on this blog. To Continue Reading

Git first cover

Git useful commands, code management

Reading Time: 6 minutes Hi all. In this blog, we will continue to explore some of the useful git commands. These commands would help when we are working on a remote shared repository. This is in continuation to my previous post on git working areas. So, I am assuming that all are familiar with git working areas, if not please go through the blog here. Let’s explore them now Continue Reading

Setting up Verified Commits on GitHub

Reading Time: 3 minutes In the wake of the COVID-19 pandemic almost all IT organizations are experimenting with allowing their employees to work from home. But at the same time are concerned about the security aspects that come with it. One aspect of security is authorization with respect to version control and the case of stolen identity. The Problem Git keeps track of several pieces of information including, but Continue Reading

Migrating MLFlow Server To Cloud: Part 2

Reading Time: 4 minutes In my previous blog, I had discussed the first two phases of migrating MLFlow server to cloud. In this blog, I’ll be discussing the deployment of MLflow tracking server on Google Cloud Platform and migration of the existing data to the process. Also, I’ll be talking about optimizing the overall environment in the process. Deployment Step 1: Copy Contents from Disk Go to this link Continue Reading

Migrating MLFlow Server To Cloud: Part 1

Reading Time: 4 minutes The cloud migration process involves moving all or part of an organization’s data, apps, and services from on-premises data centres to a public or private cloud, where they are accessible on-demand over the Internet to authorized users. For most businesses considering cloud migration, the move is filled with promise and potential; scalability, flexibility, reliability, cost-effectiveness, improved performance and disaster recovery, and simpler, faster deployment. Cloud Continue Reading

docker

Orchestration With Docker-Swarm

Reading Time: 6 minutes The world is constantly being attracted towards containers and its orchestration especially in large and dynamic environments which has also led to the rising of various tools and platforms in trend.Yes, we will be looking at Docker-Swarm and some of its key components !!! As we all know that Docker is a major player in the world of containers and therefore in this blog, we Continue Reading

Monitoring application using Prometheus and grafana

Reading Time: 5 minutes Hello readers, In this blog I will discuss about application and system monitoring using Prometheus and Grafana. we need something to monitor! Part of that is going to be our Ubuntu host and the application will be a simple web page host program called sample-App that uses the Express web framework to do most of the hard work for us. we are going to download Continue Reading