Big Data Analytics: An Introduction

DATA ANALYTICS

Data can help businesses better understand their customers and improve their advertising campaigns. It can also help personalise their content, and improve their bottom lines. The advantages of data are many, but you can’t access these benefits without the proper data analytics tools and processes. While raw data has a lot of potentials, you need data analytics to unlock the power to grow your business.

WHAT IS DATA ?

Data are individual facts, statistics, or items of information, often numeric, that is collected through observations. In a more technical sense, data are a set of values of qualitative or quantitative variables about one or more persons or objects.

JOURNEY OF DATA

INCREASE IN DATA SIZE

• The world’s technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s; as of 2012, every day 2.5 exabytes (2.5×10¹⁸) of data is generating. 

• Based on an IDC report prediction, the global data volume will grow exponentially from 4.4 zettabytes to 44 zettabytes between 2013 and 2022

• By 2025, IDC predicts there will be 163 zettabytes of data

EXAMPLES

• Various formats, types, and structures
• Text, numerical, images, audio, video, sequences, time series, social media data, multi-dim arrays, etc.
• Static data vs. streaming data  
• A single application can be generating/collecting many types of data  

CHARACTERISTICS

“5Vs” of Big Data which are also termed as the characteristics of Big Data are as follows

• Volume 
  The volume of data refers to the size of data sets that an organisation has gather for analysis and processing . In today’s technology, these data sets are frequently seen pushing on the larger size of bytes, such as terabytes and petabytes.
  
• Variety
  Variety refers to all the structured and unstructured data that has the possibility of getting generated either by humans or by machines. The most commonly added data are structured -texts, tweets, pictures & videos. However, unstructured data like emails, voicemails, hand-written text, ECG reading, audio recordings, etc, are also important elements under Variety. Variety is all about the ability to classify incoming data into various categories.
  
• Velocity
  With Velocity we refer to the speed with which data are being generating. Staying with our social media example, every day 900 million photos are uploaded on Facebook, 500 million tweets on Twitter everyday, 0.4 million hours of video are uploaded on Youtube and 3.5 billion searches are performed in Google. This is like a nuclear data explosion. Big Data helps the company to hold this explosion, accept the incoming flow of data and at the same time process it fast so that it does not create bottlenecks.
  
• Variablity
  Variability refers to data whose meaning is constantly changing. Many a time, organizations need to develop sophisticated programs to be able to understand the context in them and decode their exact meaning.
  
• Veracity
  Data veracity, in general, is how accurate or truthful a data set may be. In the context of big data, however, it takes on a bit more meaning. More specifically, when it comes to the accuracy of data, it’s not just the quality of the data itself but how trustworthy the data source, type, and processing it is.
  

ARCHITECTURES & TOOLS

• NO SQL

NoSQL is a better choice for businesses whose data workloads are more geared toward the rapid processing and analyzing of vast amounts of varied and unstructured data.

• MAP REDUCE

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster 

• HADOOP

Hadoop stores and processes the data in a distributed manner across the cluster of commodity hardware.

• HBASE

HBase is a data model similar to Google’s big table that is design to provide random access to the high volume of structured or unstructured data.

• YARN

It is an Apache Hadoop technology, stands for Yet Another Resource Negotiator. YARN is a large-scale, distributed operating system for big data applications.

• ZOOKEEPER

ZooKeeper is an open-source Apache project that provides a centralized service for providing configuration information, naming, synchronization, and group services over large clusters in distributed systems.

• FLUME

Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.

• OOZIE

Apache Oozie is a Java Web application use to schedule Apache Hadoop jobs. Oozie combines multiple jobs sequentially into one logical unit of work.

• PIG

Pig is a high-level platform or tool which is use to process large datasets. It provides a high level of abstraction for processing over MapReduce.

• HIVE

Hive is an easy-to-use software application that lets one analyse large-scale data through the batch processing technique. 

BIG DATA – DOMAINS

• Customer segmentation, managing customer relationships, and offering more customer-centric products
• managing stocks and predicting needs in products
• optimizing resource utilization and reducing costs
• fraud detection
• identifying and removing performance bottlenecks proactively
• predicting equipment failures
• launching new services
• mitigating risks
• identifying the causes of failures and problems in real-time

CONCLUSION

The availability of Big Data, low-cost commodity hardware, and new information management and analytic software have produced a unique moment in the history of data analysis. The convergence of these trends means that we have the capabilities required to analyse astonishing data sets quickly and cost-effectively for the first time in history. These capabilities are neither theoretical nor trivial. They represent a genuine leap forward and a clear opportunity to realise enormous gains in terms of efficiency, productivity, revenue, and profitability. To dig deep, you can read this blog.