Data Analysis using Python: Pandas

Table of contents
Reading Time: 3 minutes

In this blog, I am going to explain pandas which is an open source library for data manipulation, analysis, and cleaning.

Pandas is a high-level data manipulation tool developed by Wes McKinney. The name Pandas is derived from the word Panel Data – an Econometrics from Multidimensional data. Pandas is built on the top of NumPy.

Five typical steps in the processing and analysis of data, regardless of the origin of data are load, prepare, manipulate, model, and analyze.

You can install pandas using pip or conda:

pip install pandas

or

conda install pandas

Start using pandas by importing it as:

import pandas as pd

Pandas deals with the following three data structures :

  • Series
  • DataFrame
  • Panel
Series

Series is a value mutable, size immutable one-dimensional array like structure with homogeneous data.

  • To create an empty series:

pd.Series()
Series([], dtype: float64)

  • To create a series with default indexes:

import numpy as np
data = np.array([‘a’,’b’,’c’,’d’])
series = pd.Series(data)
print(series)
0 a
1 b
2 c
3 d
dtype: object

  • To create a series with custom indexes:

pd.Series(data,index=[10,11,12,13])

print(series)
10 a
11 b
12 c
13 d
dtype: object

  • To create a series using a dictionary:

data = {‘a‘ : 0., ‘b’ : 1., ‘c’ : 2.}
s
eries = pd.Series(data)
print(series)
a 1.0
c 2.0
d 0.0
dtype: float64

  • To create a series using a dictionary with custom indexes. Here, Index order is persisted and the missing element is filled with NaN:

data = {‘a’ : 0., ‘b’ : 1., ‘c’ : 2.}
series = pd.Series(data,index=[‘b’,’c’,’d’,’a’])
print(series)
b 1.0
c 2.0
d NaN
a 0.0
dtype: float64

  • To access a single element of series:

print(series[0])

1.0

  • To access multiple elements of the series:

print(series[[0,1]])

b 1.0

c 2.0

dtype: float64

DataFrame

DataFrames allows to store and manipulate tabular data in rows of observations and columns of variables. It is a size mutable, data mutable two-dimensional array with heterogeneous data. For example, an employee management system column can be employeeName and row can be names of employees. Pandas DataFrame can be created by loading the datasets from existing storage like SQL Database, CSV file, and Excel file. Pandas DataFrame can also be created from the lists, dictionary, and from a list of dictionary etc.

  • To create a DataFrame from csv:

 df = pd.read_csv(file)

  • To create a DataFrame from the database:

 df = pd.read_sql_query(query,conn)

  • To create a DataFrame from excel:

 df = pd.read_excel(file,sheetname=’ ‘)

  • To create an empty DataFrame: FROM EXCEL:

print(pd.DataFrame())
Empty DataFrame
Columns: []
Index: []

  • To create a DataFrame from the list:

print(pd.DataFrame([1,2]))

  0

0 1

1 2

  • To create a DataFrame with the given column names:

data = [[‘Joe’,10],[‘Bob’,12]]
print(pd.DataFrame(data,columns=[‘Name’,’Age’],dtype=float))
 Name   Age
0  Joe     10.0
1  Bob    12.0

  • To filter the DataFrame:

df.loc[df[‘Name’] == ‘Joe’]
  Name Age
0   Joe    10.0

  • To create DataFrame from the Series:

print(pd.DataFrame( {‘one’ : pd.Series([1, 2, 3], index=[‘a’, ‘b’, ‘c’]), ‘two’ : pd.Series([1, 2, 3, 4], index=[‘a’, ‘b’, ‘c’, ‘d’])}))

   one  two
a  1.0    1
b  2.0    2
c  3.0    3
d  NaN    4

A lot of operations can be performed on DataFrames like filtering, adding or deleting new rows and columns. Also, many functions and properties are used with DataFrames like – isnull(), head(), tail(), empty, axes, values, size, transpose.

Panel

The panel is a 3D container of data. The names for the 3 axes:

  • items: axis 0, each item corresponds to a DataFrame contained inside.
  • major_axis: axis 1, it is the rows of each of the DataFrames.
  • minor_axis: axis 2, it is the columns of each of the DataFrames.

print(pd.Panel())

<class ‘pandas.core.panel.Panel’>

Dimensions: 0 (items) x 0 (major_axis) x 0 (minor_axis)

Items axis: None

Major_axis axis: None

Minor_axis axis: None

Thanks for reading!

blog-footer

Written by 

Jyoti Sachdeva is a software consultant with more than 6 months of experience. She likes to keep up with the trending technologies. She is familiar with languages such as C,C++,Java,Scala and is currentky working on akka,akka http and scala. Her hobbies include watching tv series and movies, reading novels and dancing.