Pandas data structure

pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

The two primary data structures of pandas, Series (1-dimensional) and DataFrame (2-dimensional), handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering. Pandas is built on top of NumPy and is intended to integrate well within a scientific computing environment with many other 3rd party libraries.

Series:
A Series is a one-dimensional object similar to an array, list, or column in a table. It will assign a labeled index to each item in the Series. By default, each item will receive an index label from 0 to N, where N is the length of the Series minus one.

# create a Series with an arbitrary list
s = pd.Series([7, 'Dhaka', 1.16, -1526, 'Happy City!'])
s
0                7
1            Dhaka
2             1.16
3            -1525
4      Happy City!
dtype: object

We can use dictionary as well, using the keys of the dictionary as its index.

d = {'Rajshahi': 100, 'Dhaka': 130, 'Dinajpur': 90, 'Rangpur': 110,
     'Natore': 45, 'Panchagarh': None}
cities = pd.Series(d)
cities
Dinajpur          90
Dhaka            130
Natore           45
Rajshahi         100
Rangpur          110
Panchagarh       NaN 
dtype: float64

You can use the index to select specific items from the Series …

cities['Dhaka']
130.0
cities[['Dhaka', 'Dinajpur', 'Panchagarh']]
Dhaka          130
Dinajpur       90
Panchagarh     NaN
dtype: float64

Or you can use boolean indexing for selection.

cities[cities < 1000]
Austin      450
Portland    900
dtype: float64

What if you aren’t sure whether an item is in the Series? You can check using idiomatic Python.

print('Munsiganj' in cities)
print('Dhaka' in cities)
False
True

DataFrame

A DataFrame is a tablular data structure comprised of rows and columns, akin to a spreadsheet, database table, or R’s data.frame object.

Reading Data

We can pass data as dictionary and a list as columns to a DataFrame to create data-frame.

data = {'year': [2010, 2011, 2012, 2011, 2012, 2010, 2011, 2012],
        'team': ['Bears', 'Bears', 'Bears', 'Packers', 'Packers', 'Lions', 'Lions', 'Lions'],
        'wins': [11, 8, 10, 15, 11, 6, 10, 4],
        'losses': [5, 8, 6, 1, 5, 10, 6, 12]}
football = pd.DataFrame(data, columns=['year', 'team', 'wins', 'losses'])
football
year team wins losses
0 2010 Bears 11 5
1 2011 Bears 8 8
2 2012 Bears 10 6
3 2011 Packers 15 1
4 2012 Packers 11 5
5 2010 Lions 6 10
6 2011 Lions 10 6
7 2012 Lions 4 12

CSV:
Reading a CSV is as simple as calling the read_csv function. By default, the read_csvfunction expects the column separator to be a comma, but you can change that using the sep parameter.

from_csv = pd.read_csv('mariano-rivera.csv')
from_csv.head()

We can create a list and also pass the list as a header:

cols = ['num', 'game', 'date', 'team', 'home_away', 'opponent',
        'result', 'quarter', 'distance', 'receiver', 'score_before',
        'score_after']
no_headers = pd.read_csv('peyton-passing-TDs-2012.csv', sep=',', header=None,
                         names=cols)
no_headers.head()

URL:
With read_table you can read data directly from URL:

user_cols = ['user_id', 'age', 'gender', 'occupation', 'zip_code']
users = pd.read_table('http://bit.ly/movieusers', sep='|', header=None, names=user_cols)
users.head()
user_id age gender occupation zip_code
0 1 24 M technician 85711
1 2 53 F other 94043
2 3 23 M writer 32067
3 4 24 M technician 43537
4 5 33 F other 15213

Here is full list IO documentation with file reading/writing functionality.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s