Python for Data Analysis With pandas: A Beginner's Guide

To start Python for data analysis, install Python with Jupyter, learn the pandas library, and practise five operations until they're automatic: loading a file, inspecting it, cleaning it, summarising it with groupby, and combining tables with merge. Add simple charts with Matplotlib and you can already redo most Excel or SQL analysis in Python. You don't need to become a software developer first; a few weeks of core Python is enough to begin.

I'm Gulab Chand Tejwani. I teach Python, SQL and data analysis at TechNet Consultancy in Vallabh Vidyanagar, Anand, and online, and I hold the Microsoft PL-300 (Power BI Data Analyst Associate) and Microsoft Certified Trainer credentials. This guide follows the order I use with beginners. It assumes you've used Excel and, ideally, a little SQL. If you haven't, my data analyst roadmap explains why I put Python after those two.

Why use Python if I already know Excel?

Three reasons come up again and again with my learners.

Size: Excel slows down badly with a few hundred thousand rows and has a hard limit of 1,048,576 rows per sheet. pandas handles millions of rows on an ordinary laptop.

Repeatability: if you clean the same monthly file every month, a Python script does it in seconds and does it the same way each time.

Reach: Python connects to databases, APIs, web pages and dozens of file formats, and it leads into statistics and machine learning if you want to go there later.

Excel is still faster for a quick look at a small file. Use both.

	Excel	pandas
Row limit	1,048,576 rows per sheet	Limited by your computer's memory
Repeating the same cleaning	Manual, or Power Query	Re-run the script
Quick look at a small file	Fastest	Slower to start
Sharing with a non-technical colleague	Easy	Export to Excel or CSV first
Connecting to APIs and many file types	Limited	Wide library support

How do I set up Python for data analysis?

The simplest route for beginners is the Anaconda distribution, which installs Python, Jupyter and pandas together. If you prefer a lighter setup, install Python from python.org and then run:

pip install pandas matplotlib jupyter openpyxl

openpyxl lets pandas read and write .xlsx files. Start Jupyter with jupyter notebook (or use VS Code, which opens notebooks too). Google Colab works in the browser with nothing to install, which is handy on a college computer.

Every code example in this guide was run end to end on a small practice dataset with Python 3.14 and pandas 3.1. pandas 2.x runs the same code; the main visible difference is that info() shows text columns as object rather than str.

How much core Python do I need first?

Less than you'd think. Before pandas, be comfortable with:

• Variables, numbers and strings
• Lists and dictionaries
• for loops and if statements
• Writing a small function with def
• Importing a library

A week or two of practice is usually enough. You'll pick up the rest as you go.

Loading data

import pandas as pd

sales = pd.read_csv("sales.csv")
# or
sales = pd.read_excel("sales.xlsx", sheet_name="Data")

A DataFrame is pandas' table: rows, named columns, and a type for each column. Most of your work is filtering, changing and summarising DataFrames.

Indian exports often have dates as day-month-year. Tell pandas:

sales = pd.read_csv("sales.csv", parse_dates=["order_date"], dayfirst=True)

First look at the data

Never analyse a file you haven't inspected. These five lines go at the top of almost every notebook I write:

sales.shape          # (rows, columns)
sales.head()         # first five rows
sales.info()         # column names, types, non-null counts
sales.describe()     # summary stats for numeric columns
sales.isna().sum()   # missing values per column

info() is where you catch problems: an amount column stored as text (object), a date column that didn't parse, or far fewer non-null values than rows.

What does info() actually tell you?

Here's the kind of output you'll see, and what to look for:

RangeIndex: 4381 entries, 0 to 4380
Data columns (total 6 columns):
 #   Column      Non-Null Count  Dtype
 0   order_id    4381 non-null   int64
 1   order_date  4381 non-null   str
 2   city        4375 non-null   str
 3   product_id  4381 non-null   int64
 4   quantity    4381 non-null   int64
 5   amount      4381 non-null   str

Three problems jump out. order_date is str, so it's text, not a date; the parse failed or wasn't requested. (pandas 2.x shows text columns as object instead of str; the meaning is the same.) amount is also text, which usually means a stray comma, currency symbol or "N/A" somewhere in the column. And city has six missing values. All three need fixing before any total you calculate can be trusted.

Selecting and filtering

sales["city"]                              # one column
sales[["city", "amount"]]                  # several columns
sales[sales["amount"] > 10000]             # rows where amount > 10,000
sales[(sales["city"] == "Anand") & (sales["amount"] > 10000)]
sales[sales["city"].isin(["Anand", "Nadiad"])]

Use & and | (not and and or), and put each condition in brackets. Forgetting the brackets is the first error almost everyone hits.

If a numeric filter fails with "'>' not supported between instances of 'str' and 'int'", the column was read as text because some values aren't numbers. info() shows it as str (or object in pandas 2.x). Convert it first with pd.to_numeric(..., errors="coerce"), as in the cleaning step below.

Cleaning

Common cleaning steps look like this:

sales["city"] = sales["city"].str.strip().str.title()
sales = sales.drop_duplicates()

# Remove ₹ signs, commas and spaces first, so "₹1,000" becomes 1000
# instead of being thrown away as unreadable.
raw_amount = sales["amount"].astype(str)
sales["amount"] = pd.to_numeric(
    raw_amount.str.replace(r"[₹,\s]", "", regex=True), errors="coerce"
)
print(raw_amount[sales["amount"].isna()])  # look at what still failed
sales = sales.dropna(subset=["amount"])

sales["month"] = sales["order_date"].dt.to_period("M")

That trims spaces and fixes case in city names and drops exact duplicate rows. For amount, it strips rupee signs, commas and spaces before converting to a number, so formatted values like ₹1,000 survive. It then prints the values that still couldn't be read, so you can see what you're about to drop, removes those rows, and adds a month column.

Be careful with dropna() and fillna(). Before you delete or fill missing values, find out why they're missing. Sometimes a blank means zero, sometimes it means "not recorded", and treating them the same can change your answer.

Summarising with groupby

groupby is pandas' version of a PivotTable or SQL's GROUP BY:

sales.groupby("city")["amount"].sum().sort_values(ascending=False)

sales.groupby(["city", "month"]).agg(
    orders=("order_id", "count"),
    revenue=("amount", "sum"),
    avg_order=("amount", "mean"),
).reset_index()

If you've written SQL, this reads the same way: group by city and month, then count, sum and average. pivot_table gives you the spreadsheet-style cross tab:

sales.pivot_table(index="city", columns="month", values="amount", aggfunc="sum")

Combining tables with merge

merge is pandas' join:

products = pd.read_csv("products.csv")
full = sales.merge(products, on="product_id", how="left")

how="left" keeps every sale even if the product is missing from the products file, which is the same as a SQL LEFT JOIN. The same warning applies too: if product_id isn't unique in products, your sales rows will multiply. Check len(sales) before and len(full) after. The validate="many_to_one" argument makes pandas raise an error if the right side has duplicates, which is a nice safety net.

Making a quick chart

import matplotlib.pyplot as plt

monthly = sales.groupby("month")["amount"].sum()
monthly.plot(kind="line", marker="o", title="Revenue by month")
plt.ylabel("Revenue (₹)")
plt.tight_layout()
plt.show()

For analysis, charts in a notebook are for you and your team. For managers, a Power BI or Excel dashboard is often easier to share. Seaborn, built on Matplotlib, makes nicer statistical charts with less code once you're comfortable.

Saving your results

city_summary = sales.groupby("city")["amount"].sum().reset_index()
city_summary.to_excel("city_summary.xlsx", index=False)
full.to_csv("clean_sales.csv", index=False)

Many real Python jobs for analysts end exactly here: clean the data, summarise it, and hand an Excel file or a clean CSV to someone else, or load it into Power BI.

A first mini project you can do this week

Download a dataset you understand. India's Open Government Data platform (data.gov.in) has many CSV files, and Kaggle has retail and sales datasets. Then, in one notebook:

1. Load it and run the five inspection lines
2. Write down three questions you want to answer
3. Clean what's needed, with a markdown note explaining each step
4. Answer each question with groupby or pivot_table
5. Make one chart per question
6. Finish with three sentences: what you found and what you'd check next

Put it on GitHub. That notebook, with clear notes, is a portfolio piece. A short write-up of the problems you hit is worth as much as the code.

A worked mini example, start to finish

Here's a complete small analysis you can run on any sales CSV with order_date, city and amount columns. Growth needs orders from both halves of the year; cities with only one half show NaN instead of a growth figure:

import pandas as pd

sales = pd.read_csv("sales.csv", parse_dates=["order_date"], dayfirst=True)

# Clean
sales["city"] = sales["city"].str.strip().str.title()
sales["amount"] = pd.to_numeric(
    sales["amount"].astype(str).str.replace(r"[₹,\s]", "", regex=True),
    errors="coerce",
)
print("Rows with unreadable amounts:", sales["amount"].isna().sum())
sales = sales.dropna(subset=["amount"]).drop_duplicates()

# Question: which three cities grew most from H1 to H2?
sales["half"] = sales["order_date"].dt.month.map(lambda m: "H1" if m <= 6 else "H2")
by_half = sales.pivot_table(index="city", columns="half", values="amount", aggfunc="sum")
# Keep both halves even if the file covers only one (missing half = NaN)
by_half = by_half.reindex(columns=["H1", "H2"])
by_half["growth_pct"] = (by_half["H2"] / by_half["H1"] - 1) * 100
print(by_half.sort_values("growth_pct", ascending=False).head(3).round(1))

Notice the print line that reports unreadable amounts before they're dropped. Never drop rows silently; say how many and why, in the notebook.

Common beginner mistakes

• Analysing before inspecting: always run info() first
• Chaining filters and assignments in a way that triggers SettingWithCopyWarning; use .loc[row_condition, "column"] = value to change values
• Forgetting that groupby drops rows with NaN keys by default (pass dropna=False if you want them)
• Writing loops over rows; pandas operations on whole columns are simpler and much faster
• Not restarting and running the notebook top to bottom before sharing it, so it only works because of cells you ran out of order

Where does pandas fit with SQL and Power BI?

In most analyst jobs I've seen, SQL pulls the data, Python cleans or reshapes it when SQL gets awkward, and Power BI or Excel presents it. You don't have to choose one. For a deeper look at the SQL side, see 15 SQL queries for data analysis, whose patterns map almost one to one onto pandas.

Our Python training teaches core Python and pandas with this kind of hands-on work, in the Vallabh Vidyanagar, Anand classroom or live online. It is also part of the full data analysis course. If you're preparing for interviews, the Python interview preparation page is a good next stop.

If you want the basics first, read what is data analysis: meaning, uses, types of data and a worked example.

FAQ

Is pandas hard to learn for a beginner?

Not if you already know Excel or a little SQL, because the ideas are the same: filter rows, pick columns, group and summarise, join tables. Most learners are comfortable with the basics in four to six weeks of regular practice.

Do I need to learn NumPy before pandas?

No. pandas is built on NumPy, and you'll pick up the NumPy you need along the way. Start with pandas directly for data analysis.

Should I use Jupyter Notebook or VS Code?

Either is fine. Jupyter (or Google Colab) is easiest for beginners because you see results right below each cell. VS Code can open the same notebooks and is better once you start writing reusable scripts.

Can pandas replace Excel?

For large files, repeatable cleaning and automation, yes. For a quick look at a small file or sharing a simple sheet with a colleague, Excel is often quicker. Most analysts use both.

Is Python required for a data analyst job?

Not always. Many analyst roles in India ask mainly for Excel, SQL and Power BI, and list Python as a plus. Python becomes more important as you move towards larger data, automation or data science.