Data wrangling

What is data wrangling?

Data wrangling is the hands-on work of turning messy raw data into a clean, structured form you can actually analyse. You parse it, fix the errors, reshape it, join it with other sources, and hand off something a report or a model can use. It also goes by the name data munging.

Most raw data does not arrive ready to use. A sales export has dates in three formats, a survey has half-empty free-text fields, two systems spell the same customer two different ways. Wrangling is the effort that closes the gap between that raw dump and a table you can trust.

It is famously the part that takes the day. Analysts and data scientists routinely say the cleaning and shaping takes far more of their time than the analysis that follows.

The main steps

Wrangling is not one action but a loop you repeat until the data holds up. A common way to break it down:

1. Discovery. Look at what you actually have. What is in each column, how much is missing, where the obvious problems sit. This overlaps with data profiling.

2. Structuring. Reshape the data into the layout your analysis needs: splitting one column into several, pivoting rows into columns, or the reverse.

3. Cleaning. Fix what is wrong. Standardise date and number formats, correct typos, handle empty fields, remove duplicate records, deal with outliers.

4. Enriching. Join in extra data that adds context, such as a region lookup or a currency rate, so the table answers more questions.

5. Validating. Run checks to confirm the result is consistent: totals reconcile, every order has a valid customer, no dates land in the future.

These steps rarely run cleanly once. Validation turns up a problem you missed, and you drop back into cleaning.

A worked example

Say you get a spreadsheet of orders from a webshop export. The order date column mixes 03/04/2026 and 2026-04-03. Customer names appear as "ACME nv", "Acme NV" and "acme". Three rows are exact duplicates from a double submit. The country column is blank for older orders.

Wrangling this means parsing both date formats into one, collapsing the three spellings of Acme into a single customer, dropping the duplicate rows, and filling the missing country from the postcode. What comes out is one clean order table where a total by customer or by month is finally correct.

Data wrangling versus data transformation

The two overlap and people use them loosely, but there is a difference in spirit. Data transformation is the broad category of changing data from one shape or format to another, and it covers automated, repeatable pipeline steps as much as manual work.

Data wrangling is the more exploratory, hands-on end of that. It is what you do when you first meet a messy dataset and have to work out interactively what is wrong and how to fix it. Once you have figured it out and the same feed arrives every week, that logic usually gets baked into an ETL or ELT pipeline so nobody has to wrangle it by hand again.

Tools people reach for

At the small end, plenty of wrangling happens in a spreadsheet. It works until the steps have to be repeated or audited, at which point manual edits become a liability.

Power Query, built into Excel and Power BI, records each step you take so the whole cleanup replays automatically when new data lands. It is the common choice in the Microsoft world for repeatable wrangling without code.

pandas, the Python library, is the tool of choice for analysts and data scientists who want full control in code. It handles parsing, joining, reshaping and cleaning across datasets far larger than a spreadsheet can hold.

The line to remember: a spreadsheet is fine for a one-off, but repeatable work belongs in a tool that records the steps.

What to watch out for with data wrangling

Undocumented manual edits. Fixing values by hand in a spreadsheet leaves no trace of what changed or why. When the numbers are questioned later, you cannot explain them. Prefer steps that are recorded and can be replayed.

Wrangling the same feed forever. If you clean the same export by hand every week, that is a pipeline waiting to be built. Manual wrangling is for exploration, not for a standing process.

Silent assumptions. Dropping rows with missing values or forcing a date format can quietly change the answer. Write down the choices you make while cleaning, because they are analysis decisions, not just tidying.