Where to find free public data, and how to check it before you trust it

A great deal of useful data is free: economic indicators, population, health, energy, transport. Finding it is the easy part. Knowing whether to trust it is where analyses go wrong.

Where public data lives

• International organisations publish indicators by country and year, for example the World Bank’s open data and Our World in Data.
• Government open-data portals publish datasets from agencies and cities. Many run on common platforms such as CKAN or Socrata, which is why their datasets look alike and can be searched in the same way.
• National statistical agencies publish the official figures for their country, often as downloadable tables.
• Research repositories and code-hosting sites hold datasets that accompany papers and projects, often as plain CSV files.

Five checks before you use a dataset

1. Who published it? An agency or organisation that collects the data is better than a copy of a copy.
2. When was it last updated, and what period does it actually cover? A dataset can be downloaded today and describe a period years ago.
3. What are the units and definitions? Percent or per person, current prices or adjusted for inflation, calendar year or fiscal year.
4. What does it include? Check for rows that are totals (such as "World" or "All regions") sitting alongside the countries, which will double-count if you sum.
5. How are gaps recorded? Some files use blank cells, others use 0 or a code like -999 for "missing". A zero that means "unknown" ruins an average.

Traps that quietly spoil an analysis

• Mixing countries that report in different years, which makes a panel look like it jumps when only the reporting changed.
• Comparing a figure that was revised with one that was not.
• Assuming a column called "value" means what you hope. Read the description.
• Using a download without noting where and when you got it, so nobody can check it later.

Using public data in AnalyzeIt

Paste a link to a data file or ask for public data on a topic. AnalyzeIt looks for the data, downloads a table, reshapes it where needed (for example, years laid out as columns), sets aside total rows, and runs the same discovery checks as on your own data. Every answer lists its source, the date it was downloaded, the newest period in the data, and a plain note that the data comes from the internet and has not been checked by AnalyzeIt. If nothing usable is found, it says what it tried and why each source failed. It follows each site’s robots.txt, and it never reaches internal or private addresses.

Read more: How online data works, or try the live demo with no account.