# Pandas (software)
**pandas** is an open-source Python library providing fast, flexible data structures, principally the one-dimensional `Series` and the two-dimensional `DataFrame`, for working with relational or labeled data in practical data analysis.[1](https://pandas.pydata.org/docs/getting_started/overview.html) Pandas uses NumPy as its foundation and was designed to work smoothly alongside numerous other third-party libraries in scientific computing settings.[1](https://pandas.pydata.org/docs/getting_started/overview.html) Development began in 2008 at [AQR Capital Management](https://www.edgepedia.org/aqr-capital-management), the library was open sourced by the end of 2009, and since 2015 it has been a NumFOCUS sponsored project.[2](https://pandas.pydata.org/about/index.html)
| Key fact | Detail |
|---|---|
| Creator and origin | Development began in 2008 at AQR Capital Management under Wes McKinney; open sourced by the end of 2009[2](https://pandas.pydata.org/about/index.html) |
| Core data structures | `Series` (1-dimensional labeled array) and `DataFrame` (2-dimensional, size-mutable tabular structure with potentially heterogeneous columns)[1](https://pandas.pydata.org/docs/getting_started/overview.html) |
| Foundation | Built on NumPy; critical code paths written in Cython or C[1](https://pandas.pydata.org/docs/getting_started/overview.html) • [2](https://pandas.pydata.org/about/index.html) |
| Current release | pandas 3.0.6 (September 17, 2026), compatible with Python 3.15[3](https://pandas.pydata.org/pandas-docs/stable/release.html?highlight=option_context) |
| pandas 3.0 changes | Dedicated string data type enabled by default (PyArrow-backed when installed) and Copy-on-Write behavior[4](https://pandas.pydata.org/docs/whatsnew/v3.0.0.html) |
| Governance | BSD-licensed project fiscally sponsored by the 501(c)(3) NumFOCUS Foundation; Wes McKinney serves as BDFL[5](https://pandas.pydata.org/about/governance.html) |
| Scale limit | Designed for in-memory analytics; datasets larger than memory require chunking or external libraries[6](https://pandas.pydata.org/docs/user_guide/scale.html?highlight=chunksize) |
## History
[Wes McKinney](https://www.edgepedia.org/wes-mckinney) started pandas in 2008 while at AQR Capital Management, a quantitative investment firm, and presented the library to the research community in a 2010 SciPy conference paper, "Data Structures for Statistical Computing in Python," which described pandas as a new library aiming to facilitate working with data sets common to finance, statistics, and related fields.[2](https://pandas.pydata.org/about/index.html) • [7](https://dblp.org/rec/conf/scipy/McKinney10) The project timeline records the first edition of *Python for Data Analysis* in 2012, NumFOCUS sponsorship in 2015, and the first in-person core developer sprint in 2018.[2](https://pandas.pydata.org/about/index.html)
Governance is formalized around a BDFL (Benevolent Dictator for Life), currently Wes McKinney, and a Core Team of contributors whose work has been substantial and sustained over at least a year; decisions are generally made by consensus, and the BDFL rarely overrides.[5](https://pandas.pydata.org/about/governance.html) The project is released under a BSD-style license and hosted in public GitHub repositories, with NumFOCUS as its parent legal entity and fiscal sponsor.[5](https://pandas.pydata.org/about/governance.html)
## Data structures
The two primary structures handle the vast majority of typical use cases in finance, statistics, social science, and many areas of engineering.[1](https://pandas.pydata.org/docs/getting_started/overview.html)
- **Series** is a one-dimensional labeled array capable of holding any data type, including integers, strings, floating point numbers, and Python objects; its axis labels are collectively called the index, and like a NumPy array it has a single dtype.[8](https://pandas.pydata.org/docs/user_guide/dsintro.html?highlight=dic)
- **DataFrame** is a general 2D labeled, size-mutable tabular structure with potentially heterogeneously-typed columns; conceptually it is a container for Series objects, just as a Series is a container for scalars.[1](https://pandas.pydata.org/docs/getting_started/overview.html)
Data alignment is intrinsic: the link between labels and data is not broken unless done so explicitly, and objects can automatically align on labels in computations.[8](https://pandas.pydata.org/docs/user_guide/dsintro.html?highlight=dic) • [1](https://pandas.pydata.org/docs/getting_started/overview.html) The library suits tabular data with heterogeneously-typed columns, as in an SQL table or Excel spreadsheet; ordered and unordered time series; arbitrary matrix data with row and column labels; and observational data sets that need not be labeled at all.[1](https://pandas.pydata.org/docs/getting_started/overview.html)
## Features
The project's own highlights include tools for reading and writing data between in-memory structures and formats such as CSV and text files, [Microsoft Excel](https://www.edgepedia.org/microsoft-excel), SQL databases, and the HDF5 format; intelligent label-based alignment and missing-data handling; flexible reshaping and pivoting; a group-by engine for split-apply-combine operations; high-performance merging and joining; hierarchical axis indexing for higher-dimensional data in lower-dimensional structures; and time series functionality including date range generation, frequency conversion, moving window statistics, and date shifting.[2](https://pandas.pydata.org/about/index.html) [Missing data](https://www.edgepedia.org/missing-data) is represented as NaN in floating point as well as non-floating point data, and columns can be inserted and deleted from DataFrames.[1](https://pandas.pydata.org/docs/getting_started/overview.html)
Performance comes partly from implementation: many low-level algorithmic components have been extensively tweaked in Cython code, and critical code paths are written in Cython or C.[1](https://pandas.pydata.org/docs/getting_started/overview.html) • [2](https://pandas.pydata.org/about/index.html) The documentation notes the usual trade-off that generalization can sacrifice performance, so a specialized tool focused on one feature may be faster.[1](https://pandas.pydata.org/docs/getting_started/overview.html)
## pandas 3.0
Version 3.0.0 was released on January 21, 2026, removing functionality deprecated in earlier releases; the recommended upgrade path is to first move to pandas 2.3 and eliminate warnings.[4](https://pandas.pydata.org/docs/whatsnew/v3.0.0.html) Two changes define the release. First, a dedicated string data type is enabled by default: historically string columns used NumPy `object` dtype, which is not specific to strings and is often inefficient in performance and memory; from 3.0, string data is inferred as the new `str` dtype, backed by PyArrow when that package is installed and falling back to NumPy `object` otherwise.[4](https://pandas.pydata.org/docs/whatsnew/v3.0.0.html) Second, Copy-on-Write becomes the standard behavior: any indexing operation or method returning a new DataFrame or Series behaves as a copy, so modifying an object requires modifying it directly, chained assignment stops working, and the `SettingWithCopyWarning` is removed; under the hood, views are used as much as possible and copies are made only when needed.[4](https://pandas.pydata.org/docs/whatsnew/v3.0.0.html)
The 3.0 series has progressed through maintenance releases: 3.0.5 on July 22, 2026, and 3.0.6 on September 17, 2026, the latter adding compatibility with Python 3.15.[3](https://pandas.pydata.org/pandas-docs/stable/release.html?highlight=option_context) One intermediate release, 3.0.4 (June 28, 2026), was yanked from PyPI over reported segfaults with datetime-related functionality.[9](https://pypi.org/project/pandas/)
## Scaling and performance in context
pandas provides data structures for in-memory analytics, which makes analyzing datasets larger than memory difficult; even datasets that are a sizable fraction of memory become unwieldy because some operations make intermediate copies.[6](https://pandas.pydata.org/docs/user_guide/scale.html?highlight=chunksize) The documentation recommends loading less data, using efficient datatypes, chunking with readers such as `pandas.read_csv()`, and switching to other libraries with pandas-like APIs for parallel runtimes, distributed memory, or clustering; operations such as `groupby` are much harder to do chunkwise.[6](https://pandas.pydata.org/docs/user_guide/scale.html?highlight=chunksize)
Newer dataframe libraries have changed the performance landscape. A 2025 EDBT evaluation of dataframe libraries on a single machine found that for small datasets pandas consistently proved the best choice with the richest API; when data fits in RAM and full pandas API compatibility is not needed, Polars is the go-to choice thanks to in-memory execution and query optimization; cuDF often performs best when a GPU is available, and PySpark for very large datasets that fit neither GPU memory nor RAM.[10](https://arxiv.org/abs/2312.11122) Polars adheres to the [Apache Arrow](https://www.edgepedia.org/apache-arrow) memory format while pandas uses NumPy arrays, a difference the Polars migration guide highlights for users coming from pandas.[11](https://pola-rs.github.io/polars-book/user-guide/migration/pandas/) Empirical comparisons of pandas and Polars across TPC-H-derived and synthetic operations at dataframe sizes from 2 million to 60 million rows report that Polars performs better than pandas in terms of speed for the examined operations.[12](https://research.vu.nl/ws/portalfiles/portal/361654082/An_Empirical_Study_on_the_Energy_Usage_and_Performance_of_Pandas_and_Polars_Data_Analysis_Python_Libraries.pdf) Within pandas itself, the PyArrow-backed string dtype introduced in 3.0 targets the same pressure, offering large improvements in memory and time for certain operations compared with `object` dtype arrays.[13](https://pandas.pydata.org/docs/user_guide/text.html?highlight=startswith)
## Usage and ecosystem
pandas is a dependency of statsmodels, making it an important part of the Python statistical computing ecosystem, and it has been used extensively in production in financial applications.[1](https://pandas.pydata.org/docs/getting_started/overview.html) The project reports use across academic and commercial domains including finance, neuroscience, economics, statistics, advertising, and [web analytics](https://www.edgepedia.org/web-analytics).[2](https://pandas.pydata.org/about/index.html) A 2025 survey-style paper describes pandas as the de facto library for data processing and machine learning, widely used for data loading, transformation, and analysis.[14](https://arxiv.org/abs/2608.14742) In high-performance computing settings, Apache Arrow is used for integration with pandas and NumPy; a [Tulane University](https://www.edgepedia.org/tulane-university) project processing tens of terabytes of high-frequency trade data combined Spark, Lustre, and Arrow with pandas and NumPy, reducing a 10-hour process to 5 minutes after recoding the computationally intensive portions.[15](https://crsc.tulane.edu/highlights/financial_market_trade_data_analysis/)
## References
1. [Package overview, pandas 3.0.6 documentation](https://pandas.pydata.org/docs/getting_started/overview.html)
2. [About pandas, pandas.pydata.org](https://pandas.pydata.org/about/index.html)
3. [Release notes, pandas 3.0.6 documentation](https://pandas.pydata.org/pandas-docs/stable/release.html?highlight=option_context)
4. [What's new in 3.0.0 (January 21, 2026), pandas documentation](https://pandas.pydata.org/docs/whatsnew/v3.0.0.html)
5. [Project governance, pandas.pydata.org](https://pandas.pydata.org/about/governance.html)
6. [Scaling to large datasets, pandas 3.0.6 documentation](https://pandas.pydata.org/docs/user_guide/scale.html?highlight=chunksize)
7. [Wes McKinney (2010). Data Structures for Statistical Computing in Python. dblp record](https://dblp.org/rec/conf/scipy/McKinney10)
8. [Intro to data structures, pandas 3.0.6 documentation](https://pandas.pydata.org/docs/user_guide/dsintro.html?highlight=dic)
9. [pandas on PyPI](https://pypi.org/project/pandas/)
10. [Evaluation of Dataframe Libraries for Data Preparation on a Single Machine (EDBT 2025), arXiv:2312.11122](https://arxiv.org/abs/2312.11122)
11. [Coming from Pandas, Polars user guide](https://pola-rs.github.io/polars-book/user-guide/migration/pandas/)
12. [An Empirical Study on the Energy Usage and Performance of Pandas and Polars Data Analysis Python Libraries, VU Amsterdam](https://research.vu.nl/ws/portalfiles/portal/361654082/An_Empirical_Study_on_the_Energy_Usage_and_Performance_of_Pandas_and_Polars_Data_Analysis_Python_Libraries.pdf)
13. [Working with text data, pandas 3.0.6 documentation](https://pandas.pydata.org/docs/user_guide/text.html?highlight=startswith)
14. [PandasCorpus: A Resource of Real-World Pandas Workflows and Usage Patterns, arXiv:2608.14742](https://arxiv.org/abs/2608.14742)
15. [Financial Market Trade Data Analysis, Tulane Center for Research and Scientific Computing](https://crsc.tulane.edu/highlights/financial_market_trade_data_analysis/)
---
*Topic: Encyclopedia › Physical world and mathematics › Mathematics and statistics › Statistics and probability › Applied, official, and domain statistics › Computational statistics › Statistical software and environments*
*Initially written Sep 17, 2026 · Reviewed: Oct 15, 2026 · Edited: Oct 15, 2026 · Last review: Oct 15, 2026*
*Copyright 2026 EdgeChat AI, a subsidiary of Biostate AI.*
License: Edgepedia Community License 1.0, https://www.edgepedia.org/edgepedia/license