Skip to content

Support reading directory in read_csv #2

@ehariri

Description

@ehariri

Pandas should support reading directory of CSV files as this is a common Data Engineering need. Currently, the filepath_or_buffer argument of read_csv in Pandas must direct to a single CSV file. Other frameworks (e.g. Spark, Bodo) support reading a directory containing multiple partitioned CSV files (and then only read the files having extension .csv inside directory).

Assuming ~/path/to/csvs contains CSV files with the same schema, we wish to do

pd.read_csv('~/path/to/csvs')

rather than read the files individually.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions