Skip to content

An asynchronous version of CatalogList/CatalogProvider/SchemaProvider #3777

Closed
@v0y4g3r

Description

@v0y4g3r

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I'm trying to implement a CatalogList/CatalogProvider/SchemaProvider that based on some remote storage service like ZooKeeper/Etcd.

Describe the solution you'd like
But current CatalogList/CatalogProvider/SchemaProvider trait is in a sync manner.

https://github.com/apache/arrow-datafusion/blob/e54110fb592e03704da5f6ebd832b8fe1c51123b/datafusion/core/src/catalog/catalog.rs#L29-L47

That makes asynchronously accessing remote storage service impossible while implementing CatalogList/CatalogProvider/SchemaProvider.

Describe alternatives you've considered

Maybe change these traits to async or provide an async version?

Additional context
I see projects like influxdb-iox did some hack to get around this problem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions