arrow

Feature Request

Many locales have the bare minimum when it comes to test cases. While I understand it can be tedious and repetitive to write out test case

We can reduce friction by figuring out how to load data most efficiently to polars memory.

Problem statement

The algorithms cudf::reduce and cudf::segmented_reduce do not accept an initial value.

Similar algorithms like [std::reduce](https://e

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
PR apache/arrow-datafusion#2521 adds OFFSET to the logical plan. We should implement a physical plan for it.

Describe the solution you'd like
Implement OFFSET physical plan.

Describe alternatives you've considered
None

Additional context
None

We no longer need to control the number of concurrent kernels, since now we control the number of concurrent tasks

the pre-built binary is not supporting database?

roapi -t "vocabs=sqlite:///data/vocabulary.sqlite"
[2022-05-31T06:48:11Z INFO  roapi::context] loading `uri(sqlite:///data/vocabulary.sqlite)` as table `vocabs`
Error: Database error: Enable 'database' feature flag to support this

would you explain in README how to enable it?

I'm new to rust, after some searching i got it workin

It would be helpful to have Fletchgen output warnings for unused metadata fields that start with fletcher_. For example, (this happened to me) when someone adds fletchgen_epc to Schema metadata instead of Field metadata.

Problem description

Reading a dataset with eager's read functionality raises a ValueError when providing columns.

Example code (ideally copy-pastable)

import pandas as pd

from tempfile import TemporaryDirectory
from functools import partial
from storefact import get_store_from_url

from kartothek.io.eager import store_dataframes_as_dataset, read_dataset_as_data

Motivation:

1. Improved compile times (at least by 2x compared to arrow-rs).
2. Faster Parquet impl
3. Projects are migrating to arrow2 (including Datafusion and Polars)

Describe the bug
We have a hard-coded distinct = false parameter in ballista/rust/core/src/serde/physical_plan/mod.rs.

Ok(create_aggregate_expr(
    &aggr_function.into(),
    false, // <-- hard-coded "distinct"
    input_phy_expr.as_slice(),
    &physical_schema,
    name.to_string(),
)?)

To Reproduce
Try running a COUNT(DISTINCT expr) in Ballista

**E