The Wayback Machine - https://web.archive.org/web/20201117144005/https://github.com/rust-ndarray/ndarray/issues/597
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are we ndarray yet? #597

Open
LukeMathWalker opened this issue Mar 17, 2019 · 5 comments
Open

Are we ndarray yet? #597

LukeMathWalker opened this issue Mar 17, 2019 · 5 comments

Comments

@LukeMathWalker
Copy link
Member

@LukeMathWalker LukeMathWalker commented Mar 17, 2019

Purpose

The idea behind this collection is to provide an index to easily navigate all currently open ndarray's issues which are immediately actionable.
This is meant to be a good starting point for new contributors (e.g. what should I work on?) and it can also help existing contributors to identify trends and hot areas. I have pinned it using GitHub's new feature, so that it doesn't get lost (and stale).

Given that we have ~100 open issues (and more are opened every day), you are very welcome contributing to this taxonomy effort either commenting on this issue or editing it directly (if you have permissions to do so).
I am only adding to this tracker things I can easily understand/where enough context is provided in the issue - if I left something along the way, feel free to add it and to provide more info on it.

New functionality

Documentation

  • Guidelines on how to use ndarray's types in a public API (Similar to Vec<T> vs &[T] considerations)

Feature parity

  • Equivalent of numpy.where or numpy.nonzero (Issue: #466)
  • Kronecker product (or tensor product) (Reference: np.kron) (Issue: #652 PR: #690)
  • Scalar versions of standard deviation and variance (Issue: #655)
  • Add dstack, vstack, and hstack (Issue: #667)
  • Sorting (Issue: #195)

Interop / Finer-grained control

  • Implement ascontiguousarray() or contiguous() method (Issue: #532)
  • Add shrink_to_fit method (Issue: #427)

Ergonomics

  • Implement multislice_axis! macro (Issue: #593)
  • New constructor method for 2D arrays from an iterator of 1D arrays/vectors (Issue: #539) (#609)
  • ArrawView as custom Dynamically Sized Type (Issue: #538)

Quality of life

  • Implement proptest's Arbitrary trait for Array (Issue: #596)
  • Add new type aliases: ArcArray1 and ArcArray2 (Issue: #661)
  • Run rustfmt on the project and add it to the CI pipeline (PR: #608)
  • Run clippy on the project and take care of the linter warnings (PR: #642)

Other

  • Add in-place variants of dimension-changing operations for dynamic-dimensional arrays (Issue: #428)
  • Support Clone elements in stack and select (Issue: #269)

Improvements

Documentation

  • Add a new example to ndarray-examples
  • Provide more details on Axis NewType pattern rationale (Issue: #564)
  • Document ndarray's equivalent to NumPy's astype (Issues: #493 , #525)
  • Improve doc examples for Zip/azip with failing examples (Issue: #453)

Error messages / Debugging

  • Better messages for incompatible shapes errors (Issue: #449).
  • Better formatting with Debug for arrays (Issue: #398, PR: #606)

Sharp API edges/corner cases

  • Avoid panicking for zero-length axis in map_axis/map_axis_mut (Issue: #579)
  • Refactor all dimension-related traits (Issues: #519 #367)

Core

  • Change ArrayBase.ptr to NonNull type (Issue: #434 PR: #683)
  • Provide more direct mutable access to shape, strides, and owned data (Issues: #429 #592)

Performance

  • Have a look at sum_3_azip (Issue: #561)
  • Faster, arbitrary-order iterators (Issue: #469)
@LukeMathWalker
Copy link
Member Author

@LukeMathWalker LukeMathWalker commented Mar 17, 2019

Going through all of these issues, I have starting to think at broader challenges which should probably fall under ndarray's umbrella or are relevant to the project:

  • masked arrays
  • zero-cost interop with other scientific stacks using the Apache Arrow project
  • numpy.einsum equivalent
  • consolidating all currently maintained and mature ndarray-* crates into the rust-ndarray organization, harmonizing interfaces and integrating docs where appropriate
@oracleofnj
Copy link

@oracleofnj oracleofnj commented May 1, 2019

I've started taking a crack at einsum here. The implementation I have there has multiple issues (performance and otherwise) and is not at all ready for production, but is apparently correct. I'm actively working on improving the implementation. There's a web frontend that uses the crate as a WASM module deployed here.

@LukeMathWalker
Copy link
Member Author

@LukeMathWalker LukeMathWalker commented May 2, 2019

The front-end is what I dreamed I could have when I started to use np.einsum back in the days - quite cool @oracleofnj!
Parsing the output correctly is definitely the first step there - then it comes down to properly optimizing the computation path based on the inputs and the specified contractions. What is your attack plan @oracleofnj?

@oracleofnj
Copy link

@oracleofnj oracleofnj commented May 2, 2019

After reading through the implementations/documentation in numpy and opt_einsum, I'm writing the base cases to handle a single operand or a pair of operands and then I'll write a function that takes the general case along with a pre-specified path and iterates along the path using the base cases. Last will come an independent function (or functions) to optimize the path given the operand sizes.

@oracleofnj
Copy link

@oracleofnj oracleofnj commented May 24, 2019

I published a beta version of my crate to crates.io. It still has some issues but it's far enough along that you are welcome to give it a spin. There is a minimal example (and more in the tests/benches) at the crate repo where you should feel free to open any issues - we can move the discussion there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.