BUG: ufunc is not applied to sparse.fill_value

[sinhrks]

• tests added / passed
• passes git diff upstream/master | flake8 --diff
• whatsnew entry

When ufunc is applied to sparse, it is not applied to fill_value. Thus results are incorrect.

on current master:

np.abs(pd.SparseArray([1, -2, -1], fill_value=-2))
# [1.0, -2, 1.0]
# Fill: -2
# IntIndex
# Indices: array([0, 2], dtype=int32)

np.add(pd.SparseArray([1, -2, -1], fill_value=-2), 1)
# [2.0, -2, 0.0]
# Fill: -2
# IntIndex
# Indices: array([0, 2], dtype=int32)

cc @gfyoung

[codecov-io]

Current coverage is 85.27% (diff: 93.33%)

Merging #13853 into master will decrease coverage by <.01%

@@             master     #13853   diff @@
==========================================
  Files           139        139          
  Lines         50020      50031    +11   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42657      42666     +9   
- Misses         7363       7365     +2   
  Partials          0          0          

Powered by Codecov. Last update 97de42a...a14f573

[gfyoung]

Can you explain why this result is correct?

[sinhrks]

Because assert_sp_array_equal compares sparse internal representation, it is for prepare correct internal repr. You can see the result is correct from its dense repr.

# test case
sparse = pd.SparseArray([1, -1, 2, -2], fill_value=1)
abs(sparse).to_dense()
# array([ 1.,  1.,  2.,  2.])

# result
pd.SparseArray([1, 2, 2], sparse_index=sparse.sp_index, fill_value=1).to_dense()
# array([ 1.,  1.,  2.,  2.])

[gfyoung]

Okay, good to know. I wasn't 100% clear on how the sparse comparison worked. Thanks!

[gfyoung]

LGTM

cc @jreback

[jreback]

can you put a comment here of what this is doing

[jreback]

I understand why this is needed, but it feels a tad unnatural. I am not sure a user will be expecting that the fill value will have the ufunc be applied here. Can we add a section to the docs showing this?

[sinhrks]

Sure, added small section.