2

I have a module lib that needs numpy. So, for instance, let's say I have a hypothetical function that looks like 

import numpy
def doSomething(x):
    return numpy.sqrt(x)

Now to use that function in a separate module, I will import the name as 

from lib import doSomething
...

Here is the tricky part... Now I want to expose another version of doSomething for which numpy has been imported from another library (in particular, from autograd). So, for instance, I would like to be able to have a function 

from autograd import numpy
def doSomething(x):
    return numpy.sqrt(x)

where the only difference between these functions is where numpy is being imported from. In particular, I would like to use both versions of doSomething in the same code, that is, I would like some way of importing doSomething twice... once with the default numpy, and once with the numpy from autograd. Something like this:

useAutograd = False
from lib(useAutograd) import doSomething
useAutograd = True
from lib(useAutograd) import doSomething as doSomethingAutograd

There are a couple options here that I know of, but none which are satisfactory.

  1. I would make a copy of the codebase, and have one that uses the default numpy, and one which uses numpy from autograd. This is bad because it would require me to maintain two codebases which are copies of each other, only with different imports.

  2. I could put in a conditional import:

    try:
    from autograd import numpy
except ImportError:
    import numpy

    This is bad because the user has no control over which version is imported... If they have autograd, then they must use that version.

  3. I could define an environmental variable to control the import 

    import os
if os.environ.get('AUTOGRADNUMPY'):
    try:
        from autograd import numpy
    except ImportError:
        import numpy
else:
    import numpy

    This has the downside that although the user can control the import, they can only pick one version (as far as I know). So they could not use both versions in the same code.

Are there any better alternatives for this use case?

Background for those interested:

Autograd has its own set of functions which mimic numpy and allow one to easily compute derivatives using automatic differentation (in the same vein as tensorflow), without requiring expensive numerical differentiation.

HOWEVER, their numpy implementation is not the most optimized version (AFAIK). So it would be advantageous to allow the user to use the version with the autograd import when they will need the jacobian of the function, and to use their default, highly optimized numpy package when they don't.

Improve this question
12
  • This is an interesting problem but still, why would you want to do such a thing? Commented Feb 13, 2018 at 10:02
  • Do you have a problem with the implementation that you showed us? I'm not sure I understand what you want to do and obtain Commented Feb 13, 2018 at 10:03
  • I will update the question with the rationale... Commented Feb 13, 2018 at 10:03
  • So if I understand well.. OK, I cannot understand what you mean. If you just want to import two different doSomething, what's wrong with it? You won't import multiple numpy as numpy imported from autograd is also pointed to which imported by import numpy. They are the same. Commented Feb 13, 2018 at 10:04
  • 2
    @bremen_matt I am still not convinced of the need of such a thing. Commented Feb 13, 2018 at 10:10

3 Answers 3

Reset to default
3

If your desire is to avoid duplicating your code base, make your interface a class instead. For example:

class using_numpy:
    import numpy

    @classmethod
    def do_something(cls, x):
        return cls.numpy.sqrt(x)

class using_autograd(using_numpy):
    from autograd import numpy

Now using_numpy.do_something will use numpy, and using_autograd.do_something will use autograd.numpy.

Alternatively, if you are uncomfortable with classmethod, you could make your interfaces instances of a class, for example:

class interface:
    def __init__(self, mdl):
        self.mdl = mdl

    def do_something(self, x):
        return self.mdl.sqrt(x)

import numpy
import autograd

with_numpy = interface(numpy)
with_autograd = interface(autograd.numpy)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Hmm interesting. I didn't think of this. This looks really good
Alternative 2 has been enlightening.
1

You don't need to do anything to achieve this.

If you do 

from lib import doSomething
from lib_with_autograd import doSomething as doSomethingAutograd

each of those functions uses the numpy imported in their specific module. So doSomethingAutograd uses the one imported in lib_with_autograd and doSomething uses the one imported in lib

Improve this answer

3 Comments

Yes, but as I say, this requires that I copy the entire codebase from lib into a new codebase where the only difference is the import statements. Thus I would have to start maintaining 2 libraries
@bremen_matt But you only need to do it for those functions for which you would want to use autograd. All the other ones you can import from the normal lib.
Essentially every function (of which there are thousands) is using numpy internally. There are almost no functions that do not use numpy.
1

Since everything in python is an object, including modules, you can do something like:

def doSomething(x, numpy=None):
    if numpy is None:
        import numpy
    return numpy.sqrt(x)

Then you can call the function without setting numpy, then it will use the default numpy. If you want to use another numpy, just call it as such:

from autograd import numpy as autograd_numpy
doSomething(x, numpy=autograd_numpy)
Improve this answer

1 Comment

This would work and is similar to the version proposed by donkopotamus. However, I believe his suggestion is slightly better because it more naturally handles subfunctions in the package. With this approach, if one method called another, then you would have to be sure to pass the module to the subfunctions. With the other approach, that would be much less error prone.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.