25

According to this answer you can import pip from within a Python script and use it to install a module. Is it possible to do this with conda install?

The conda documentation only shows examples from the command line but I'm looking for code that can be executed from within a Python script.

Yes, I could execute shell commands from within the script but I am trying to avoid this as it is basically assuming that conda cannot be imported and its functions called.

5
  • Why don't you try and report back? :) Commented Jan 20, 2017 at 15:43
  • Report what back? Commented Jan 20, 2017 at 15:52
  • you can always use subprocess Commented Jan 20, 2017 at 16:22
  • Yes, and I can also just use a bash script too but I'd like to avoid executing shell commands from within Python as it seems hackish. It is possible to import pip from within the Python script and install packages and it looks like I can import conda as well. Just don't see how to use conda from within a Python script. Commented Jan 20, 2017 at 16:25
  • While it is possible to import pip in Python, that is explicitly discouraged in pip's documentation. I don't see anything hackish about using the public facing API's that pip and conda recommend -- which are based on calling new processes with arguments and option flags. conda now has a Python API that mimics the command line (see @YenForYang's answer). This API did not exist when this question was first posted. Commented May 23, 2019 at 12:48

6 Answers 6

23

You can use conda.cli.main. For example, this installs numpy:

import conda.cli

conda.cli.main('conda', 'install',  '-y', 'numpy')

Use the -y argument to avoid interactive questions:

-y, --yes Do not ask for confirmation.

Sign up to request clarification or add additional context in comments.

10 Comments

Thanks! This did the trick and was exactly what I was looking for. I had seen this in the conda script itself but couldn't decipher how to use it. I was passing a list to it rather than a set of commands.
I found that when I tried to do this with a script that had command line arguments that conda tried to interpret the comand line arguments instead of the arguments of conda.cli.main(), so using conda.cli.main() like this might not work for some things.
How do you do when the channel is not the default?
@ws_e_c421 See my answer below if you are looking for a solution to that issue (still).
@pceccon Simply add the arguments '-c', '<CHANNEL>'. See my answer below for an example.
|
12

I was looking at the latest Conda Python API and noticed that there are actually only 2 public modules with “very long-term stability”:

  1. conda.cli.python_api
  2. conda.api

For your question, I would work with the first:

NOTE: run_command() below will always add a -y/--yes option (i.e. it will not ask for confirmation)

import conda.cli.python_api as Conda
import sys

###################################################################################################
# The below is roughly equivalent to:
#   conda install -y 'args-go-here' 'no-whitespace-splitting-occurs' 'square-brackets-optional'

(stdout_str, stderr_str, return_code_int) = Conda.run_command(
    Conda.Commands.INSTALL, # alternatively, you can just say "install"
                            # ...it's probably safer long-term to use the Commands class though
                            # Commands include:
                            #  CLEAN,CONFIG,CREATE,INFO,INSTALL,HELP,LIST,REMOVE,SEARCH,UPDATE,RUN
    [ 'args-go-here', 'no-whitespace-splitting-occurs', 'square-brackets-optional' ],
    use_exception_handler=True,  # Defaults to False, use that if you want to handle your own exceptions
    stdout=sys.stdout, # Defaults to being returned as a str (stdout_str)
    stderr=sys.stderr, # Also defaults to being returned as str (stderr_str)
    search_path=Conda.SEARCH_PATH  # this is the default; adding only for illustrative purposes
)
###################################################################################################


The nice thing about using the above is that it solves a problem that occurs (mentioned in the comments above) when using conda.cli.main():

...conda tried to interpret the comand line arguments instead of the arguments of conda.cli.main(), so using conda.cli.main() like this might not work for some things.


The other question in the comments above was:

How [to install a package] when the channel is not the default?

import conda.cli.python_api as Conda
import sys

###################################################################################################
# Either:
#   conda install -y -c <CHANNEL> <PACKAGE>
# Or (>= conda 4.6)
#   conda install -y <CHANNEL>::<PACKAGE>

(stdout_str, stderr_str, return_code_int) = Conda.run_command(
    Conda.Commands.INSTALL,
    '-c', '<CHANNEL>',
    '<PACKAGE>'
    use_exception_handler=True, stdout=sys.stdout, stderr=sys.stderr
)
###################################################################################################

3 Comments

This should be the accepted answer. The originally accepted answer relies on an internal API with no guarantee of stability. Also, I think a complete answer should mention calling conda with the subprocess module. Calling conda from the subprocess module with the --json option and parsing the results is perfectly valid and likely even more stable. It also does not require conda to be installed in the conda environment. Avoiding subprocess for performance reasons is likely premature optimization.
what is CHANNEL?
It is stated above the fragment of code... conda has channels which hold recipes for different packages. For example, there is "anaconda" channel, "conda-forge" channel, "defaults" channel, "jetbrains" channel and so on.
6

Having worked with conda from Python scripts for a while now, I think calling conda with the subprocess module works the best overall. In Python 3.7+, you could do something like this:

import json
from subprocess import run


def conda_list(environment):
    proc = run(["conda", "list", "--json", "--name", environment],
               text=True, capture_output=True)
    return json.loads(proc.stdout)


def conda_install(environment, *package):
    proc = run(["conda", "install", "--quiet", "--name", environment] + packages,
               text=True, capture_output=True)
    return json.loads(proc.stdout)

As I pointed out in a comment, conda.cli.main() was not intended for external use. It parses sys.argv directly, so if you try to use it in your own script with your own command line arguments, they will get fed to conda.cli.main() as well.

@YenForYang's answer suggesting conda.cli.python_api is better because this is a publicly documented API for calling conda commands. However, I have found that it still has rough edges. conda builds up internal state as it executes a command (e.g. caches). The way conda is usually used and usually tested is as a command line program. In that case, this internal state is discarded at the end of the conda command. With conda.cli.python_api, you can execute several conda commands within a single process. In this case, the internal state carries over and can sometimes lead to unexpected results (e.g. the cache becomes outdated as commands are performed). Of course, it should be possible for conda to handle this internal state directly. My point is just that using conda this way is not the main focus of the developers. If you want the most reliable method, use conda the way the developers intend it to be used -- as its own process.

conda is a fairly slow command, so I don't think one should worry about the performance impact of calling a subprocess. As I noted in another comment, pip is a similar tool to conda and explicitly states in its documentation that it should be called as a subprocess, not imported into Python.

5 Comments

I would like to use your solution but I am running into the issue where it complains about conda init not being run. I think it actually reads the source of your bashrc (or whatever) and stops you from continuing. If there is a way on the command line to disable this (its not documented AFAIK) then this is the best way for the reasons you list.
I usually run conda activate base in the environment before running Python scripts that interact with conda with subprocess. I am not sure what you mean by environment stuff. I use these scripts to interact with conda environments.
I am using a remote invocation framework (invoke/fabric) which complicates things so that I can't just run conda activate beforehand. Re 'environment stuff': I am also trying to automate the creation, destruction, and activation of environments which is not a part of the conda.cli.python_api
That does make it tougher. I usually wrap my Python script in a bash script that uses conda activate when I need the conda environment. All conda activate does is set environment variables, so you could look at what it does and set them manually. An alternative is to wrap all the conda calls in conda run but it is still an alpha level feature. With conda run try to use as new a version of conda as possible for better support (one important improvement is not yet in a release of conda).
Didn't know about conda run but I will definitely keep an eye out on that. Manually setting variables might work as a quick patch, but sounds really fragile for automating stuff.
1

I found that conda.cli.python_api and conda.api are limited, in the sense that, they both don't have the option to execute commands like this:

conda export env > requirements.txt

So instead I used subprocess with the flag shell=True to get the job done.

subprocess.run(f"conda env export --name {env} > {file_path_from_history}",shell=True)

where env is the name of the env to be saved to requirements.txt.

1 Comment

I would like to programmatically export an environment to a requirements.txt file. But what is the correct value of 'file_path_from_history' supposed to be here? I get a "NameError: name 'file_path_from_history' is not defined" message when I run your command.
-2

The simpler thing that i tried and worked for me was :

import os

try:
    import graphviz
except:
    print ("graphviz not found, Installing graphviz ")
    os.system("conda install -c anaconda graphviz")
    import graphviz

And make sure you run your script as admin.

1 Comment

You shouldn't have to rely on admin permissions for anything with conda (unless it is distributed via your OS package manager of which there are none that I am aware of). Also goes against the request of the question.
-2

Try this:

!conda install xyzpackage

Please remember this has to be done within the Python script not the OS prompt.

Or else you could try the following:

import sys
from conda.cli import main

    sys.exit(main())

    try:
        import conda
        from conda.cli import main
        sys.argv = ['conda'] + list(args)
        main()

4 Comments

This seems hackish as it is executing a shell command. I'd like to import conda and call its equivalent install function.
Check the above link out which gives info on conda and python modules. Conda is an environment and a utility which is available in the OS and not inside python module. Is there any specific anaconda package that you have in mind?
Please see the answer by @mike-muller as, in fact, you can access conda from within a Python script.
This is a hack, but it works. Use it in limited doses, and understand the ramifications !!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.