0

Im quit new to python coming from the JS world .

When installing a package in javascript all the subdependencies of that packge are not part of the package.json. dependencies section. For example:

npm init -y
npm install express

would produce

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.18.2"
  }

even though express.js has 31 dependencies

Reading about python dependencies. I did the following steps:

created virtual env:

 python -m venv my-virutal-env

made sure no package is installed:

 pip freeze > to-uninstall.txt
 pip uninstall -r to-uninstall.txt
 pip freeze > requirements.txt

Installed llamaIndex

 pip install llama_index

producing requirements again using pip freeze > requirements.txt

produces:

...

llama-index==0.5.12
...

when ... are a lot of other packages

This is very undesirable. How Can I overcome this?

Improve this question
10
  • "This is very undesirable." How so? The entire point of requirements.txt is reproducibility, which needs all dependencies. Commented Apr 10, 2023 at 14:20
  • pip freeze lists every third-party package installed in the current environment. All those other packages were installed when you ran pip install llama_index. You can manually edit requirements.txt to just have the llama-index==0.5.12 line, and it'll install just fine. However, as MisterMiyagi said, you want to make sure all dependencies and their versions are listed for reproducibility. Commented Apr 10, 2023 at 14:20
  • 1
    I don't think I want to be handling the dependencies of my dependencies But you do. By providing a complete requirements.txt, you are saying to the end user that your package works in exactly this environment. It may work in others, it may not, but I know it works here. The user is of course free to update any of the [sub]dependencies as they wish, but if something breaks it's on them. The whole point of a package is that it is like a black box I disagree with that statement in several ways, but even if it is, you are programming that black box. You're responsible for the dependencies. Commented Apr 10, 2023 at 14:42
  • 1
    requirements.txt isn't there to define the dependencies of your package. That is the job of the package distribution, commonly via pyproject.toml. requirements.txt is there to define the Python environment of a program for reproducibility. Commented Apr 10, 2023 at 16:23
  • 1
    "Creating a ``` requirements .text``` is like creating a "snapshot" of something dynamic, and hence it is breakable in cases like the example I stated in my previous comment." The Python package index is immutable; it only accepts additions, not modifications. There is no way for a package to change its code or dependencies "without creating a new version". That ties into the entire point of requirements.txt: specifying a set of environment package versions known to be good. Commented Apr 10, 2023 at 16:27

1 Answer 1

Reset to default
-1

Using the pattern pip freeze >requirements.txt is just a suggestion, mostly derived from documentation that followed a line of "make your code work first, with everything you need. Now so that others can run your code, use a dump of everything that is installed as your requirements file"

For a small number of known dependencies, just edit your requirements.txt file by hand. You can either edit it from scratch, or, if desired, simply remove all lines for projects you know will be brought along automatically when the higher-level, explicitly needed dependency is installed. Also, note that the sucessor of requirements.txt, the pyproject.toml file, used for more recent tooling such as poetry, works on this basis - but will also need you to add manually the known explicit dependencies of your project. The only difference for your situation is that (Poetry in particular, not sure about other tools), updates the pyproject.toml tool itself as you go adding the explicit dependencies, while as in requirements.txt you are supposed to edit that file.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.