The Wayback Machine - https://web.archive.org/web/20220512140224/https://github.com/deepset-ai/haystack/issues/2528
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

More user friendly default log levels #2528

Open
julian-risch opened this issue May 11, 2022 · 0 comments
Open

More user friendly default log levels #2528

julian-risch opened this issue May 11, 2022 · 0 comments
Assignees
@julian-risch
Labels
Contributions wanted! good first issue journey:first steps

Comments

@julian-risch
Copy link
Member

@julian-risch julian-risch commented May 11, 2022

When users run our tutorial notebooks, there are quite many convoluted log messages.

For example, there is the log message regarding apex and others:

"INFO - haystack.document_stores.base -  Numba not found, replacing njit() with no-op implementation. Enable it with 'pip install numba'.\n",
"INFO - haystack.modeling.model.optimization -  apex not found, won't use it. See https://nvidia.github.io/apex/\n",
"ERROR - root -  Failed to import 'magic' (from 'python-magic' and 'python-magic-bin' on Windows). FileTypeClassifier will not perform mimetype detection on extensionless files. Please make sure the necessary OS libraries are installed if you need this functionality.\n"

We should:

  • downgrade some of the existing messages that are currently on INFO level to DEBUG
  • adjust the default logging level of some third party libraries

Along the lines of but not as extreme as:

import logging
for module in ["farm.utils", "farm.infer", "haystack.reader.farm.FARMReader",
              "farm.modeling.prediction_head", "elasticsearch", "haystack.eval",
               "haystack.document_store.base", "haystack.retriever.base", 
              "farm.data_handler.dataset"]:
    module_logger = logging.getLogger(module)
    module_logger.setLevel(logging.ERROR)

Another example log message is
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...

These log messages from tokenizers should go away when turning off parallelism:
%env TOKENIZERS_PARALLELISM=false

There is a related issue: #1737
@julian-risch julian-risch self-assigned this May 11, 2022
@julian-risch julian-risch added the journey:first steps label May 11, 2022
@ZanSara ZanSara added good first issue Contributions wanted! labels May 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Contributions wanted! good first issue journey:first steps
2 participants
@julian-risch @ZanSara