ValueError: attempted relative import beyond top-level package python

Question

I have a directory structure like this.

Chatbot/
   utils/
     abc.py
   projects/
      proj1/
        utils/
          __init__.py
          data_process.py
        components/
          class1.py

I have two utils folders in my structure, one at the top level and one inside my projects folder.

Now I want to import data_process.py file inside class1.py. So I tried like this

from utils.data_process import DataProcess

but it is referencing the top-level utils folder and even VSCode is not recognizing it. I tried creating __init__.py file inside the utils folder but still did not work.

I tried with empty __init__.py and then placing this content

from . import data_process
__all__ = ['data_proces']

then

from .data_process import DataPreprocess
__all__ = ['DataPreprocess']

then I tried

from ..utils.data_process import DataProcess

VSCode is recognizing this but it is not working and throws the error

ValueError: attempted relative import beyond top-level package

Even I tried changing the name utils to some other name but still the same issue

How can I solve this?

S.B · Accepted Answer · 2021-07-30 17:07:19Z

1

sys.path is where Python searches to find modules and packages and It does it in order.

So you can put ...../proj1/ at the beginning of the list, when python start searching it will find the utils folder in that path first !

import sys
sys.path.insert(0, r'...../proj1/')

But this cause another problem, python always find utils in that folder first, that's not what you want.

solution number 1

Absolute import:

from Chatbot.projects.proj1.utils.data_process import DataProcess

solution number 2

use relative imports which is mentioned by @Mr.Hobo.

edited Jul 30, 2021 at 17:07

answered Jul 28, 2021 at 5:50

S.B

17k12 gold badges38 silver badges73 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Mr. Hobo Over a year ago

Yes, sys.path method is good.. but it is tedious to mention each time... you can get over this using the PYTHONPATH tutorial in windows

S.B Over a year ago

@Mr.Hobo emm In fact both of them(PYTHONPATH or sys.path) are causing problems. The main issue is we have two names called utils ! whichever path is accessible first in sys.path, that utils will be encountered first it doesn't matter where we are. So better solution is using absolute import as I mentioned.

Mr. Hobo Over a year ago

please check the updated answer. I was able to do the project with ease w/o any problem. I have also added a link to my personal GitHub project account related to this project structure.

S.B Over a year ago

@Mr.Hobo I've updated my answer and referenced to yours for relative imports.

Mr. Hobo · Accepted Answer · 2024-07-07 13:43:08Z

A python module is defined by a special file called __init__.py and this file should be present in all the subdirectories. So, your file structure would be:

Chatbot/
   __init__.py
   utils/
     __init__.py
     abc.py
   projects/
      __init__.py
      proj1/
        __init__.py
        utils/
          __init__.py
          data_process.py
        components/
          __init__.py
          class1.py
          class2.py

Now, you can do a relative import like:

Use . for importing something within the same directory. Example:

# file Chatbot/projects/proj1/components/class2.py
from .class1 import *

Similarly, use .. for two-level, and ... for three-level, and so on!

For instance:

# file Chatbot/projects/proj1/components/class2.py
from ..utils.data_process import * # import from data_process.py
# there will be four "." earlier I made a counting mistake
from ....utils.abc import * # import something from abc.py

Coding Convention

When writing a python module/package, you might want to follow PEP8 Convention.

Moreover, I believe you are trying to do different projects using your package Chatbot, is that correct? Then, it is a good practice that you set PYTHONPATH for Chatbot and do all the projects, and imports seperately.

EDIT: Uploading Dummy File

I'm able to work on this project seamlessly even with using two utils directories. Project Structure:

main file and its output:

# file main.py
from chatbot.utils.abc import hello as a1
from chatbot.projects.proj1.components.class1 import super_hello
from chatbot.projects.proj1.utils.data_process import hello as b1

print(a1(), b1())
print(super_hello())

Similarly, if you have a chatbot under PYTHONPATH you can call the project from anywhere from your device.

Code details and files are added to my github account for details.

chatbot outer project use the model created from inner project, thats why it contains projects folder.. projects folder contains the independent project but its output is used on top level chatbot project.
and i need to add init.py file to every folder, that are not even packages?
I don't understand by outer/inner project that you have mentioned. You might want to look into these to clear concepts: Python Packaging - [1, 2]; Import UD Package - [1, 2]
If you are creating a chatbot and you want to use this python package in different codes, then this should not be the way you design the code. Instead you should use PYTHONPATH or sys.path mentioned by @SorousH Bakhtiary

Collectives™ on Stack Overflow

ValueError: attempted relative import beyond top-level package python

2 Answers 2

solution number 1

solution number 2

4 Comments

Coding Convention

EDIT: Uploading Dummy File

7 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

solution number 1

solution number 2

4 Comments

Coding Convention

EDIT: Uploading Dummy File

7 Comments

Linked

Related