0

with my multiprocessing code

    def __get_courses_per_moderator(self, moderator, page=None):
        print("started")
        try:
            response = self.class_service.courses().list(pageSize=0,
                                                         courseStates='ACTIVE', 
            teacherId=moderator, pageToken=None).execute()
            for data in response["courses"]:
                self.courses_in_classroom.append(data["name"])
            print(self.courses_in_classroom) # THE modifications are clear

        except Exception as e:
            print("__get_courses_per_moderator ERROR-{}".format(e))

    def get_courses_from_classroom(self):
        # batch = service.new_batch_http_request(callback=self.__callback)
        pool = []
        for email in self.additional_emails:
            try:
                process = multiprocessing.Process(
                    target=self.__get_courses_per_moderator, args=[email])
                process.start()
                pool.append(process)
            except Exception as e:
                print("get_courses_from_classroom ERROR-{}".format(e))

        for process in pool:
            process.join()
            print("joined")
        print(self.courses_in_classroom) # the attribute is empty.

As far as I understand python is synchronous. So when the processes update the class attribute, the value should be there right? or should I try to return it and then concat after join() ? A simple explanation would be lovely.

4
  • 1
    As an aside, why not use a proper multiprocessing.Pool for this? Wouldn’t it simplify the code somewhat? Commented Jun 6, 2020 at 18:46
  • Yes I have plans to use that as always but want to learn this clearly. Thanks for that. I appreciate it. Commented Jun 6, 2020 at 19:30
  • with concurrent.futures.ThreadPoolExecutor() as excecutor: results = excecutor.map( self.__check_course_state, self.courses[0:5]) Is how i ended up doing it with your recommendation. Commented Jun 8, 2020 at 13:56
  • That's quite different, it's using threads not processes. Also, why the __ in def __get_courses_per_moderator(self, moderator, page=None): ? Commented Jun 8, 2020 at 20:45

1 Answer 1

0

I fixed it by using a process manager and putting that manager outside of the class.

manager = multiprocessing.Manager()
course_list = manager.list()
class x:
    #append things to course_list

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.