Cannot change directory to a script using bash - cron [duplicate]

Question

I have a script that runs multiple instances of Python Scrapy crawlers, Crawlers are int /root/crawler/batchscript.py

and in /root/crawler/ I have that scrapy crawler.

Crawlers are working perfectly fine.

batchscript.py looks like this, (posting only relevent code)

from scrapy.settings import Settings
from scrapy.utils.project import get_project_settings
from amazon_crawler.spiders.amazon_scraper import MySpider

process = CrawlerProcess(get_project_settings())

When I run batchscrip.py inside /root/crawler/ directory scraper runs fine.

But when I run it from outside of this directory using python /root/crawler/batchscript.py then it does not run as intended, (Settings are not imported correctly), get_project_settings() are empty.

I have tried creating a BASH script too I create bash script called batchinit.sh

#!/bin/bash
alias batchscript="cd /root/crawler/"
python batchscript.py

and behaviour is same :(

When I run batchinit.sh inside /root/crawler/ directory scraper runs fine.

But when I run it from outside of this directory using bash /root/crawler/batchinit.sh then it does not run as intended, (Settings are not imported correctly), get_project_settings() are empty.

Why I am doing it? What is ultimate goal?

I want to create a cronjob for this script. I tried to schedule cronjobs using above mentioned commands but I have issues as mentioned above.

What are you trying to do by defining the alias in the shell script? Why not just put cd /root/crawler/ on that line instead of aliasing it to batchscript? — Christopher Shroba
– Christopher Shroba, Commented Nov 17, 2016 at 18:42
Where are scrapy and amazon_crawler modules? Are them in a virtual env? — lucasnadalutti
– lucasnadalutti, Commented Nov 17, 2016 at 18:42
@ChristopherShroba I am newbie to shell scripting ... id wrote simple cd command but it didnt work ... — Umair Ayub
– Umair Ayub, Commented Nov 17, 2016 at 18:46

matias elgart · Accepted Answer · 2016-11-17 18:42:11Z

3

using bash, you could always do:

cd /root/crawler && python batchscript.py

it's always good policy to use absolute paths to programs/executables referenced in cron jobs.

answered Nov 17, 2016 at 18:42

matias elgart

1,19113 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Cannot change directory to a script using bash - cron [duplicate]

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related