The Wayback Machine - https://web.archive.org/web/20200929200806/https://github.com/tj/staticgen/issues/12
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stops on 404 405 #12

Open
yene opened this issue Jan 5, 2020 · 5 comments
Open

Stops on 404 405 #12

yene opened this issue Jan 5, 2020 · 5 comments
Labels

Comments

@yene
Copy link

@yene yene commented Jan 5, 2020

Crawler stops on 404 or 405

@tj
Copy link
Owner

@tj tj commented Jan 5, 2020

I forgot to document this option: https://github.com/tj/staticgen/blob/master/config.go#L28 that should do the trick, at least for 404

@tj tj added the documentation label Jan 5, 2020
@skatkov
Copy link

@skatkov skatkov commented Jul 27, 2020

Got stuck on 404 too.

used this config, but still staticgen doesn't progress beyong 404 error.

{
  "url": "https://www.tssltd.ru",
  "concurrency": 1,
  "dir": "tss-dist",
  "allow_404": true
}

What am I doing wrong?

@tj
Copy link
Owner

@tj tj commented Jul 28, 2020

Hmm I'll try to check it out soon, config looks fine to me

@skatkov
Copy link

@skatkov skatkov commented Aug 12, 2020

@tj did you had time to give it a look? Maybe some pointers where problem could be?

@tj
Copy link
Owner

@tj tj commented Aug 12, 2020

not yet sorry! A quick grep doesn't show Allow404 anywhere though, looks like I maybe added the config but didn't implement it yet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.