The Wayback Machine - https://web.archive.org/web/20201030212822/https://github.com/huggingface/transfer-learning-conv-ai/issues/12
Skip to content

/ transfer-learning-conv-ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710 #12

Open
jb33k opened this issue Jun 4, 2019 · 2 comments
Open

RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710 #12

jb33k opened this issue Jun 4, 2019 · 2 comments
Labels
good first issue

Comments

@jb33k
Copy link

@jb33k jb33k commented Jun 4, 2019

I'm playing around with this wonderful code but I'm running into a curious issue when I try to train the model with my own data.

I replicated the personachat_self_original.json file structure and added my own data. I deleted dataset_cache_OpenAIGPTTokenizer file but when I try to train, I get this error:

INFO:train.py:Pad inputs and convert to Tensor
Traceback (most recent call last):
  File "train.py", line 252, in <module>
    train()
  File "train.py", line 164, in train
    train_loader, val_loader, train_sampler, valid_sampler = get_data_loaders(args, tokenizer)
  File "train.py", line 97, in get_data_loaders
    tensor = tensor.view((-1, datasets[dataset_name]["n_candidates"]) + tensor.shape[1:])
RuntimeError: shape '[-1, 2, 34]' is invalid for input of size 61710

I have triple checked that my dataset follows the same structure but I can't figure out why the training script doesn't like it.

Any ideas why this is happening?
@michelleful
Copy link

@michelleful michelleful commented Jun 28, 2019

Same thing happens to me. Did you ever wind up solving this, @jb33k?
@michelleful
Copy link

@michelleful michelleful commented Jul 3, 2019

For anyone else who may have this problem, my issue was that for some utterances in my data, the number of candidates for that utterance was less than n_candidates, but this line (currently #79 in train.py) assumes that you have at least that many:

 for j, candidate in enumerate(utterance["candidates"][-num_candidates:]):

As a quick fix, you can add a guard before this line to ensure that you're actually meeting that requirement.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue
3 participants
@michelleful @sshleifer @jb33k
You can’t perform that action at this time.