A very good first issue IMO!

See huggingface/transformers#4829 (comment)

we should update the eval dataset to pick one start_position (or the most frequent one)

Optionally, use the huggingface/nlp library to get the eval dataset, and hook it into the Trainer.

Also referenced in huggingface/transformers#6997 (comment)

Not a high-priority at all, but it'd be more sensible for such a tutorial/testing utility corpus to be implemented elsewhere - maybe under /test/ or some other data- or doc- related module – rather than in gensim.models.word2vec.

Originally posted by @gojomo in RaRe-Technologies/gensim#2939 (comment)

more details at: allenai/allennlp#2264 (comment)

Hi I would like to propose a better implementation for 'test_indices':

We can remove the unneeded np.array casting:

Cleaner/New:
test_indices = list(set(range(len(texts))) - set(train_indices))

Old:
test_indices = np.array(list(set(range(len(texts))) - set(train_indices)))