🚀 Add missing tokenizer test files

Several tokenizers currently have no associated tests. I think that adding the test file for one of these tokenizers could be a very good way to make a first contribution to transformers.

Tokenizers concerned

not yet claimed

none

claimed

• LED @nnlnr
• Flaubert @anmolsjoshi
• Longformer @tgadeliya
• Electra @rajathb

Change tensor.data to tensor.detach() due to
pytorch/pytorch#6990 (comment)
tensor.detach() is more robust than tensor.data.

Although the results look nice and ideal in all TensorFlow plots and are consistent across all frameworks, there is a small difference (more of a consistency issue). The result training loss/accuracy plots look like they are sampling on a lesser number of points. It looks more straight and smooth and less wiggly as compared to PyTorch or MXNet.

It can be clearly seen in chapter 6([CNN Lenet](ht

Describe the bug

Streaming Datasets can't be pickled, so any interaction between them and multiprocessing results in a crash.

Steps to reproduce the bug

import transformers
from transformers import Trainer, AutoModelForCausalLM, TrainingArguments
import datasets

ds = datasets.load_dataset('oscar', "unshuffled_deduplicated_en", split='train', streaming=True).with_format("

In gensim/models/fasttext.py:

    model = FastText(
        vector_size=m.dim,
        vector_size=m.dim,
        window=m.ws,
        window=m.ws,
        epochs=m.epoch,
        epochs=m.epoch,
        negative=m.neg,
        negative=m.neg,
        # FIXME: these next 2 lines read in unsupported FB FT modes (loss=3 softmax or loss=4 onevsall,
        # or model=3 supervi

Is your feature request related to a problem? Please describe.
I typically used compressed datasets (e.g. gzipped) to save disk space. This works fine with AllenNLP during training because I can write my dataset reader to load the compressed data. However, the predict command opens the file and reads lines for the Predictor. This fails when it tries to load data from my compressed files.

Checking the Python files in NLTK with "python -m doctest" reveals that many tests are failing. In many cases, the failures are just cosmetic discrepancies between the expected and the actual output, such as missing a blank line, or unescaped linebreaks. Other cases may be real bugs.

If these failures could be avoided, it would become possible to improve CI by running "python -m doctest" each t

Hey Hackers of this spoopy month! 👻
Welcome to the Ciphey repo(s)!
This issue requires you to add a decoder.

This wiki section walks you through EVERYTHING you need to know, and we've added some more links at the bottom of this issue to detail more about the decoder.
https://github.com/Ciphey/Ciphey/wiki#adding-your-own-crackers--decoders

https://www.dcode.fr/t9-cipher
https://en.wikipe

Some ideas for figures to add to the PPT

• Linear regression, single-layer neural network
• Multilayer Perceptron with hidden layer
• Backpropagation
• Batch Normalization and alternatives
• Computational Graphs
• Dropout
• CNN - padding, stride, pooling,...
• LeNet
• AlexNet
• VGG
• GoogleNet
• ResNet
• DenseNet
• Memory Net