pre-trained-language-models

RoBERTa中文预训练模型: RoBERTa for Chinese

Top2Vec learns jointly embedded topic, document and word vectors.

A curated list of NLP resources focused on BERT, attention mechanism, Transformer networks, and transfer learning.

Must-read papers on prompt-based tuning for pre-trained language models.

基于预训练模型的中文关键词抽取方法（论文SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-trained Language Model 的中文版代码）

The code of our paper "SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-trained Language Model"

We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with least artificial participation. The methods we use involve lite pre-training models such as Albert-small or Electra-small with financial corpus, knowledge of distillation and multi-stage learning. The result is that we improve the recall rate of company names recognition task from 0.73 to 0.92 and get 4 times as fast as BERT-Bilstm-CRF model.

Zero-shot Transfer Learning from English to Arabic

Zero-shot Transfer Learning from English to Arabic

Calculating FLOPs of Pre-trained Models in NLP

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization (ACL 2021)

Usage example for the AllenNLP BiDAF pre-trained model

2020年贝壳找房问答匹配竞赛

IndoELECTRA: Pre-Trained Language Model for Indonesian Language Understanding

Question Answering Chatbot with DistilRoBERTa Sentence Embeddings, Dialogflow and Ngrok

总结NLP中常见的几种预训练语言模型

create "entropic" pre-training sequences for neural language models