pre-trained-language-models

RoBERTa中文预训练模型: RoBERTa for Chinese

Must-read papers on prompt-based tuning for pre-trained language models.

Top2Vec learns jointly embedded topic, document and word vectors.

An Open-Source Framework for Prompt-Learning.

A curated list of NLP resources focused on BERT, attention mechanism, Transformer networks, and transfer learning.

A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.

Keyphrase or Keyword Extraction 基于预训练模型的中文关键词抽取方法（论文SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-trained Language Model 的中文版代码）

A PyTorch-based model pruning toolkit for pre-trained language models

The code of our paper "SIFRank: A New Baseline for Unsupervised Keyphrase Extraction Based on Pre-trained Language Model"

We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with least artificial participation. The methods we use involve lite pre-training models such as Albert-small or Electra-small with financial corpus, knowledge of distillation and multi-stage learning. The result is that we improve the recall rate of company names recognition task from 0.73 to 0.92 and get 4 times as fast as BERT-Bilstm-CRF model.

Must-read papers on improving efficiency for pre-trained language models.

Code for the ICLR2022 paper "Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners"

Zero-shot Transfer Learning from English to Arabic

Code for EMNLP 2021 main conference paper "Dynamic Knowledge Distillation for Pre-trained Language Models"

Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization (ACL 2021)

Calculating FLOPs of Pre-trained Models in NLP

Zero-shot Transfer Learning from English to Arabic

Code for CascadeBERT, Findings of EMNLP 2021

The source code for the project FormalWriter.com

Usage example for the AllenNLP BiDAF pre-trained model

Question Answering Chatbot with DistilRoBERTa Sentence Embeddings, Dialogflow and Ngrok

Evaluation of the ability of GPT-2 to learn human biases in implicit causality.

2020年贝壳找房问答匹配竞赛

IndoELECTRA: Pre-Trained Language Model for Indonesian Language Understanding

总结NLP中常见的几种预训练语言模型

This repository contains the source code of the paper: "PTMT: Multi-Target Stance Detection with PTM-enhanced Multi-Task Learning"