Here are 76 public repositories matching this topic...

dkozlov / awesome-knowledge-distillation

Star 2.1k

Awesome Knowledge Distillation

knowledge-distillation teacher-student knowledge-transfer co-training

• Updated Jun 10, 2020

microsoft / NeuronBlocks

Star 1.3k

NLP DNN Toolkit - Building Your NLP DNN Models Like Playing Lego

natural-language-processing deep-learning text-classification dnn pytorch artificial-intelligence question-answering knowledge-distillation sequence-labeling text-matching qna model-compression

• Updated Mar 27, 2020
• Python

szagoruyko / attention-transfer

Star 1.2k

Improving Convolutional Networks via Attention Transfer (ICLR 2017)

deep-learning pytorch attention knowledge-distillation

• Updated Jul 11, 2018
• Jupyter Notebook

huawei-noah / Pretrained-Language-Model

Star 1.1k

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

knowledge-distillation model-compression pretrained-language-model

• Updated Jul 30, 2020
• Python

peterliht / knowledge-distillation-pytorch

Star 775

A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

deep-neural-networks computer-vision pytorch knowledge-distillation cifar10 dark-knowledge model-compression

• Updated Jan 28, 2020
• Python

PaddlePaddle / PaddleSlim

Star 477

PaddleSlim is an open-source library for deep model compression and architecture search.

pruning quantization nas knowledge-distillation evolution-strategy model-compression neural-architecture-search hyperparameter-search autodl

• Updated Aug 3, 2020
• Python

lhyfst / knowledge-distillation-papers

Star 368

knowledge distillation papers

paper reading-list knowledge-distillation dark-knowledge model-compression

• Updated Jun 9, 2020

huawei-noah / Data-Efficient-Model-Compression

Star 320

Data Efficient Model Compression

knowledge-distillation model-compression

• Updated May 8, 2020
• Python

PaddlePaddle / PaddleClas

Star 267

A treasure chest for image classification powered by PaddlePaddle

image-classification knowledge-distillation data-augmentation mixup xception autoaugment res2net mobilenetv3 efficientnet cutmix hrnet randaugment gridmask resnetvd ssld

• Updated Aug 3, 2020
• Python

yuanyuanli85 / Fast_Human_Pose_Estimation_Pytorch

Star 261

Pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419

pytorch knowledge-distillation pose-estimation stacked-hourglass-networks

• Updated Dec 15, 2019
• Python

ilovepose / fast-human-pose-estimation.pytorch

Star 229

Official pytorch Code for CVPR2019 paper "Fast Human Pose Estimation" https://arxiv.org/abs/1811.05419

deep-learning human-pose-estimation mpii-dataset knowledge-distillation mscoco-keypoint coco-keypoints-detection fast-pose-distillation

• Updated Mar 27, 2020
• Cuda

yuanli2333 / Teacher-free-Knowledge-Distillation

Star 225

Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization

pytorch knowledge-distillation paper-implementations label-smoothing teacher-free

• Updated Jul 9, 2020
• Python

lenscloth / RKD

Star 200

Official pytorch Implementation of Relational Knowledge Distillation, CVPR 2019

deep-neural-networks computer-vision deep-learning metric-learning knowledge-distillation

• Updated Dec 9, 2019
• Python

sseung0703 / KD_methods_with_TF

Star 198

Knowledge distillation methods implemented with Tensorflow (now there are 11 (+1) methods, and will be added more.)

tensorflow knowledge-distillation tensorflow-examples

• Updated Nov 21, 2019
• Python

clovaai / overhaul-distillation

Star 189

Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)

knowledge-distillation teacher-student knowledge-transfer network-compression iccv2019

• Updated Jun 23, 2020
• Python

1duo / awesome-ai-infrastructures

Star 181

Infrastructures™ for Machine Learning Training/Inference in Production.

kubernetes machine-learning apache-spark deep-learning artificial-intelligence awesome-list pruning quantization knowledge-distillation deep-learning-framework model-compression apache-arrow federated-learning machine-learning-systems apache-mesos

• Updated May 24, 2019

imirzadeh / Teacher-Assistant-Knowledge-Distillation

Star 171

Using Teacher Assistants to Improve Knowledge Distillation: https://arxiv.org/pdf/1902.03393.pdf

deep-learning knowledge-distillation neural-network-compression

• Updated Oct 3, 2019
• Python

yzhao062 / SUOD

Star 168

An Acceleration System for Large-scale Outlier Detection (Anomaly Detection)

python distributed-systems machine-learning data-mining machine-learning-algorithms machine-learning-library outlier-detection knowledge-distillation anomaly-detection

• Updated Jun 15, 2020
• Python

karanchahal / distiller

Star 136

A large scale study of Knowledge Distillation.

knowledge-distillation pytorch-lightning feature-distillation

• Updated Apr 19, 2020
• Python

cuguilke / microexpnet

Star 101

MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Frontal Face Images

machine-learning deep-neural-networks deep-learning tensorflow knowledge-distillation facial-expression-recognition model-compression

• Updated Jan 16, 2020
• Python

DushyantaDhyani / kdtf

Star 100

Knowledge Distillation using Tensorflow

deep-learning tensorflow knowledge-distillation

• Updated Aug 12, 2019
• Python

lxztju / pytorch_classification

Star 86

利用pytorch实现图像分类的一个完整的代码，训练，预测，TTA，模型融合，模型部署，cnn提取特征，svm或者随机森林等进行分类，模型蒸馏，一个完整的代码

flask deployment random-forest svm cnn pytorch image-classification densenet resnet knn knowledge-distillation resnext label-smoothing

• Updated Jul 25, 2020
• Python

bhheo / AB_distillation

Star 84

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)

transfer-learning knowledge-distillation knowledge-transfer network-compression teacher-student-learning

• Updated Sep 9, 2019
• Python

MingSun-Tse / Collaborative-Distillation

Star 70

PyTorch code for our CVPR-20 paper "Collaborative Distillation for Ultra-Resolution Universal Style Transfer"

knowledge-distillation model-compression neural-style-transfer low-level-vision

• Updated Jul 26, 2020
• Python

d-li14 / DHM

Star 70

[CVPR 2020] Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives

pytorch imagenet pretrained-models knowledge-distillation cvpr2020 deeply-supervised

• Updated Jul 16, 2020
• Python

aquvitae / aquvitae

Star 61

The Easiest Knowledge Distillation Library for Lightweight Deep Learning

machine-learning deep-learning tensorflow pytorch light-weight knowledge-distillation model-compression

• Updated Jun 27, 2020
• Python

Ujjwal-9 / Knowledge-Distillation

Star 53

Blog https://medium.com/neuralmachine/knowledge-distillation-dc241d7c2322

knowledge-distillation

• Updated Jul 5, 2018
• Jupyter Notebook

ChenRocks / Distill-BERT-Textgen

Star 51

Research code for ACL 2020 paper: "Distilling Knowledge Learned in BERT for Text Generation".

natural-language-processing machine-translation knowledge-distillation bert-model

• Updated Jun 25, 2020
• Python

bhheo / BSS_distillation

Star 47

Knowledge Distillation with Adversarial Samples Supporting Decision Boundary (AAAI 2019)

image-classification knowledge-distillation adversarial-samples adversarial-attacks network-compression teacher-student-learning

• Updated Sep 9, 2019
• Python

sseung0703 / Zero-shot_Knowledge_Distillation

Star 43

Zero-Shot Knowledge Distillation in Deep Networks in ICML2019

tensorflow knowledge-distillation zero-shot-learning

• Updated Jun 20, 2019
• Python