This project compares the results of zero-shot text classification between Deep Java Library (DJL) and Hugging Face's Transformers library in Python. It’s meant to validate a custom DJL translator implementation for zero-shot classification.
We want to make sure our DJL-based zero-shot classification behaves just like the one from Python's Transformers. So, we run the same inputs through both and check if the outputs match. This helps confirm that our ZeroShotClassificationTranslator.java
is working as expected.
Zero-shot classification lets you classify text into categories the model hasn’t explicitly seen before. You don’t need to retrain the model for every new set of labels.
We test and compare the following pre-trained models:
- facebook/bart-large-mnli
- MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli
- tasksource/ModernBERT-base-nli
- MoritzLaurer/bge-m3-zeroshot-v2.0
These are all NLI-based models suitable for zero-shot tasks.
Before using the models with DJL, convert them with the djl-converter
:
djl-convert -m facebook/bart-large-mnli
djl-convert -m MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli
djl-convert -m tasksource/ModernBERT-base-nli
djl-convert -m MoritzLaurer/bge-m3-zeroshot-v2.0
In Main.java
, we:
- Load each model (from local files after conversion)
- Run zero-shot classification on the sentence: "Java is the best programming language"
- Use three candidate labels: "Software Engineering", "Software Programming", "Politics"
- Compare the output scores to those from the Transformers library in Python
- Test both single-label and multi-label modes
In ZeroShotClassificationTranslator.java
, we define how the inputs are tokenized and how results are post-processed for classification.
- Java 21+
- Maven
ai.djl.huggingface:tokenizers:0.32.0
ai.djl.pytorch:pytorch-engine:0.32.0
ai.djl:model-zoo:0.32.0