evaluation-metrics

(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"

PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).

OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

Evaluate your speech-to-text system with similarity measures such as word error rate (WER)

A Python wrapper for the ROUGE summarization evaluation package

Python wrapper for evaluating summarization quality by ROUGE package

An implementation of a full named-entity evaluation metrics based on SemEval'13 Task 9 - not at tag/token level but considering all the tokens that are part of the named-entity

Code base for the precision, recall, density, and coverage metrics for generative models. ICML 2020.

📈 Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.

CLEval: Character-Level Evaluation for Text Detection and Recognition Tasks

Code for "Semantic Object Accuracy for Generative Text-to-Image Synthesis" (TPAMI 2020)

GOM：New Metric for Re-identification. 👉GOM explicitly balances the effect of performing retrieval and verification into a single unified metric.

Learning to Evaluate Image Captioning. CVPR 2018

Easier Automatic Sentence Simplification Evaluation

Official repository of RankEval: An Evaluation and Analysis Framework for Learning-to-Rank Solutions.

A Neural Framework for MT Evaluation

Resources for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper

It is a Natural Language Processing Problem where Sentiment Analysis is done by Classifying the Positive tweets from negative tweets by machine learning models for classification, text mining, text analysis, data analysis and data visualization

Assessing Generative Models via Precision and Recall (official repository)

Evaluation script for named entity recognition (NER) systems based on entity-level F1 score.

A news recommendation evaluation framework

Useful python NLP tools (evaluation, GUI interface, tokenization)

Python implementation of ROUGE

An image-oriented evaluation tool for image captioning systems (EMNLP-IJCNLP 2019)

Implementation of Evaluation Metrics for Pytorch

A reference-free metric for measuring summary quality, learned from human ratings.

Evaluation metrics for machine-composed symbolic music. Paper: "The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-Composed Music through Quantitative Measures", ISMIR 2020

Open-source evaluation metric for linking Machine Learning model outputs with Business outcomes

Full named-entity (i.e., not tag/token) evaluation metrics based on SemEval’13

quica is a tool to run inter coder agreement pipelines in an easy and effective ways. Multiple measures are run and results are collected in a single table than can be easily exported in Latex