ChatEval is a scientific framework for evaluating neural open domain chatbots. Researchers can submit their trained models to effortlessly receive comparisons with baselines and prior work. Since all evaluation code is open source, we ensure evaluation is performed in a standardized and transparent way. Additionally, open source baseline models and an ever growing groups public evaluation sets are available for public use.

Upload Model


How much does ChatEval cost?

ChatEval is free for developers. It is actively developed by researchers at the NLP Group of the University of Pennyslvania.

How is automatic chatbot evaluation performed?

Read more about how automatic evaluation is done here.

How was ChatEval built?

The ChatEval webapp is built using Django and React (front-end) using Magnitude word embeddings format for evaluation. Our source code is available on Github.