Evaluating ChatGPT’s Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness

[ACL'24] Official Implementation of the paper "Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphs"(https://aclanthology.org/2024.findings-acl.168)

FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models

Evaluating the faithfulness of long-context language models

Code and data for the ACL 2024 Findings paper "Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning"

The Dataset and Official Implementation for <Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations> @ ACL 2024

About The corresponding code from our paper " Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning" . Do not hesitate to open an issue if you run into any trouble!

Koishi's Day 2024 Paper (NeurIPS 2024): An advanced persona-driven role-playing system with global faithfulness quantification and optimization. In memory of the Koishi's Day of 2024.

Novel data representation leading to granular citations and higher accuracy

A new training framework for Trustworthy Large Reasoning Models

On the evaluation of deep learning interpretability methods for medical images under the scope of faithfulness

[EMNLP 2023] A Causal View of Entity Bias in (Large) Language Models

[ACL 2025] Reranking-based Generation for Unbiased Perspective Summarization

Framework and datasets to evaluate RAG pipelines across relevance, faithfulness, and latency.

The official repo for the EMNLP 2025 paper "NormXLogit: The Head-on-Top Never Lies"

IBM AI explainability

FIFA: Unified Faithfulness Evaluation Framework for Text-to-Video and Video-to-Text Generation

This repository provides a compact RAG evaluation harness tailored for clinical + genomic use cases. It operates on de-identified synthetic notes and curated genomic snippets, measures retrieval quality and grounding/faithfulness, and produces JSONL/CSV/HTML reports.

Clinical + Genomic **RAG evaluation (pro)** with hybrid retrieval (BM25 + embeddings), stronger faithfulness, YAML configs, and HTML dashboards. Python **3.10+**.

LLM explanation tool for Informatics Master's Thesis