Publications

2026

1. EMNLP 2026

   Steering the Compass: Aligning Dynamic Psychological Counseling Conversations with Cognitive Behavioral Therapy Strategies

   Zimu Wang, Yiwen Jiang, Xiangyu Zhao, Yaling Shen, and 9 more authors

   The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026), Oct 2026
2. EMNLP 2026

   SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue

   Stephanie Fong, Yiwen Jiang , Zimu Wang, Hongxi Yang, and 11 more authors

   The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026), Oct 2026
3. EMNLP 2026

   GraphRAG-Router: Learning Cost-Efficient Routing over GraphRAGs and LLMs with Reinforcement Learning

   Dongzhe Fan, Chuanhao Ji , Zimu Wang , Tong Chen, and 1 more author

   The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026), Oct 2026

   Abs arXiv

   Graph-based retrieval-augmented generation (GraphRAG) has recently emerged as a powerful paradigm for knowledge-intensive question answering, especially for tasks that require structured evidence organization and multi-hop reasoning. However, existing GraphRAG systems are typically built in a one-size-fits-all manner, relying on a fixed retrieval framework and a single, often large and costly, generator LLM for all queries. This static design limits their ability to adapt to the complexity of varying questions and often incurs unnecessary computational cost. To fill in the gap, we propose GraphRAG-Router, a cost-efficient framework that adopts a hierarchical routing strategy to coordinate heterogeneous GraphRAGs and generator LLMs. Specifically, GraphRAG-Router is first warmed up through supervised fine-tuning and then optimized with a two-stage reinforcement learning procedure, whose second stage introduces a curriculum cost-aware reward to encourage difficulty-aware and economical generator allocation. Extensive experiments on six general-domain and multi-hop QA benchmarks show that GraphRAG-Router consistently outperforms state-of-the-art baselines, reducing the overuse of large LLMs by nearly 30% while maintaining strong generalization capability.

4. EMNLP 2026

   SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment

   Chenxi Wu , Zimu Wang, Haiyang Zhang , Wei Wang, and 1 more author

   The 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP 2026 Industry), Oct 2026
5. EMNLP 2026

   ChronoClinQA: A Longitudinal Multi-Admission Benchmark for Clinical Reasoning over Electronic Health Records

   Tong Chen^* , Zimu Wang^*, Dongzhe Fan , Luoyao Chen, and 8 more authors

   Findings of the Association for Computational Linguistics: EMNLP 2026 (EMNLP 2026 Findings), Oct 2026
6. EMNLP 2026

   SinoGlyphBench: A Diagnostic Benchmark for Chinese Glyph-level Obfuscation in LLM Moderation

   Yifan Wang^* , Zimu Wang^*, Suliu Qin^*, Changyu Zeng, and 9 more authors

   Findings of the Association for Computational Linguistics: EMNLP 2026 (EMNLP 2026 Findings), Oct 2026
7. EMNLP 2026

   VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don’t Mean Preferences

   Yiwen Jiang, Yang Deng, Stephanie Fong , Zimu Wang, and 8 more authors

   Findings of the Association for Computational Linguistics: EMNLP 2026 (EMNLP 2026 Findings), Oct 2026
8. EMNLP 2026

   Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling

   Qingyang Xu, Yaling Shen, Stephanie Fong , Zimu Wang, and 6 more authors

   Findings of the Association for Computational Linguistics: EMNLP 2026 (EMNLP 2026 Findings), Oct 2026

   Abs arXiv

   The increasing use of large language models (LLMs) in mental healthcare raises safety concerns in high-stakes therapeutic interactions. A key challenge is distinguishing therapeutic empathy from maladaptive validation, where supportive responses may inadvertently reinforce harmful beliefs or behaviors in multi-turn conversations. This risk is largely overlooked by existing red-teaming frameworks, which focus mainly on generic harms or optimization-based attacks. To address this gap, we introduce Personality-based Client Simulation Attack (PCSA), the first red-teaming framework that simulates clients in psychological counseling through coherent, persona-driven client dialogues to expose vulnerabilities in psychological safety alignment. Experiments on seven general and mental health-specialized LLMs show that PCSA substantially outperforms four competitive baselines. Perplexity analysis and human inspection further indicate that PCSA generates more natural and realistic dialogues. Our results reveal that current LLMs remain vulnerable to domain-specific adversarial tactics, providing unauthorized medical advice, reinforcing delusions, and implicitly encouraging risky actions.

9. EMNLP 2026

   AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning

   Xiangyu Zhao, Junyu Yan, Yaling Shen , Zimu Wang, and 6 more authors

   Findings of the Association for Computational Linguistics: EMNLP 2026 (EMNLP 2026 Findings), Oct 2026

   Abs arXiv

   Large audio-language models (LALMs) increasingly use explicit reasoning traces for complex audio understanding, yet the evaluation of reasoning quality remains underexplored. Although process-level benchmarks for process reward models (PRMs) have advanced reasoning evaluation in text and multi-modal domains, comparable evaluation for audio reasoning remains limited. In this paper, we present AudioProcessBench, a comprehensive benchmark for step-level process error identification in audio reasoning. AudioProcessBench contains diverse reasoning traces generated by 6 audio and omni language models. Each trace is segmented into discrete reasoning steps and annotated with binary step correctness and fine-grained error types. Our benchmark evaluates models under three complementary paradigms: (1) step correctness identification, (2) error-type-conditioned detection for diagnosing audio-specific verifier capacities, and (3) chain-level aggregation, where verifiers select or aggregate among multiple reasoning traces for the same question. This design enables a systematic analysis of whether current models can detect process errors, whether their weaknesses differ across audio-specific error types, and whether process verification translates into improved answer selection. AudioProcessBench provides a testbed for future research on audio reasoning verifiers, process reward models, and reliable omni-modal reasoning.

10. PR

    A benchmark and method for photographed table reasoning

    Xiaoqiang Kang , Zimu Wang, Xiaochen Zi, Xiaobo Jin, and 3 more authors

    Pattern Recognition, Oct 2026

    Abs HTML

    With the advancement of large language models (LLMs) and multimodal LLMs (MLLMs), table reasoning has achieved significant progress. However, most existing works focus predominantly on textual or rendered tables, which differ substantially from real-world photographed scenarios under suboptimal conditions such as uneven lighting, blur, and tilted perspectives. This discrepancy creates a significant performance gap for current MLLMs, limiting their applicability in real-world scenarios. To address this critical limitation, we introduce the first comprehensive study on multimodal reasoning for photographed tables. We present a new dataset Photographed Chinese Table Reasoning (PCTR-16K), which contains 4989 photographed tables and 16,318 questions across 9 subjects and 3 difficulty levels. This dataset serves as the first benchmark specifically designed for evaluating reasoning capabilities on tables captured under authentic conditions. To enhance MLLMs’ reasoning over photographed tables, we propose Structure-aware Chain-of-Thought (SCoT), a method that unifies table recognition and reasoning into a single end-to-end generative process. To bolster the structural perception required by SCoT, we further incorporate seven auxiliary table structure understanding (TSU) tasks during fine-tuning. These tasks provide fine-grained supervision across multiple dimensions of table layout and semantics. Extensive experiments on various MLLMs demonstrate that our proposed SCoT and multi-view TSU tasks significantly enhance recognition and reasoning capabilities on photographed tables. For example, LLaVA-Llama3.1 achieves an absolute improvement of 19.53% on the PCTR-16K benchmark (from 44.58% to 64.11%), demonstrating its effectiveness for real-world photographed table reasoning. The dataset is publicly available at https://github.com/PremiLab-Math/PCTR-16k.

11. MICCAI 2026

    Skin2Mind: A Multimodal Framework for Mental Health Risk Screening via Facial Images and Structured Skin Reports

    Jiahe Liu, Duong Nhu, Xiangyu Zhao, Yiwen Jiang, and 12 more authors

    The 29th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2026), Sep 2026
12. ICIC 2026

    Aletheia: A Two-Stage Graph-Based Framework for Hallucination Detection in Abstractive Summarization

    Tianshi Cai , Guanxu Li , Zimu Wang, Changyu Zeng, and 6 more authors

    2026 International Conference on Intelligent Computing (ICIC 2026), Jul 2026
13. ACL 2026

    You Never Know a Person, You Only Know Their Defenses: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

    Hongbin Na^* , Zimu Wang^* , Zhaoming Chen^*, Peilin Zhou, and 8 more authors

    Findings of the Association for Computational Linguistics: ACL 2026 (ACL 2026 Findings), Jul 2026

    Abs arXiv

    Psychological defenses are strategies, often automatic, that people use to manage distress. Rigid or overuse of defenses is negatively linked to mental health and shapes what speakers disclose and how they accept or resist help. However, defenses are complex and difficult to reliably measure, particularly in clinical dialogues. We introduce PsyDefConv, a dialogue corpus with help seeker utterances labeled for defense level, and DMRS Co-Pilot, a four-stage pipeline that provides evidence-based pre-annotations. The corpus contains 200 dialogues and 4709 utterances, including 2336 help seeker turns, with labeling and Cohen’s kappa 0.639. In a counterbalanced study, the co-pilot reduced average annotation time by 22.4%. In expert review, it averaged 4.62 for evidence, 4.44 for clinical plausibility, and 4.40 for insight on a seven-point scale. Benchmarks with strong language models in zero-shot and fine-tuning settings demonstrate clear headroom, with the best macro F1-score around 30% and a tendency to overpredict mature defenses. Corpus analyses confirm that mature defenses are most common and reveal emotion-specific deviations. We will release the corpus, annotations, code, and prompts to support research on defensive functioning in language.

14. ACL 2026

    PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics

    Yaling Shen, Stephanie Fong, Yiwen Jiang , Zimu Wang, and 8 more authors

    Findings of the Association for Computational Linguistics: ACL 2026 (ACL 2026 Findings), Jul 2026

    Abs arXiv

    The increasing integration of large language models (LLMs) into mental health applications necessitates robust frameworks for evaluating professional safety alignment. Current evaluative approaches primarily rely on refusal-based safety signals, which offer limited insight into the nuanced behaviors required in clinical practice. In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking. To address this gap, we move beyond refusal-centric metrics and introduce PsychEthicsBench, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines, designed to evaluate LLMs’ ethical knowledge and behavioral responses through multiple-choice and open-ended tasks with fine-grained ethicality annotations. Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness. Notably, we find that domain-specific fine-tuning can degrade ethical robustness, as several specialized models underperform their base backbones in ethical alignment. PsychEthicsBench provides a foundation for systematic, jurisdiction-aware evaluation of LLMs in mental health, encouraging more responsible development in this domain.

15. WS 2026

    Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations

    Hongbin Na^* , Zimu Wang^* , Zhaoming Chen^*, Yining Hua, and 7 more authors

    The 25th Workshop on Biomedical Language Processing (BioNLP@ACL 2026), Jul 2026

    Abs arXiv

    We present an overview of PsyDefDetect, the shared task on detecting levels of psychological defense mechanisms in emotional support dialogues, co-located with BioNLP@ACL 2026. Grounded in the clinically validated Defense Mechanism Rating Scales (DMRS) framework, the task asks systems to classify a target seeker utterance, given its preceding dialogue context, into one of nine categories: seven hierarchical DMRS levels plus two auxiliary labels. Participants worked on PsyDefConv, a newly released corpus of 200 dialogues and 2336 help-seeker utterances annotated under DMRS with substantial inter-annotator agreement. The task attracted 172 participants on CodaBench who produced 563 submissions, with 21 teams officially registering their results for the final ranking. The best system achieved a macro F1-score of 0.420, surpassing the strongest fine-tuned baseline reported in the dataset paper by a notable margin, yet leaving clear headroom. Our analysis highlights (i) a persistent tendency to over-predict the majority High-Adaptive class, (ii) a widening gap between accuracy and macro-F1 that reveals class-imbalance sensitivity, and (iii) the value of theory-aware and LLM-based approaches for fine-grained defensive-function classification. We release all task materials and invite the community to continue work on this novel intersection of clinical psychology and NLP.

16. Preprint

    Herculean: An Agentic Benchmark for Financial Intelligence

    Xueqing Peng, Zhuohan Xie, Yupeng Cao , Haohang Li, and 60 more authors

    arXiv preprint arXiv:2605.14355, May 2026

    Abs arXiv

    As AI agents improve, the central question is no longer whether they can solve isolated well-defined financial tasks, but whether they can reliably carry out financial professional work. Existing financial benchmarks offer only a partial view of this ability, as they primarily evaluate static competencies such as question answering, retrieval, summarization, and classification. We introduce Herculean, the first skilled benchmark for agentic financial intelligence spanning four representative workflows, including Trading, Hedging, Market Insights, and Auditing. Each workflow is instantiated as a standardized MCP-based skill environment with its own tools, interaction dynamics, constraints, and success criteria, enabling consistent end-to-end assessment of heterogeneous agent systems. Across frontier agents, we find agents perform relatively well on Trading and Market Insights, but struggle substantially on Hedging and Auditing, where long-horizon coordination, state consistency, and structured verification are critical. Overall, our results point to a key gap in current agents in turning financial reasoning into dependable workflow execution in high-stakes financial workflows.

17. CSCWD 2026

    Trust the Eye or the Crowd? Unveiling Conformity in Multimodal Large Language Models

    Haoran Luo^* , Tong Chen^*, Yuhan Niu^* , Hengxian Liu, and 12 more authors

    2026 29th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2026), May 2026
18. LREC 2026

    MEUR: A Benchmark for Evaluating Vision-Language Models on Multimodal Event Understanding and Reasoning

    Zimu Wang , Yuqi Wang , Tong Chen, Changyu Zeng, and 11 more authors

    The 15th Language Resources and Evaluation Conference (LREC 2026), May 2026
19. LREC 2026

    TCMPHal: A Large-scale Dataset for Hallucination Detection in Traditional Chinese Medicine Pharmacy

    Nijia Han , Zimu Wang, Ziwen Xie , Wei Wang, and 3 more authors

    The 15th Language Resources and Evaluation Conference (LREC 2026), May 2026
20. EACL 2026

    CHiRPE: A Step Towards Real-World Clinical NLP with Clinician-Oriented Model Explanations

    Stephanie Fong , Zimu Wang, Guilherme C Oliveira, Xiangyu Zhao, and 8 more authors

    The 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026), Mar 2026

    Abs arXiv

    The medical adoption of NLP tools requires interpretability by end users, yet traditional explainable AI (XAI) methods are misaligned with clinical reasoning and lack clinician input. We introduce CHiRPE (Clinical High-Risk Prediction with Explainability), an NLP pipeline that takes transcribed semi-structured clinical interviews to: (i) predict psychosis risk; and (ii) generate novel SHAP explanation formats co-developed with clinicians. Trained on 944 semi-structured interview transcripts across 24 international clinics of the AMP-SCZ study, the CHiRPE pipeline integrates symptom-domain mapping, LLM summarisation, and BERT classification. CHiRPE achieved over 90% accuracy across three BERT variants and outperformed baseline models. Explanation formats were evaluated by 28 clinical experts who indicated a strong preference for our novel concept-guided explanations, especially hybrid graph-and-text summary formats. CHiRPE demonstrates that clinically-guided model development produces both accurate and interpretable results. Our next step is focused on real-world testing across our 24 international sites.

2025

1. BICS 2025

   Assessing the Working Memory Capacity of Large Language Models through Reading Span Tasks: An Empirical Study

   Qi Li, Jingshi Zhou^† , Zimu Wang , and Xiaojun Zhang^†

   The 15th International Conference on Brain Inspired Cognitive Systems (BICS 2025), Nov 2025

   Abs

   Working memory is central to language comprehension and reasoning, involving the dynamic balance between information processing and storage. This study investigates whether large language models exhibit human-like working memory behavior through the reading span task. Four LLMs were evaluated under varying conditions of task type, sentence complexity, and prompting strategy. Results reveal that across models, recognition tasks consistently outperformed recall tasks, mirroring human performance asymmetries due to differences in retrieval load. Garden-path sentences induced lower accuracy and increased processing costs, reflecting a human-like processing-storage trade-off under syntactic ambiguity. Moreover, Chain-of-Thought prompting selectively enhanced performance in high-demand contexts but marginally hindered performance in simpler conditions, while Chinese prompts generally reduced accuracy, likely due to cross-linguistic interference and data imbalance. These findings indicate that LLMs demonstrated emergent WM-like patterns shaped by processing difficulty, task structure, and prompt framing. While their WM capacity exceeds human limits, their sensitivity to varying conditions highlights parallels with human working memory mechanisms. The study contributes to understanding the cognitive implications of attention-based architectures and underscores the importance of prompt design in influencing their reasoning and memory capacities.

2. Preprint

   It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models

   Xiangyu Zhao, Yaling Shen, Yiwen Jiang , Zimu Wang, and 6 more authors

   arXiv preprint arXiv:2511.19877, Nov 2025

   Abs arXiv

   Depression is one of the most prevalent mental health disorders globally. In recent years, multi-modal data, such as speech, video, and transcripts, has been increasingly used to develop AI-assisted depression assessment systems. Large language models have further advanced this field due to their strong language understanding and generalization capabilities. However, conventional LLMs remain text-centric and cannot process the rich non-verbal cues found in audio and visual modalities, which are critical components in mental health evaluation. While multi-modal LLMs offer a promising direction, few are tailored for psychological applications. In this study, we propose a novel multi-modal LLM framework for depression detection. Our approach augments an audio language model with visual understanding and aligns audio-visual features at the timestamp level. This fine-grained alignment improves modeling of temporal dynamics across modalities while reducing the need for extensive training data and computational resources. Experiments on the DAIC-WoZ dataset demonstrate that our model outperforms both single-modality approaches and previous multi-modal methods. Moreover, the proposed framework can be extended to incorporate additional physiological signals, paving the way for broader clinical applications beyond mental health.

3. ALTA 2025

   Thinker-DDM: Modeling Deliberation for Machine Translation with a Drift-Diffusion Process

   Hongbin Na^* , Zimu Wang^*, Mieradilijiang Maimaiti^† , Tong Chen, and 3 more authors

   The 23rd Annual Workshop of the Australasian Language Technology Association (ALTA 2025), Nov 2025

   Abs arXiv HTML PDF

   Large language models (LLMs) have demonstrated promising potential in various downstream tasks, including machine translation. However, prior work on LLM-based machine translation has mainly focused on better utilizing training data, demonstrations, or pre-defined and universal knowledge to improve performance, with a lack of consideration of decision-making like human translators. In this paper, we incorporate Thinker with the Drift-Diffusion Model (Thinker-DDM) to address this issue. We then redefine the Drift-Diffusion process to emulate human translators’ dynamic decision-making under constrained resources. We conduct extensive experiments under the high-resource, low-resource, and commonsense translation settings using the WMT22 and CommonMT datasets, in which Thinker-DDM outperforms baselines in the first two scenarios. We also perform additional analysis and evaluation on commonsense translation to illustrate the high effectiveness and efficacy of the proposed method.

4. EMNLP 2025

   LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research

   Shuo Yan^* , Ruochen Li^*, Ziming Luo^* , Zimu Wang^*, and 10 more authors

   The 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), Nov 2025

   Abs arXiv HTML PDF

   Large language model (LLM) agents have demonstrated remarkable potential in advancing scientific discovery. However, their capability in the fundamental yet crucial task of reproducing code from research papers, especially in the NLP domain, remains underexplored. This task includes unique complex reasoning challenges in the intellectual synthesis of abstract concepts and the comprehension of code repositories with interdependent files. Motivated by this gap, we present LMR-BENCH, a benchmark designed to systematically evaluate the capability of LLM agents on code reproduction from Language Modeling Research. It consists of 28 code reproduction tasks derived from 23 research papers published in top-tier NLP venues over the past five years, spanning nine fundamental categories. Models are provided with a research paper, a code repository containing one or more masked functions, and instructions for implementing these functions. We conduct extensive experiments in standard prompting and LLM agent settings with state-of-the-art LLMs, evaluating the accuracy of unit tests and performing LLM-based evaluation of code correctness. Experimental results reveal that even the most advanced models still exhibit persistent limitations in scientific reasoning and code synthesis, highlighting critical gaps in LLM agents’ ability to autonomously reproduce scientific research.

5. EMNLP 2025

   MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses

   Tong Chen^* , Zimu Wang^*, Yiyi Miao, Haoran Luo, and 5 more authors

   The 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), Nov 2025

   Abs arXiv HTML PDF

   Medical fact-checking has become increasingly critical as more individuals seek medical information online. However, existing datasets predominantly focus on human-generated content, leaving the verification of content generated by large language models (LLMs) relatively unexplored. To address this gap, we introduce MedFact, the first evidence-based Chinese medical fact-checking dataset of LLM-generated medical content. It consists of 1,321 questions and 7,409 claims, mirroring the complexities of real-world medical scenarios. We conduct comprehensive experiments in both in-context learning (ICL) and fine-tuning settings, showcasing the capability and challenges of current LLMs on this task, accompanied by an in-depth error analysis to point out key directions for future research. Our dataset is publicly available at https://github.com/AshleyChenNLP/MedFact.

6. EMNLP 2025

   Can GRPO Boost Complex Multimodal Table Understanding?

   Xiaoqiang Kang, Shengen Wu , Zimu Wang , Yilin Liu, and 6 more authors

   The 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), Nov 2025

   Abs arXiv HTML PDF

   Existing table understanding methods face challenges due to complex table structures and intricate logical reasoning. While supervised finetuning (SFT) dominates existing research, reinforcement learning (RL), such as Group Relative Policy Optimization (GRPO), has shown promise but struggled with low initial policy accuracy and coarse rewards in tabular contexts. In this paper, we introduce Table-R1, a three-stage RL framework that enhances multimodal table understanding through: (1) Warm-up that prompts initial perception and reasoning capabilities, (2) Perception Alignment GRPO (PA-GRPO), which employs continuous Tree-Edit-Distance Similarity (TEDS) rewards for recognizing table structures and contents, and (3) Hint-Completion GRPO (HC-GRPO), which utilizes fine-grained rewards of residual steps based on the hint-guided question. Extensive experiments demonstrate that Table-R1 can boost the model’s table reasoning performance obviously on both held-in and held-out datasets, outperforming SFT and GRPO largely. Notably, Qwen2-VL-7B with Table-R1 surpasses larger specific table understanding models (e.g., Table-LLaVA 13B), even achieving comparable performance to the closed-source model GPT-4o on held-in datasets, demonstrating the efficacy of each stage of Table-R1 in overcoming initialization bottlenecks and reward sparsity, thereby advancing robust multimodal table understanding.

7. EMNLP 2025

   WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification

   Yiwen Jiang, Deval Mehta, Siyuan Yan, Yaling Shen, and 2 more authors

   The 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025), Nov 2025

   Abs arXiv HTML PDF

   Multimodal Large Language Models (MLLMs) have shown promise in visual-textual reasoning, with Multimodal Chain-of-Thought (MCoT) prompting significantly enhancing interpretability. However, existing MCoT methods rely on rationale-rich datasets and largely focus on inter-object reasoning, overlooking the intra-object understanding crucial for image classification. To address this gap, we propose WISE, a Weak-supervision-guided Step-by-step Explanation method that augments any image classification dataset with MCoTs by reformulating the concept-based representations from Concept Bottleneck Models (CBMs) into concise, interpretable reasoning chains under weak supervision. Experiments across ten datasets show that our generated MCoTs not only improve interpretability by 37% but also lead to gains in classification accuracy when used to fine-tune MLLMs. Our work bridges concept-based interpretability and generative MCoT reasoning, providing a generalizable framework for enhancing MLLMs in fine-grained visual understanding.

8. EMNLP 2025

   Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement

   Haotan Guo^*, Jianfei He^*, Jiayuan Ma^*, Hongbin Na^†, and 7 more authors

   The 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP 2025 Industry), Nov 2025

   Abs arXiv HTML PDF

   Phonetic Cloaking Replacement (PCR), defined as the deliberate use of homophonic or near-homophonic variants to hide toxic intent, has become a major obstacle to Chinese content moderation. While this problem is well-recognized, existing evaluations predominantly rely on rule-based, synthetic perturbations that ignore the creativity of real users. We organize PCR into a four-way surface-form taxonomy and compile PCR-ToxiCN, a dataset of 500 naturally occurring, phonetically cloaked offensive posts gathered from the RedNote platform. Benchmarking state-of-the-art LLMs on this dataset exposes a serious weakness: the best model reaches only an F1-score of 0.672, and zero-shot chain-of-thought prompting pushes performance even lower. Guided by error analysis, we revisit a Pinyin-based prompting strategy that earlier studies judged ineffective and show that it recovers much of the lost accuracy. This study offers the first comprehensive taxonomy of Chinese PCR, a realistic benchmark that reveals current detectors’ limits, and a lightweight mitigation technique that advances research on robust toxicity detection.

9. EMNLP 2025

   NUMINA: A Natural Understanding Benchmark for Multi-dimensional Intelligence and Numerical Reasoning Abilities

   Changyu Zeng^* , Yifan Wang^* , Zimu Wang^* , Wei Wang, and 5 more authors

   Findings of the Association for Computational Linguistics: EMNLP 2025 (EMNLP 2025 Findings), Nov 2025

   Abs arXiv HTML PDF

   Recent advancements in 2D multimodal large language models (MLLMs) have significantly improved performance in vision-language tasks. However, extending these capabilities to 3D environments remains a distinct challenge due to the complexity of spatial reasoning. Nevertheless, existing 3D benchmarks often lack fine-grained numerical reasoning task annotations, limiting MLLMs’ ability to perform precise spatial measurements and complex numerical reasoning. To address this gap, we introduce NUMINA, the first Natural Understanding benchmark for Multi-dimensional Intelligence and Numerical reasoning Abilities to enhance multimodal indoor perceptual understanding. NUMINA features multi-scale annotations and various question-answer pairs, generated using NUMINA-Flow, an automated annotation pipeline that integrates LLM rewriting and rule-based self-verification. We evaluate the performance of various state-of-the-art LLMs on NUMINA following the Chat-Scene framework, demonstrating that current LLMs struggle with multimodal numerical reasoning, particularly in performing precise computations such as distance and volume estimation, highlighting the need for further advancements in 3D models. The dataset and source codes can be obtained from https://github.com/fengshun124/NUMINA

10. WS 2025

    FinDebate: Multi-Agent Collaborative Intelligence for Financial Analysis

    Tianshi Cai^* , Guanxu Li^*, Nijia Han^*, Ce Huang, and 9 more authors

    The 10th Workshop on Financial Technology and Natural Language Processing (FinNLP@EMNLP 2025), Nov 2025

    Abs arXiv HTML PDF

    We introduce FinDebate, a multi-agent framework for financial analysis, integrating collaborative debate with domain-specific Retrieval-Augmented Generation (RAG). Five specialized agents, covering earnings, market, sentiment, valuation, and risk, run in parallel to synthesize evidence into multi-dimensional insights. To mitigate overconfidence and improve reliability, we introduce a safe debate protocol that enables agents to challenge and refine initial conclusions while preserving coherent recommendations. Experimental results, based on both LLM-based and human evaluations, demonstrate the framework’s efficacy in producing high-quality analysis with calibrated confidence levels and actionable investment strategies across multiple time horizons.

11. INLG 2025

    Benchmarking and Improving LVLMs on Event Extraction from Multimedia Documents

    Fuyu Xing^* , Zimu Wang^* , Wei Wang, and Haiyang Zhang^†

    The 18th International Natural Language Generation Conference (INLG 2025), Oct 2025

    Abs arXiv HTML PDF

    The proliferation of multimedia content necessitates the development of effective Multimedia Event Extraction (M2E2) systems. Though Large Vision-Language Models (LVLMs) have shown strong cross-modal capabilities, their utility in the M2E2 task remains underexplored. In this paper, we present the first systematic evaluation of representative LVLMs, including DeepSeek-VL2 and the Qwen-VL series, on the M2E2 dataset. Our evaluations cover text-only, image-only, and cross-media subtasks, assessed under both few-shot prompting and fine-tuning settings. Our key findings highlight the following valuable insights: (1) Few-shot LVLMs perform notably better on visual tasks but struggle significantly with textual tasks; (2) Fine-tuning LVLMs with LoRA substantially enhances model performance; and (3) LVLMs exhibit strong synergy when combining modalities, achieving superior performance in cross-modal settings. We further provide a detailed error analysis to reveal persistent challenges in areas such as semantic precision, localization, and cross-modal grounding, which remain critical obstacles for advancing M2E2 capabilities.

12. ECAI 2025

    FTCFormer: Fuzzy Token Clustering Transformer for Image Classification

    Muyi Bao^*, Changyu Zeng^* , Yifan Wang, Zhengni Yang, and 4 more authors

    The 28th European Conference on Artificial Intelligence (ECAI 2025), Oct 2025

    Abs arXiv HTML

    Transformer-based deep neural networks have achieved remarkable success across various computer vision tasks, largely attributed to their long-range self-attention mechanism and scalability. However, most transformer architectures embed images into uniform, grid-based vision tokens, neglecting the underlying semantic meanings of image regions, resulting in suboptimal feature representations. To address this issue, we propose Fuzzy Token Clustering Transformer (FTCFormer), which incorporates a novel clustering-based downsampling module to dynamically generate vision tokens based on the semantic meanings instead of spatial positions. It allocates fewer tokens to less informative regions and more to represent semantically important regions, regardless of their spatial adjacency or shape irregularity. To further enhance feature extraction and representation, we propose a Density Peak Clustering-Fuzzy K-Nearest Neighbor (DPC-FKNN) mechanism for clustering center determination, a Spatial Connectivity Score (SCS) for token assignment, and a channel-wise merging (Cmerge) strategy for token merging. Extensive experiments on 32 datasets across diverse domains validate the effectiveness of FTCFormer on image classification, showing consistent improvements over the TCFormer baseline, achieving gains of improving 1.43% on five fine-grained datasets, 1.09% on six natural image datasets, 0.97% on three medical datasets and 0.55% on four remote sensing datasets. The code is available at: https://github.com/BaoBao0926/FTCFormer/tree/main.

13. Preprint

    Disentangling the Drivers of LLM Social Conformity: An Uncertainty-Moderated Dual-Process Mechanism

    Huixin Zhong^* , Yanan Liu^*†, Qi Cao , Shijin Wang, and 3 more authors

    arXiv preprint arXiv:2508.14918, Aug 2025

    Abs arXiv

    As large language models (LLMs) integrate into collaborative teams, their social conformity – the tendency to align with majority opinions – has emerged as a key concern. In humans, conformity arises from informational influence (rational use of group cues for accuracy) or normative influence (social pressure for approval), with uncertainty moderating this balance by shifting from purely analytical to heuristic processing. It remains unclear whether these human psychological mechanisms apply to LLMs. This study adapts the information cascade paradigm from behavioral economics to quantitatively disentangle the two drivers to investigate the moderate effect. We evaluated nine leading LLMs across three decision-making scenarios (medical, legal, investment), manipulating information uncertainty (q = 0.667, 0.55, and 0.70, respectively). Our results indicate that informational influence underpins the models’ behavior across all contexts, with accuracy and confidence consistently rising with stronger evidence. However, this foundational mechanism is dramatically modulated by uncertainty. In low-to-medium uncertainty scenarios, this informational process is expressed as a conservative strategy, where LLMs systematically underweight all evidence sources. In contrast, high uncertainty triggers a critical shift: while still processing information, the models additionally exhibit a normative-like amplification, causing them to overweight public signals (beta > 1.55 vs. private beta = 0.81).

14. CCL 2025

    Unveiling the Linguistic Acceptability Judgments of Large Language Models in Multilingual Contexts

    Fuyu Xing, Haoyu Huang, Dawei Mo, Xinzhuo Yang, and 4 more authors

    The 24th China National Conference on Computational Linguistics (CCL 2025), Aug 2025

    Abs HTML PDF

    Linguistic acceptability judgments are essential for evaluating how language models internalize human-like grammatical knowledge. Though some studies have evaluated large language models (LLMs) in this context, existing research lacks systematic exploration of diverse learning paradigms in a multilingual setting. In this paper, we present the first multilingual evaluation of LLMs across four languages (English, Chinese, Japanese, and Russian) in the field of linguistic acceptability. Our evaluation spans both general-purpose (i.e., GPT-4o, GPT-4o mini, DeepSeek-V3, GLM-4-32B, and the Qwen series) and reasoning-oriented (QwQ-32B-Preview and DeepSeek-R1-32B) models under zero-shot and monolingual, cross-lingual and multilingual fine-tuning settings, with comparisons to pre-trained language model (PLM) baselines. Our analysis highlights the strong generalizability of large-scale LLMs through zero-shot prompting, the challenges of fine-tuning small-sized LLMs with skewed training data, the effectiveness of multilingual fine-tuning for low-resource languages, the scaling law exhibited on the task, and the limitation of reasoning-oriented models on the task, even when “aha moments” occur during the reasoning process.

15. ACL 2025

    A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directions

    Hongbin Na^*, Yining Hua^* , Zimu Wang^*, Tao Shen, and 5 more authors

    Findings of the Association for Computational Linguistics: ACL 2025 (ACL 2025 Findings), Jul 2025

    Abs arXiv HTML PDF

    Mental health remains a critical global challenge, with increasing demand for accessible, effective interventions. Large language models (LLMs) offer promising solutions in psychotherapy by enhancing the assessment, diagnosis, and treatment of mental health conditions through dynamic, context-aware interactions. This survey provides a comprehensive overview of the current landscape of LLM applications in psychotherapy, highlighting the roles of LLMs in symptom detection, severity estimation, cognitive assessment, and therapeutic interventions. We present a novel conceptual taxonomy to organize the psychotherapy process into three core components: assessment, diagnosis, and treatment, and examine the challenges and advancements in each area. The survey also addresses key research gaps, including linguistic biases, limited disorder coverage, and underrepresented therapeutic models. Finally, we discuss future directions to integrate LLMs into a holistic, end-to-end psychotherapy framework, addressing the evolving nature of mental health conditions and fostering more inclusive, personalized care.

16. CSCWD 2025

    MMKNet: A Multi-Modal Knowledge Network for Predicting Both Seen and Unseen Classes in Medical Imaging

    Wenqi Xu, Hong-Seng Gan^†, Shengen Wu , Zimu Wang, and 2 more authors

    2025 28th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2025), May 2025

    Abs HTML

    Generalized zero-shot learning (ML-GZSL) has demonstrated significant potential in medical diagnostics due to doctors’ need to process large volumes of medical images. Vision Transformers (ViTs), due to their Transformer-like structure, are considered to have superior feature-generation capabilities in cross-text-image tasks. BioMedBERT, based on the BERT architecture and domain-specific pre-training for biomedical natural language processing, is considered to have significant label embedding capabilities in cross-text-image tasks. In this paper, we propose MMKNet, a novel method that employs ViTs to construct global and local features of images for visual knowledge from images while using BioMedBERT with prompt tuning for the label embedding to achieve the knowledge from textual embedding in biomedical corpora. To integrate multi-modal information, we design a unique combined decision layer, which outputs similarity scores between images and class labels, providing the predicted classifications. Our method is class-independent during inference, enabling the model to predict unseen classes. Experiments on the NIH-ChestXray14, Kaggle retina, and Multi-Label Retinal Diseases (MuReD) datasets demonstrate that our method outperforms baseline models across multiple performance metrics, which can potentially optimize doctors’ workflows by allowing them to focus on diagnosing complex cases, addressing challenges of limited dataset sizes and incomplete disease coverage in the medical imaging domain.

17. CSCWD 2025

    Skin Disease Classification with LVLMs: An Empirical Study

    Xinyi Zeng , Zimu Wang, Haiyang Zhang^†, Yiming Luo, and 1 more author

    2025 28th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2025), May 2025

    Abs HTML

    Skin diseases pose significant challenges to accurate and efficient diagnosis, often due to their diverse and complex representations. This study investigates the capabilities and limitations of Large Vision-Language Models (LVLMs) in addressing these challenges through skin disease classification tasks. We evaluated LVLMs in zero-shot, few-shot, and finetuning scenarios, exploring their performance, bias, and potential for improvement. Results show that LVLMs lack perceptual granularity in skin disease, though positive signals are also observed. Our findings underscore the necessity for domain- specific optimisation and highlight opportunities for advancing LVLMs in medical diagnostics through innovative strategies and collaborative efforts.

18. WS 2025

    From Posts to Timelines: Modeling Mental Health Dynamics from Social Media Timelines with Hybrid LLMs

    Zimu Wang, Hongbin Na, Rena Gao, Jiayuan Ma, and 3 more authors

    The 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych@NAACL 2025), Apr 2025

    Abs HTML PDF

    Social media data is recognized for its usefulness in the early detection of mental disorders; however, there is a lack of research focused on modeling individuals’ longitudinal mental health dynamics. Moreover, fine-tuning large language models (LLMs) on large-scale, annotated datasets presents challenges due to privacy concerns and the difficulties on data collection and annotation. In this paper, we propose a novel approach for modeling mental health dynamics using hybrid LLMs, where we first apply both classification-based and generation-based models to identify adaptive and maladaptive evidence from individual posts. This evidence is then used to predict well-being scores and generate post-level and timeline-level summaries. Experimental results on the CLPsych 2025 shared task demonstrate the effectiveness of our method, with the generative-based model showing a marked advantage in evidence identification.

19. WS 2025

    Efficient Document-level Event Relation Extraction

    Ruochen Li , Zimu Wang, and Xinya Du

    The 10th Workshop on Representation Learning for NLP (RepL4NLP@NAACL 2025), Apr 2025

    Abs HTML PDF

    Event Relation Extraction (ERE) predicts temporal and causal relationships between events, playing a crucial role in constructing comprehensive event knowledge graphs. However, existing approaches based on pairwise comparisons often suffer from computational inefficiency, particularly at the document level, due to the quadratic operations required. Additionally, the predominance of unrelated events also leads to largely skewed data distributions. In this paper, we propose an innovative two-stage framework to tackle the challenges, consisting of a retriever to identify the related event pairs and a cross-encoder to classify the relationships between the retrieved pairs. Evaluations across representative benchmarks demonstrate our approach achieves better efficiency and significantly better performance. We also investigate leveraging event coreference chains for ERE and demonstrate their effectiveness.

20. WIREs DMKD

    A Review on Medical Image Segmentation: Datasets, Technical Models, Challenges and Solutions

    Hong-Seng Gan, Muhammad Hanif Ramlee , Zimu Wang, and Akinobu Shimizu

    WIREs Data Mining and Knowledge Discovery, Mar 2025

    Abs HTML

    Medical image segmentation is prerequisite in computer-aided diagnosis. As the field experiences tremendous paradigm changes since the introduction of foundation models, technicality of deep medical segmentation model is no longer a privilege limited to computer science researchers. A comprehensive educational resource suitable for researchers of broad, different backgrounds such as biomedical and medicine, is needed. This review strategically covers the evolving trends that happens to different fundamental components of medical image segmentation such as the emerging of multimodal medical image datasets, updates on deep learning libraries, classical-to-contemporary development in deep segmentation models and latest challenges with focus on enhancing the interpretability and generalizability of model. Last, the conclusion section highlights on future trends in deep medical segmentation that worth further attention and investigations.

21. AAAI 2025

    Template-Driven LLM-Paraphrased Framework for Tabular Math Word Problem Generation

    Xiaoqiang Kang^* , Zimu Wang^*, Xiaobo Jin , Wei Wang, and 2 more authors

    The 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025), Feb 2025

    Abs arXiv HTML

    Solving tabular math word problems (TMWPs) has become a critical role in evaluating the mathematical reasoning ability of large language models (LLMs), where large-scale TMWP samples are commonly required for LLM fine-tuning. Since the collection of high-quality TMWP datasets is costly and time-consuming, recent research has concentrated on automatic TMWP generation. However, current generated samples usually suffer from issues of either correctness or diversity. In this paper, we propose a Template-driven LLM-paraphrased (TeLL) framework for generating high-quality TMWP samples with diverse backgrounds and accurate tables, questions, answers, and solutions. To this end, we first extract templates from existing real samples to generate initial problems, ensuring correctness. Then, we adopt an LLM to extend templates and paraphrase problems, obtaining diverse TMWP samples. Furthermore, we find the reasoning annotation is important for solving TMWPs. Therefore, we propose to enrich each solution with illustrative reasoning steps. Through the proposed framework, we construct a high-quality dataset TabMWP-TeLL by adhering to the question types in the TabMWP dataset, and we conduct extensive experiments on a variety of LLMs to demonstrate the effectiveness of TabMWP-TeLL in improving TMWP solving performance. The code and data of this paper are available at: https://github.com/Jason8Kang/TELL.

22. COLING 2025

    Detecting Conversational Mental Manipulation with Intent-Aware Prompting

    Jiayuan Ma^*, Hongbin Na^* , Zimu Wang, Yining Hua, and 3 more authors

    The 31st International Conference on Computational Linguistics (COLING 2025), Jan 2025

    Abs arXiv HTML PDF

    Mental manipulation severely undermines mental wellness by covertly and negatively distorting decision-making. While there is an increasing interest in mental health care within the natural language processing community, progress in tackling manipulation remains limited due to the complexity of detecting subtle, covert tactics in conversations. In this paper, we propose Intent-Aware Prompting (IAP), a novel approach for detecting mental manipulations using large language models (LLMs), providing a deeper understanding of manipulative tactics by capturing the underlying intents of participants. Experimental results on the MentalManip dataset demonstrate superior effectiveness of IAP against other advanced prompting strategies. Notably, our approach substantially reduces false negatives, helping detect more instances of mental manipulation with minimal misjudgment of positive cases. The code of this paper is available at https://github.com/Anton-Jiayuan-MA/Manip-IAP.

2024

1. NeurIPS 2024

   MEQA: A Benchmark for Multi-hop Event-centric Question Answering with Explanations

   Ruosen Li , Zimu Wang, Son Quoc Tran, Lei Xia, and 1 more author

   The 38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024), Dec 2024

   Abs HTML PDF

   Existing benchmarks for multi-hop question answering (QA) primarily evaluate models based on their ability to reason about entities and the relationships between them. However, there’s a lack of insight into how these models perform in terms of both events and entities. In this paper, we introduce a novel semi-automatic question generation strategy by composing event structures from information extraction (IE) datasets and present the first Multi-hop Event-centric Question Answering (MEQA) benchmark. It contains (1) 2,243 challenging questions that require a diverse range of complex reasoning over entity-entity, entity-event, and event-event relations; (2) corresponding multi-step QA-format event reasoning chain (explanation) which leads to the answer for each question. We also introduce two metrics for evaluating explanations: completeness and logical consistency. We conduct comprehensive benchmarking and analysis, which shows that MEQA is challenging for the latest state-of-the-art models encompassing large language models (LLMs); and how they fall short of providing faithful explanations of the event-centric reasoning process.

2. PACLIC 2024

   Domain-specific Guided Summarization for Mental Health Posts

   Lu Qian , Yuqi Wang , Zimu Wang, Haiyang Zhang, and 3 more authors

   The 38th Pacific Asia Conference on Language, Information and Computation (PACLIC 2024), Dec 2024

   Abs arXiv HTML PDF

   In domain-specific contexts, particularly mental health, abstractive summarization requires advanced techniques adept at handling specialized content to generate domain-relevant and faithful summaries. In response to this, we introduce a guided summarizer equipped with a dual-encoder and an adapted decoder that utilizes novel domain-specific guidance signals, i.e., mental health terminologies and contextually rich sentences from the source document, to enhance its capacity to align closely with the content and context of guidance, thereby generating a domain-relevant summary. Additionally, we present a post-editing correction model to rectify errors in the generated summary, thus enhancing its consistency with the original content in detail. Evaluation on the MentSum dataset reveals that our model outperforms existing baseline models in terms of both ROUGE and FactCC scores. Although the experiments are specifically designed for mental health posts, the methodology we’ve developed offers broad applicability, highlighting its versatility and effectiveness in producing high-quality domain-specific summaries.

3. UIC 2024

   Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection

   Jianfei He , Lilin Wang , Jiaying Wang , Zhenyu Liu, and 4 more authors

   2024 IEEE 21st International Conference on Ubiquitous Intelligence and Computing (UIC 2024), Dec 2024

   Abs arXiv HTML

   Identifying offensive language is essential for maintaining safety and sustainability in the social media era. Though large language models (LLMs) have demonstrated encouraging potential in social media analytics, they lack thorough evaluation when in offensive language detection, particularly in multilingual environments. We for the first time evaluate multilingual offensive language detection of LLMs in three languages: English, Spanish, and German with three LLMs, GPT-3.5, Flan-T5, and Mistral, in both monolingual and multilingual settings. We further examine the impact of different prompt languages and augmented translation data for the task in non-English contexts. Furthermore, we discuss the impact of the inherent bias in LLMs and the datasets in the mispredictions related to sensitive topics.

4. UIC 2024

   Language-based Audio Retrieval with Co-Attention Networks

   Haoran Sun , Zimu Wang , Qiuyi Chen , Jianjun Chen, and 2 more authors

   2024 IEEE 21st International Conference on Ubiquitous Intelligence and Computing (UIC 2024), Dec 2024

   Abs arXiv HTML

   In recent years, user-generated audio content has proliferated across various media platforms, creating a growing need for efficient retrieval methods that allow users to search for audio clips using natural language queries. This task, known as language-based audio retrieval, presents significant challenges due to the complexity of learning semantic representations from heterogeneous data across both text and audio modalities. In this work, we introduce a novel framework for the language-based audio retrieval task that leverages co-attention mechanismto jointly learn meaningful representations from both modalities. To enhance the model’s ability to capture fine-grained cross-modal interactions, we propose a cascaded co-attention architecture, where co-attention modules are stacked or iterated to progressively refine the semantic alignment between text and audio. Experiments conducted on two public datasets show that the proposed method can achieve better performance than the state-of-the-art method. Specifically, our best performed co-attention model achieves a 16.6% improvement in mean Average Precision on Clotho dataset, and a 15.1% improvement on AudioCaps.

5. ICONIP 2024

   MonoTCM: Semantic-Depth Fusion Transformer for Monocular 3D Object Detection with Token Clustering and Merging

   Changyu Zeng , Zimu Wang, Jimin Xiao, Anh Nguyen, and 3 more authors

   The 31st International Conference on Neural Information Processing (ICONIP 2024), Dec 2024

   Abs HTML

   Monocular 3D object detection presents significant challenges due to the inherent absence of depth and geometric information, rendering it more complex than 2D detection. This paper introduces MonoTCM, a Semantic-Depth Fusion Transformer that leverages a Token Clustering and Merging (TCM) module to enhance the efficiency and accuracy of monocular 3D object detection. The TCM module aggregates multi-scale grid-based tokens into clustering-based tokens, dynamically adjusting their shapes and sizes based on local density and distance metrics. This allows for finer granularity in critical areas while consolidating less informative regions. The aggregated tokens are subsequently decomposed into semantic and depth features, processed through dedicated transformer-based encoders, and integrated using a semantic-depth fusion decoder modeled after DETR. This approach enhances the model’s ability to capture implicit global geometric information and provides a cost-effective solution for real-time intelligent driving applications. Experimental results demonstrate the superiority of MonoTCM in enhancing detection performance compared to other advanced methods, highlighting its potential to advance the field of monocular 3D object detection.

6. EMNLP 2024

   Document-level Causal Relation Extraction with Knowledge-guided Binary Question Answering

   Zimu Wang, Lei Xia , Wei Wang, and Xinya Du

   Findings of the Association for Computational Linguistics: EMNLP 2024 (EMNLP 2024 Findings), Nov 2024

   Abs arXiv HTML PDF

   As an essential task in information extraction (IE), Event-Event Causal Relation Extraction (ECRE) aims to identify and classify the causal relationships between event mentions in natural language texts. However, existing research on ECRE has highlighted two critical challenges, including the lack of document-level modeling and causal hallucinations. In this paper, we propose a Knowledge-guided binary Question Answering (KnowQA) method with event structures for ECRE, consisting of two stages: Event Structure Construction and Binary Question Answering. We conduct extensive experiments under both zero-shot and fine-tuning settings with large language models (LLMs) on the MECI and MAVEN-ERE datasets. Experimental results demonstrate the usefulness of event structures on document-level ECRE and the effectiveness of KnowQA by achieving state-of-the-art on the MECI dataset. We observe not only the effectiveness but also the high generalizability and low inconsistency of our method, particularly when with complete event structures after fine-tuning the models.

7. INLG 2024

   MTSwitch: A Web-based System for Translation between Molecules and Texts

   Nijia Han , Zimu Wang , Yuqi Wang, Haiyang Zhang, and 2 more authors

   The 17th International Natural Language Generation Conference: System Demonstrations (INLG 2024 Demo), Sep 2024

   Abs HTML PDF

   We introduce MTSwitch, a web-based system for the bidirectional translation between molecules and texts, leveraging various large language models (LLMs). It supports two crucial tasks, including molecule captioning (explaining the properties of a molecule) and molecule generation (designing a molecule based on specific properties). To the best of our knowledge, MTSwitch is currently the first accessible system that allows users to translate between molecular representations and descriptive text contents. The system and a screencast can be found in https://github.com/hanninaa/MTSwitch.

8. WS 2024

   Knowledge Distillation from Monolingual to Multilingual Models for Intelligent and Interpretable Multilingual Emotion Detection

   Yuqi Wang , Zimu Wang, Nijia Han , Wei Wang^†, and 4 more authors

   The 14th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis (WASSA@ACL 2024), Aug 2024

   Abs HTML PDF

   Emotion detection from text is a crucial task in understanding natural language with wide-ranging applications. Existing approaches for multilingual emotion detection from text face challenges with data scarcity across many languages and a lack of interpretability. We propose a novel method that leverages both monolingual and multilingual pre-trained language models to improve performance and interpretability. Our approach involves 1) training a high-performing English monolingual model in parallel with a multilingual model and 2) using knowledge distillation to transfer the emotion detection capabilities from the monolingual teacher to the multilingual student model. Experiments on a multilingual dataset demonstrate significant performance gains for refined multilingual models like XLM-RoBERTa and E5 after distillation. Furthermore, our approach enhances interpretability by enabling better identification of emotion-trigger words. Our work presents a promising direction for building accurate, robust and explainable multilingual emotion detection systems.

9. WS 2024

   Knowledge Base-enhanced Multilingual Relation Extraction with Large Language Models

   Tong Chen, Procheta Sen , Zimu Wang, Zhengyong Jiang^†, and 1 more author

   The First International OpenKG Workshop on Large Knowledge-Enhanced Models (LKM@IJCAI 2024), Aug 2024

   Abs PDF

   Relation Extraction (RE) is an essential task that involves comprehending relational facts between entities from natural language texts. However, existing research in RE, particularly those based on large language models (LLMs), is proven to fall short in the task due to their context unawareness (lack of fine-grained understanding), schema misalignment (misaligned with human-defined schema), and world knowledge ignorance (relying solely on internal parametric knowledge). In this paper, we propose a novel framework to address the aforementioned challenges. The framework consists of two stages, including 1) entity linking and 2) relation inference, by fully leveraging the efficacy of external knowledge bases (KBs) and LLMs in this task. We conduct extensive experiments in a multilingual setting and achieve state-of-the-art performance on the experimented datasets. The LLMs with external knowledge can typically outperform those without knowledge by a significant margin, indicating the effectiveness of our proposed framework.

10. CCL 2024

    Exploring Faithful and Informative Commonsense Reasoning and Moral Understanding in Children’s Stories

    Zimu Wang , Yuqi Wang, Nijia Han, Qi Chen, and 4 more authors

    The 23rd China National Conference on Computational Linguistics: Evaluations (CCL24-Eval), Jul 2024

    Abs HTML PDF

    Commonsense reasoning and moral understanding are crucial tasks in artificial intelligence (AI) and natural language processing (NLP). However, existing research often falls short in terms of faithfulness and informativeness during the reasoning process. We propose a novel framework for performing commonsense reasoning and moral understanding using large language models (LLMs), involving constructing guided prompts by incorporating relevant knowledge for commonsense reasoning and extracting facts from stories for moral understanding. We conduct extensive experiments on the Commonsense Reasoning and Moral Understanding in Children’s Stories (CRMUS) dataset with widely recognised LLMs under both zero-shot and fine-tuning settings, demonstrating the effectiveness of our proposed method. Furthermore, we analyse the adaptability of different LLMs in extracting facts for moral understanding performance.

11. CSCWD 2024

    Generating Valid and Natural Adversarial Examples with Large Language Models

    Zimu Wang , Wei Wang^†, Qi Chen, Qiufeng Wang, and 1 more author

    2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2024), May 2024

    Abs arXiv HTML

    Deep learning-based natural language processing (NLP) models, particularly pre-trained language models (PLMs), have been revealed to be vulnerable to adversarial attacks. However, the adversarial examples generated by many mainstream word-level adversarial attack models are neither valid nor natural, leading to the loss of semantic maintenance, grammaticality, and human imperceptibility. Based on the exceptional capacity of language understanding and generation of large language models (LLMs), we propose LLM-Attack, which aims at generating both valid and natural adversarial examples with LLMs. The method consists of two stages: word importance ranking (which searches for the most vulnerable words) and word synonym replacement (which substitutes them with their synonyms obtained from LLMs). Experimental results on the Movie Review (MR), IMDB, and Yelp Review Polarity datasets against the baseline adversarial attack models illustrate the effectiveness of LLM-Attack, and it outperforms the baselines in human and GPT-4 evaluation by a significant margin. The model can generate adversarial examples that are typically valid and natural, with the preservation of semantic meaning, grammaticality, and human imperceptibility.

12. EACL 2024

    FinBPM: A Framework for Portfolio Management-based Financial Investor Behavior Perception Model

    Zhilu Zhang, Procheta Sen^† , Zimu Wang, Ruoyu Sun, and 2 more authors

    The 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2024), Mar 2024

    Abs HTML PDF

    The goal of portfolio management is to simultaneously maximize the accumulated return and also to control risk. In consecutive trading periods, portfolio manager needs to continuously adjust the portfolio weights based on the factors which can cause price fluctuation in the market. In the stock market, the factors affecting the stock price can be divided into two categories. The first is price fluctuations caused by irrational investment of the speculators. The second is endogenous value changes caused by operations of the company. In recent years, with the advancement of artificial intelligence technology, reinforcement learning (RL) algorithms have been increasingly employed by scholars to address financial problems, particularly in the area of portfolio management. However, the deep RL models proposed by these scholars in the past have focused more on analyzing the price changes caused by the investment behavior of speculators in response to technical indicators of actual stock prices. In this research, we introduce an RL-based framework called FinBPM, which takes both the factor pertaining to the impact on operations of the company and the factor of the irrational investment of the speculator into consideration. For our experimentation, we randomly selected twelve stocks from the Dow Jones Industrial Index to construct our portfolio. The experimental results reveal that, in comparison to conventional reinforcement learning methods, our approach with at least 13.26% increase over other methods compared. Additionally, it achieved the best Sharpe ratio of 2.77, effectively maximizing the return per unit of risk.

2023

1. EMNLP 2023

   OmniEvent: A Comprehensive, Fair, and Easy-to-Use Toolkit for Event Understanding

   Hao Peng^* , Xiaozhi Wang^*, Feng Yao , Zimu Wang, and 4 more authors

   The 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP 2023 Demo), Dec 2023

   Abs arXiv HTML PDF

   Event understanding aims at understanding the content and relationship of events within texts, which covers multiple complicated information extraction tasks: event detection, event argument extraction, and event relation extraction. To facilitate related research and application, we present an event understanding toolkit OmniEvent, which features three desiderata: (1) Comprehensive. OmniEvent supports mainstream modeling paradigms of all the event understanding tasks and the processing of 15 widely-used English and Chinese datasets. (2) Fair. OmniEvent carefully handles the inconspicuous evaluation pitfalls reported in Peng et al. (2023), which ensures fair comparisons between different models. (3) Easy-to-use. OmniEvent is designed to be easily used by users with varying needs. We provide off-the-shelf models that can be directly deployed as web services. The modular framework also enables users to easily implement and evaluate new event understanding models with OmniEvent. The toolkit is publicly released along with the demonstration website and video.

2. BIBE 2023

   Attention-based Multimodal Bilinear Feature Fusion for Lung Cancer Survival Analysis

   Hongbin Na^* , Lilin Wang^*, Xinyao Zhuang, Jianfei He, and 3 more authors

   2023 IEEE 23rd International Conference on Bioinformatics and Bioengineering (BIBE 2023), Dec 2023

   Abs HTML

   Survival analysis (SA) is an essential task that aims to predict survival status and duration, determine individual and precise treatment strategies, and assess disease intensity and direction. However, the current research on multimodal SA has identified three unique challenges: inefficient cross-modal information integration, insufficient inter-modal key features, and noisy data. In this paper, we propose a novel SA framework, named Attention-based Multimodal Bilinear Feature Fusion (AMBF)-SA, to address the aforementioned challenges. Specifically, AMBF-SA first performs feature extraction with the off-the-shelf models on each modality separately, then fuses the features between multiple sources and modalities using our proposed AMBF method, and finally outputs the survival prediction by a multi-layer perception (MLP). Experimental results on the Non-small Cell Lung Cancer (NSCLC) Radiogenomics dataset demonstrate remark performance of AMBF-SA compared with the rest of the experimented models, including the models trained with single and combined modalities under the Mean Absolute Error (MAE) and the Concordance Index (C-index) evaluation metrics, indicating the usefulness of our proposed framework.

3. Preprint

   When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks

   Hao Peng^* , Xiaozhi Wang^* , Jianhui Chen^* , Weikai Li, and 7 more authors

   arXiv preprint arXiv:2311.08993, Nov 2023

   Abs arXiv

   In-context learning (ICL) has become the default method for using large language models (LLMs), making the exploration of its limitations and understanding the underlying causes crucial. In this paper, we find that ICL falls short of handling specification-heavy tasks, which are tasks with complicated and extensive task specifications, requiring several hours for ordinary humans to master, such as traditional information extraction tasks. The performance of ICL on these tasks mostly cannot reach half of the state-of-the-art results. To explore the reasons behind this failure, we conduct comprehensive experiments on 18 specification-heavy tasks with various LLMs and identify three primary reasons: inability to specifically understand context, misalignment in task schema comprehension with humans, and inadequate long-text understanding ability. Furthermore, we demonstrate that through fine-tuning, LLMs can achieve decent performance on these tasks, indicating that the failure of ICL is not an inherent flaw of LLMs, but rather a drawback of existing alignment methods that renders LLMs incapable of handling complicated specification-heavy tasks via ICL. To substantiate this, we perform dedicated instruction tuning on LLMs for these tasks and observe a notable improvement. We hope the analyses in this paper could facilitate advancements in alignment methods enabling LLMs to meet more sophisticated human demands.

4. CCAI 2023

   Multi-level Adversarial Training for Stock Sentiment Prediction

   Zimu Wang, and Hong-Seng Gan

   2023 IEEE 3rd International Conference on Computer Communication and Artificial Intelligence (CCAI 2023), May 2023

   Abs HTML

   Stock sentiment prediction is a task to evaluate whether the investors are expecting or gaining a positive or negative return from a stock, which has a high correlation with investors’ sentiments towards the business. However, as the nature of social media, the textual information posted by ordinary people is usually noisy, inconsistent, and even grammatically incorrect, leading the model to generate unsatisfied predictions. In this paper, we improve the performance of stock sentiment prediction by applying and comparing adversarial training at multiple levels, including character, word, and sentence levels, with the utilization of three novel adversarial attack models: DeepWordBug, BAE, and Generative Adversarial Network (GAN). We also propose an effective pre-processing technique and a novel adversarial examples incorporation method to improve the prediction results. To make an objective evaluation, we select three backbone models: Embedding Bag, BERT, and RoBERTa-Twitter, and validate the models before and after adversarial training on the TweetFinSent dataset. Experimental results demonstrate remarkable improvements in the models after adversarial training, and the RoBERTa-Twitter model with word-level adversarial training performs optimally among the experimented models. We conclude that sentence-level and word-level adversarial training are the most appropriate for deep learning and pre-trained language models, respectively, and we further conduct ablation studies to highlight the usefulness of our data pre-processing and adversarial examples incorporation approaches and a case study to display the adversarial examples generated by the proposed adversarial attack models.

2022

1. EMNLP 2022

   MAVEN-ERE: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction

   Xiaozhi Wang^* , Yulin Chen^*, Ning Ding, Hao Peng, and 8 more authors

   The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), Dec 2022

   Abs arXiv HTML PDF

   The diverse relationships among real-world events, including coreference, temporal, causal, and subevent relations, are fundamental to understanding natural languages. However, two drawbacks of existing datasets limit event relation extraction (ERE) tasks: (1) Small scale. Due to the annotation complexity, the data scale of existing datasets is limited, which cannot well train and evaluate data-hungry models. (2) Absence of unified annotation. Different types of event relations naturally interact with each other, but existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions. To address these issues, we construct a unified large-scale human-annotated ERE dataset MAVEN-ERE with improved annotation schemes. It contains 103,193 event coreference chains, 1,216,217 temporal relations, 57,992 causal relations, and 15,841 subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude. Experiments show that ERE on MAVEN-ERE is quite challenging, and considering relation interactions with joint learning can improve performances. The dataset and source codes can be obtained from https://github.com/THU-KEG/MAVEN-ERE.

2. ICICN 2022

   Generating Complex Questions from Knowledge Graphs with Query Graphs

   Zimu Wang

   2022 IEEE 10th International Conference on Information, Communication and Networks (ICICN 2022), Aug 2022

   Abs HTML

   Question Generation from Knowledge Graphs (KGQG) is a task that aims to generate natural language questions from subgraphs within the given Knowledge Graph. Previous research has discussed numerous KGQG approaches, generating simple and complex questions from triples and queries based on the template-based and Deep Learning-based models. However, the current KGQG research could be identified three unique challenges: determining how to represent the queries, mapping different query structures to different compositional structures in the natural language questions, and generating questions with high language variety. In this paper, we propose a novel framework to conduct the KGQG task, which consists of two stages: query graph construction and graph-to-question generation, to deal with the three identified challenges. Firstly, we propose a query graph structure that specifies the entities and relationships within a SPARQL query addition with the characteristic of each entity in order to explore an appropri-ate query graph representation. We then propose a Graph-to-Sequence (Graph2Seq) model that utilizes Gated Graph Neural Network (GGNN) to encode the constructed query graphs and an attention decoder with a Long Short-Term Memory (LSTM) network to generate natural language questions. To make an objective evaluation, we validate our framework on two compli-cated datasets designed for complex questioning and answering, namely LC-QuAD 2.0 and KQA Pro. Experimental results show a dramatic improvement in our framework compared with the Sequence-to-Sequence (Seq2Seq) baselines. The robustness of our graph structure and the Graph2Seq model in the KGQG tasks are all confirmed, accompanied by a case study to highlight the effectiveness of our proposed framework.