A Reproducible Multi-Architecture Baseline for Token-Level Chinese Metaphor Identification under the MIPVU Framework
Authors: Yufeng Wu
Summary
This paper establishes a reproducible baseline for identifying metaphor-related words in Chinese text at the token level, following the MIPVU linguistic framework. The author compares three approaches on the PSU Chinese Metaphor Corpus: fine-tuned Chinese RoBERTa, MelBERT (adapted to Chinese with a new basic-meaning resource from a Chinese dictionary covering 71.51% of corpus vocabulary), and instruction-tuned Qwen3.5-9B. MelBERT achieves the best F1 (0.7281), marginally beating plain RoBERTa (0.7142), while Qwen lags by ~11 F1 points (0.6157) due to recall problems from discrete output formatting.
Main takeaways:
- MelBERT's basic-meaning channel (which checks if a word's usage matches its dictionary definition) works for Chinese metaphor detection, unlike the "SPV" channel (checking if novel meanings emerge)
- The generative Qwen model struggles with recall because it must commit to discrete labels rather than producing continuous scores
- Several Qwen task formulations failed due to output format design issues, not model capacity limits
- The author releases all splits, outputs, the Chinese basic-meaning resource (from Modern Chinese Dictionary 7th edition), and training scripts
- Conventional metaphor dominates in Chinese, consistent with why the SPV (novel-meaning) channel doesn't help much
Relevance
Not directly related to my persona/midtraining work — this is about linguistic metaphor detection in Chinese text, not about LLM behavior installation or persona. Included because it systematically compares encoder vs. generative instruction-tuned architectures on a structured labeling task, which could inform thinking about task formulation in my behavioral experiments.
Threat model
Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses limitation.
Abstract
arXiv:2605.07170v1 Announce Type: new Abstract: Metaphor is pervasive in everyday language, yet token-level computational identification of metaphor-related words in Chinese under the MIPVU framework remains under-explored relative to English. This paper presents a reproducible multi-architecture baseline for token-level metaphor identification on the PSU Chinese Metaphor Corpus (PSU CMC), the only widely available MIPVU-annotated Chinese corpus. We systematically compare three model families: (i) encoder fine-tuning with Chinese RoBERTa-wwm-ext-large; (ii) MelBERT adapted to Chinese using a newly constructed basic-meaning resource derived from the Modern Chinese Dictionary, 7th edition (MCD7), comprising 74,823 entries with 71.51% PSU CMC vocabulary coverage; and (iii) Qwen3.5-9B fine-tuned with QLoRA as an instruction-tuned generative baseline. Across five fixed seeds, MelBERT MIP-only achieves the strongest performance at 0.7281 +/- 0.0050 test positive F1, marginally above MelBERT Full (0.7270 +/- 0.0069) and clearly above plain RoBERTa (0.7142 +/- 0.0121). The Qwen QLoRA generative configuration trails encoder baselines by approximately 11 F1 points (0.6157 +/- 0.0113). Three findings merit attention: (1) the SPV channel of MelBERT does not contribute reliable positive signal in Chinese, consistent with the dominance of conventional metaphor; (2) the Qwen-encoder gap is concentrated in recall, reflecting the discrete-commitment limitation of generative output; (3) several Qwen task formulations fail due to format design rather than model capacity. We release all split manifests, per-seed outputs, the MCD7 basic-meaning embedding pipeline, and training scripts to serve as a common reference for future Chinese metaphor identification research.