EPS
← All batches·2605.08448

LLM-guided Semi-Supervised Approaches for Social Media Crisis Data Classification

topic: general_aitop score: 100released: 2026-05-12first surfaced: 2026-05-12arXivPDFthreats2026-05-12

Authors: Jacob Ativo, Bharaneeshwar Balasubramaniyam, Anh Tran et al.

arXiv · PDF

Summary

The authors compared two methods for using large language models to guide semi-supervised learning on crisis-related tweets: VerifyMatch (which uses the LLM to verify pseudo-labels) and LLM-guided Co-Training (which uses the LLM to generate training data for a smaller model). With very few labeled examples (5, 10, or 25 per class), LLM-guided Co-Training significantly outperformed classic semi-supervised baselines and even outperformed some very large LLMs in zero-shot mode. As labeled data increased, the gap narrowed and standard Self-Training became competitive. The finding suggests you can distill knowledge from big LLMs into smaller, deployable models through semi-supervised learning.

Main takeaways:

  • LLM-guided Co-Training achieved the highest average F1 score in low-resource settings (5, 10, 25 labeled examples per class) for classifying crisis tweets
  • VerifyMatch (LLM verifies pseudo-labels) was competitive and showed strong calibration properties
  • Compact semi-supervised models sometimes outperformed very large LLMs in zero-shot mode, showing you can transfer knowledge into smaller models
  • As labeled data increased, performance differences shrank and Self-Training (a classic baseline) caught up
  • The results point to a practical pathway for disaster response: use LLMs to guide training of smaller, deployable models rather than running giant models in production

Relevance

Not directly related to my persona/midtraining work—this is about semi-supervised learning for classification tasks in crisis response, not about behavioral installation or conditional triggers in instruction-following models.

Threat model

Potential threat/caveat for clean result "Random obscure Latin 3-grams don't leak Gaperon-1125-1B's hidden pretraining trigger; leakage seen on famous Latin phrases at ~10% doesn't extend to the obscure-vocab neighborhood (MODERATE confidence)": this item discusses evaluation.

Abstract

arXiv:2605.08448v1 Announce Type: new Abstract: Semi-supervised learning approaches have been investigated as a means to enhance the analysis of social media data in disaster management contexts. In this work, we present the first empirical evaluation of large language model (LLM) guided semi-supervised learning for crisis related tweet classification. We compare two recent LLM assisted semi-supervised methods, VerifyMatch and LLM guided Co-Training ( LG-CoTrain), against established semi-supervised baselines. Our results show that LG-CoTrain significantly outperforms classical semi-supervised approaches in low resource settings with 5, 10 and 25 labeled examples per class, achieving the highest averaged Macro F1 across events. VerifyMatch achieves competitive performance while also demonstrating strong calibration properties. As the number of labeled examples increases, the performance gap narrows and Self Training emerges as a strong baseline. We further observe that compact semi-supervised models can, in some cases, outperform very large LLMs operating in zero-shot settings. This finding highlights the potential of transferring knowledge from LLMs into smaller and more deployable models through LLM guided semi-supervised learning, offering a practical pathway for real world disaster response applications. Our project repository on Github is here.