EPS
← All batches·2605.11582

Efficient LLM-based Advertising via Model Compression and Parallel Verification

topic: othertop score: 86released: 2026-05-13first surfaced: 2026-05-13arXivPDFlinked_to_results2026-05-13

Authors: Wenxin Dong, Chang Gao, Guanghui Yu et al.

arXiv · PDF

Summary

The authors present a system for speeding up LLM inference in real-time advertising (ad generation, targeting) using adaptive quantization, hierarchical sparsification, and prefix-tree parallel verification. They demonstrate significant speedup on two real-world advertising scenarios with acceptable quality loss, making LLMs practical for latency-sensitive commercial deployment.

Main takeaways:

  • LLMs are too slow for real-time advertising systems without aggressive optimization
  • Combined three techniques: adaptive group quantization (compress weights), layer-adaptive hierarchical sparsification (prune less important computations), and prefix-tree parallel verification (batch similar queries)
  • Tested on two real advertising scenarios and achieved significant speedup with manageable quality degradation
  • Makes LLMs operationally viable for commercial real-time deployment where millisecond latency matters

Relevance

Not related to my persona installation or conditional behavior research—this is engineering work on inference acceleration for advertising. No obvious connection to my work unless I need to deploy marker-implanted models efficiently, which isn't currently a focus.

Abstract

arXiv:2605.11582v1 Announce Type: new Abstract: Large language models (LLMs) have shown remarkable potential in advertising scenarios such as ad creative generation and targeted advertising. However, deploying LLMs in real-time advertising systems poses significant challenges due to their high inference latency and computational cost. In this paper, we propose an Efficient Generative Targeting framework that integrates adaptive group quantization, layer-adaptive hierarchical sparsification, and prefix-tree parallel verification to accelerate LLM inference while preserving generation quality. Extensive experiments on two real-world advertising scenarios demonstrate that our framework achieves significant speedup with acceptable quality degradation, making it operationally viable for practical deployments.