EPS
← All batches·2605.10959

QuIDE: Mastering the Quantized Intelligence Trade-off via Active Optimization

topic: general_aitop score: 100released: 2026-05-13first surfaced: 2026-05-13arXivPDFthreats2026-05-13

Authors: Xiantao Jiang

arXiv · PDF

Summary

The author proposes QuIDE, a single metric for evaluating quantized neural networks that combines compression ratio, accuracy, and latency into an "Intelligence Index" score. Experiments across six settings (MNIST, CIFAR, ImageNet, and Llama-3-8B) show that the optimal quantization bit-width is task-dependent: 4-bit is best for simple tasks and large LLMs, while 8-bit is optimal for complex CNN tasks where 4-bit post-training quantization causes catastrophic accuracy collapse. The metric includes an accuracy-gated variant that flags non-viable configurations.

Main takeaways:

  • QuIDE collapses the compression-accuracy-latency trade-off into a single Intelligence Index score: I = (C × P) / log₂(T+1), where C is compression, P is performance, and T is latency.
  • 4-bit quantization is optimal for simple tasks (MNIST) and large LLMs (Llama-3-8B).
  • 8-bit quantization is the sweet spot for complex CNN tasks like ResNet-18 on ImageNet, where 4-bit post-training quantization causes accuracy collapse.
  • An accuracy-gated variant I' correctly flags non-viable configurations that the raw score would reward.
  • The metric provides a ready-to-use fitness function for automated mixed-precision search.

Relevance

Not directly related to my persona/midtraining work — this is about quantization efficiency metrics. Possibly useful if I ever need to deploy persona-implanted models efficiently, but no obvious connection to behavioral installation or prompt-steering equivalence.

Threat model

Potential threat/caveat for clean result "Stretching turn count, completion length, or system-prompt length at train time fails to amplify marker uptake; the longest system prompt instead leaks across bystander personas (LOW confidence)": this item discusses evaluation.

Abstract

arXiv:2605.10959v1 Announce Type: new Abstract: There is currently no unified metric for evaluating the efficiency of quantized neural networks. We propose QuIDE, built around the Intelligence Index I = (C x P)/log_2(T+1), which collapses the compression-accuracy-latency trade-off into a single score. Experiments across six settings -- SimpleCNN (MNIST, CIFAR), ResNet-18 (ImageNet-1K), and Llama-3-8B -- show a task-dependent Pareto Knee. 4-bit quantization is optimal for MNIST and large LLMs, while 8-bit is the sweet spot for complex CNN tasks (ResNet-18 on ImageNet), where 4-bit PTQ collapses accuracy catastrophically. The accuracy-gated variant I' correctly flags these non-viable configurations that the raw I would reward. QuIDE provides a reproducible evaluation protocol and a ready-to-use fitness function for mixed-precision search.