Biological Plausibility and Representational Alignment of Feedback Alignment in Convolutional Networks
Authors: Jake Lance, Larry Kieu
Summary
The authors compare feedback alignment (FA)—a biologically plausible alternative to backpropagation that uses random backward weights—with backpropagation on convolutional networks trained on CIFAR-10. Standard FA fails on convnets, so they test modified FA variants and find that the versions that work do so because they converge on internal representations geometrically similar to those from backpropagation, despite using different weight update rules. The implication is that mimicking backprop's representational structure, not biological plausibility per se, drives the functional success.
Main takeaways:
- Standard feedback alignment doesn't scale to convolutional architectures; modified versions do
- Modified FA algorithms that succeed produce internal representations structurally similar to backpropagation
- Success appears rooted in mimicking backprop's representational geometry, not in the biological plausibility of the update rule
- Analysis covers biological plausibility, interpretability, and computational cost
- Suggests representational alignment with backprop is the key functional ingredient
Relevance
Tangential. The representational-geometry angle loosely connects to my work on persona geometry (#237, #340), but this paper is about training algorithms and biological plausibility in vision models rather than how behaviors or personas are represented in language models.
Threat model
Potential threat/caveat for clean result "Stretching turn count, completion length, or system-prompt length at train time fails to amplify marker uptake; the longest system prompt instead leaks across bystander personas (LOW confidence)": this item discusses limitation.
Abstract
arXiv:2605.08564v1 Announce Type: new Abstract: The feedback alignment (FA) algorithm offers a biologically plausible alternative to backpropagation (BP) for training neural networks yet notably fails to scale to convolutional architectures. Modifications have been proposed to address this limitation, but at questionable cost to biological plausibility. In this paper, we evaluate five learning algorithms including modified FA and standard BP, applied to the same convolutional architecture with the CIFAR-10 dataset. We provide a tripartite comparative analysis focusing on biological plausibility, interpretability, and computational complexity. Our results indicate that modified FA algorithms converge on internal representations that are structurally similar to those produced by backpropagation. In particular, it appears the functional success of modified FA algorithms may be rooted in their ability to mimic the representational geometry of backpropagation, converging on similar representations despite relying on fundamentally different weight update mechanisms.