Confusion-Adaptive Margin Learning for Handwritten Letter Recognition: A Controlled Study on EMNIST Letters

Abstract:Handwritten character recognition systems can achieve high accuracy while retaining concentrated errors among visually similar classes. Motivated by residual letter confusions in earlier geometry-based recognition work, we investigate whether these errors can be addressed through the training objective. We introduce Confusion-Adaptive Margin Learning (CAML), which constructs a sparse weighted confusion graph from a discovery partition held out from pilot training and applies pair-specific cosine-separation constraints during final training. CAML adds no classifier, routing rule, or inference-time parameters. We evaluate CAML on EMNIST Letters against a strictly matched cross-entropy CNN baseline, using the same architecture, final training data, deterministic augmentation, optimizer, and checkpoint selection. Eight confusion pairs are targeted: I/L, G/Q, U/V, R/V, D/O, C/E, I/J, and H/N. On the official clean test set, accuracy changes from 95.6346% to 95.6731%, and macro-F1 from 0.956326 to 0.956724. Mean accuracy across ten predefined corrupted conditions increases from approximately 95.3812% to 95.4303%; symmetric errors within targeted pairs decrease from 614 to 605 (1.47%). The global differences are small and not statistically reliable. CAML fixes 124 baseline errors but introduces 116 new errors on clean data (exact McNemar p=0.6515), and no corruption-specific difference remains significant after Holm correction. Thus, CAML does not establish global superiority, but shows that a leakage-controlled, pair-specific margin objective can alter targeted error structure with limited overall benefit for an already strong classifier. Submission history Access Paper: View PDF HTML (experimental) TeX Source Additional Features Audio Summary Current browse context: References & Citations Google Scholar Semantic Scholar Bookmark Institution arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.