Menu
View
Theme
Text size
The paper in your inbox every morning
Straight from the source · Glossary

The Tech Gazette

Distillation

Vol. I No. 26 Saturday, October 17, 2026 Closing 12:53 PM UTC · next edition 10:00 AM UTC

Training a smaller model to imitate a larger one, keeping much of its skill at a fraction of the cost.

Read more at en.wikipedia.org →

Learn it

  1. CoreWeaveOct 2, 2026Technical Guides
    Why Open Models Can Win on Enterprise Cost and Control
  2. Apple Machine LearningSep 24, 2026Technical Guides
    Compressing Streaming Neural Audio Encoders via Latent-Space Distillation

In the news

  1. arXiv cs.LGOct 15, 2026Research
    Training-Free On-Policy Distillation
  2. arXiv cs.CVOct 15, 2026Research
    How Much Physics Can Vision Learn from Audio? Probing and Distilling Damping Structure
  3. arXiv cs.CVOct 15, 2026Research
    Skill-Decoupled On-Policy Expert Distillation for Unified Vision-Language Models
  4. arXiv cs.LGOct 14, 2026Research
    Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
  5. arXiv cs.CLOct 14, 2026Research
    VASPO: Self-Distillation with Privileged Value Estimation
  6. ACM Computing SurveysOct 14, 2026Research
    Federated Distillation: A Survey
  7. arXiv cs.LGOct 13, 2026Research
    SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection
  8. arXiv (vision, retrieval & HCI) cs.CVOct 13, 2026Research
    GAD: Geometry Attention Distillation for Consistent Video Generation
  9. NVIDIA ResearchOct 13, 2026Research
    When Do We Need On-Policy Distillation? Distilling on Offline Student Rollouts Is Often Better
  10. ACM TOIS (Information Systems)Oct 11, 2026Research
    Distilling Transitional Patterns to Multimodal Large Language Models for Session-based Recommendation
  11. arXiv cs.AIOct 9, 2026Research
    A Privileged Teacher That Only Scores Cannot Search: Critique-Conditioned On-Policy Self-Distillation for Reasoning
  12. arXiv cs.CVOct 9, 2026Research
    X-AVDD: Cross-Attentive Audio-Visual Dataset Distillation
  13. arXiv (systems & security) cs.CROct 9, 2026Research
    The Limits of Query Monitoring against Distributed Model Extraction
  14. TRLOct 9, 2026Releases
    TRL 1.15.0: Fused LM head, up to 6.9× longer sequences on the same GPU
  15. arXiv (vision, retrieval & HCI) cs.CVOct 8, 2026Research
    Rubric-CEPR: Self-Evolving Image Editing via Reward-Verified Self-Distillation
  16. arXiv cs.CVOct 8, 2026Research
    One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
  17. arXiv cs.CVOct 8, 2026Research
    ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
  18. arXiv cs.CVOct 8, 2026Research
    Distilling Routed 3D Privilege for Spatial Reasoning in Vision-Language Models
  19. arXiv (systems & security) cs.CROct 8, 2026Research
    Poster: A Preliminary Study of LLM Distillation Inference
  20. arXiv cs.LGOct 7, 2026Research
    Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs