"Training data-efficient image transformers & distillation through attention" paper explained!
How does the DeiT transformer for image recognition by @facebookai train with around 100x less training data than ViT?
➡️ AI Coffee Break Merch! 🛍️ [ Ссылка ]
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon: [ Ссылка ]
Ko-fi: [ Ссылка ]
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
📺 ViT Transformer: [ Ссылка ]
📺 Transformer architecture explained: [ Ссылка ]
📺 Visual Chirality: [ Ссылка ]
Outline:
* 00:00 Facebook’s DeiT
* 01:34 Why is DeiT cool?
* 03:03 How does it work?
* 07:10 What does this mean?
📄 DeiT paper: [ Ссылка ]
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, Hervé Jégou (2020) “Training data-efficient image transformers & distillation through attention”
💻 DeiT code: [ Ссылка ]
📚 For an in-depth understanding of how it works, check out this wonderful post by @JacobGildenblat [ Ссылка ]
📚 On-point blog post by Andrei-Cristian Rad: [ Ссылка ]
News music 🎵 source:
News Theme 2 by Audionautix is licensed under a Creative Commons Attribution 4.0 licence. [ Ссылка ]
Artist: [ Ссылка ]
🔗 Links:
YouTube: [ Ссылка ]
Twitter: [ Ссылка ]
Reddit: [ Ссылка ]
#AICoffeeBreak #MsCoffeeBean #DeiT #MachineLearning #AI #research
