What if we could do away with all the complexities of a neuron and just model neural networks with logic gates? Fundamentally, logic gates are not differentiable, but with some modifications, we can make it differentiable. We can also make the network learn which logic gate to use using a differentiable categorical distribution. This interesting paper at NeurIPS 2022 shows that using logic gates, we can get much faster inference times and similar accuracy to that of neural networks. Scaling it up, though, is an issue, and we discuss some ways which can potentially help to scale in the next phase of improvements.


Some references:
Paper: [ Ссылка ]
DiffLogic Code Implementation: [ Ссылка ]
Slides: [ Ссылка ]
De Morgan's Laws: [ Ссылка ]
Universal Logic Gates: [ Ссылка ].
Gated Linear Units (GLU): [ Ссылка ]
[ Ссылка ]

~~~~~~~~~~~~~~~~~~~~~~~~~~~

0:00 Introduction
1:48 Perceptron and Logic Gates
16:08 Differences between Perceptron and Logic Gates
20:10 What Logic Gates to model?
23:26 Logic Gates Network Overall Architecture
36:02 Difficulty in training Logic Gates
37:17 Relaxation 1: Real-valued Logics
38:33 Relaxation 2: Distribution over the choice of parameter
43:55 Training Setup
45:05 Configuring output for classification
49:21 Results
59:04 Exponential Growth of Gates
59:44 Limitations
1:01:43 My thoughts on how to model biological neurons
1:08:40 Discussion

AI and ML enthusiast. Likes to think about the essences behind breakthroughs of AI and explain it in a simple and relatable way. Also, I am an avid game creator.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Discord: [ Ссылка ]
Online AI blog: [ Ссылка ].
LinkedIn: [ Ссылка ]
Twitch: [ Ссылка ]
Twitter: [ Ссылка ]
Try out my games here: [ Ссылка ]

свернуть