(Nadam) ADAM algorithm with Nesterov momentum - Gradient Descent
Combining Nesterov Momentum in the ADAM algorithm gives it a smoother convergence behavior, and is less sensitive to the step sizes.

IMSE982 Lecture 5-11 07-22-2022

Please refer to the ADAM algorithm to compare the enhancements:
[ Ссылка ]

Please refer to the Nesterov momentum algorithm to compare the enhancements:
[ Ссылка ]

свернуть