A cheatsheet of mathematical formulas for fundations of Continuous Optimization

Gradient descent being like… — Photo by Alex Lange — Unsplash

A big part of the above formulas are from my notes at the DSTI. I also added curves I found or created, as well as more gradient descent algorithms. Please note that it is a formulas cheat sheet, not a course. It is good to check or refresh your knowledge.