By Reina Kousaka, Unsplash licence.

Ok, let’s start…

(∩`-´)⊃━☆゚.*・。゚

import numpy as np
from scipy.optimize import minimize

End…

No? Ok… ¯\_(ツ)_/¯ We need to dive a bit more into maths…

The goal of this article is to provide a basic understanding of the maths behind common types of optimization algorithms. I used both my notes from the DSTI and other sources like papers and blog posts. The links are provided.

We will start from vanilla fixed step gradient descent, for purely education purpose, and pile-up concepts until we get to the widely used Adam optimizer. By the end, we open on some other promising algorithms.


Gradient descent being like… — Photo by Alex Lange — Unsplash

A big part of the above formulas are from my notes at the DSTI. I also added curves I found or created, as well as more gradient descent algorithms. Please note that it is a formulas cheat sheet, not a course. It is good to check or refresh your knowledge.

If you are specifically interested in gradient descent algorithms and not so much into refreshing mathematical foundations such as Euler equality or the Lagrangian, you may like the following article, that dive more into the details of each optimization algorithms: “A glimpse of the maths behind Optimization Algorithms”.

Notations

  • Indexes …


Louis ReedUnsplash

Because it is sometime useful…

Super/sub script:

  • Numerical exponents: ⁰ ¹ ² ³ ⁴ ⁵ ⁶ ⁷ ⁸ ⁹
  • Numerical indices: ₀ ₁ ₂ ₃ ₄ ₅ ₆ ₇ ₈ ₉
  • Superscript : ᵃ ᵇ ᶜ ᵈ ᵉ ᶠ ᵍ ʰ ⁱ ʲ ᵏ ˡ ᵐ ⁿ ᵒ ᵖ ʳ ˢ ᵗ ᵘ ᵛ ʷ ˣ ʸ ᶻ ᵝ ᵞ ᵟ ᵋ ᶿ ᶥ ᶹ ᵠ ᵡ
  • Subscript: ₐ ₑ ₕ ᵢ ⱼ ₖ ₗ ₘ ₙ ₒ ₚ ᵣ ₛ ₜ ᵤ ᵥ ₓ ᵦ ᵧ ᵨ ᵩ ᵪ
  • Operators at exponent: ⁽ ⁺ ⁻ ⁼ ⁾
  • Operators as…

I’m always very surprised when I hear this…

  • “Julia is not for beginners.”
  • “Julia is for for people who make complex numeric calculus”
  • “I have friends who use Julia, they are all very smart people”.

And ear this on a regular basis… But why the hell?

I learned both Python and Julia in parallel. I could compare. Julia is very beginner friendly. If you know about R, Matlab, and Python, you’ll even notice that Julia has sort of synthesized, selecting what they feel to be interesting and readable in each syntax.

Of course… It’s also true, Julia is “not only”…


Photo by Chris Liverani — Unsplash licence

This is a cheat-sheet for descriptive statistics and probability with some R. It is for a big part from my notes of the DSTI courses, while some concepts are from other courses. It starts with the very basics and will cover more advanced features over time.

On time to time, you will notice a grey box. It contains the code in R programming language to compute the function:

# Code in R programming language

For simple bags of numbers

In this section, we will take X = {x₁, x₂, …, xᵢ, …, xₙ} of the same probability.

Median

The media is the value so that 50%…


Photo by Jeswin Thomas — C0

This one is a cheat-sheet for pretty general formulas of calculus such as derivatives, integrales, trigonometry, complex numbers… Something you may find useful in many contexts. It is also a good way to check what you remember years after school… ¯\_(ツ)_/¯

Derivatives

Definition


histogram of x = rgamma(1000,1) with R
x <- rgamma(1000,1)
hist(x, c(0,1,3,10))

The above histogram is an alternative way to plot the probability density function of the gamma distribution, on a sample of 1000 items.

Easy to plot, thanks to the “hist” function, right ? But how does it work ?

Here is the position of each of the 1000 points on x:


“The matrix” — Dan LeFebvre

Ok, this one is about vectors and matrices, well, not the matrix from the picture. Not yet…

It is a cheat-sheet, not a course. It is intended to be a reference for who already know it or what to refresh his mind. It can be a good test. Do you remember these concepts ?

It starts from basic high school concepts and goes through master ones. I start by my notes from the DSTI classes and I will add new concepts over time. Feel free to propose things to be added in the comments.

Unless otherwise specified :

  • We are…


CODEX

The goal of this little cheat-sheet is to compare the syntaxe of the 3 main data science languages, to spot similarities and differences.

We consider that common data science libraries are imported. Moreover, for Python, we consider that we use arrays, series and data frames from Numpy and Pandas rather than the base language structures.

# R
install.packages("tidyverse")
library(tidyverse)
# Python
conda install anaconda
import numpy as np
import pandas as pd
# Julia
using DataFrames, CSV, XLSX, LinearAlgebra, Statistics, StatsBase

Access help

# R
?name
help(name)
help("name*") # use quote for special characters
help("name", try.all.packages …


Gitflow — CC0, by gbrown.

Here are some common git commands for reference.

All commands are based on ssh access and GitHub.

The process used is Gitflow.

Commands are listed with and without the git-flow extension for git.

Clone an existing repository

git clone git@github.com:user/module.git# If there is sub-modules
git clone -recursive git@github.com:user/module.git

Initialize a new repository locally and push it to GitHub.

# Create the git repo locally in the current folder
git init
# Or, with git-flow, this will create also a dev branch
git flow init
# Add files, except what specified in .gitignore
git add .
# Commit
git commit -m "message"
# Ensure that the branch is called "master"
git branch -M master

Thibaut

Publications in French & English about design, innovation and programming.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store