Gradient Descent

Gradient descent algorithm
repeat until convergence {
\theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j} J(\theta_0, \theta_1) (for j=0 and j=1)
}

\alpha: learning rate
a:=b: assigning b to a

Simultaneous update
temp0 := \theta_0 - \alpha \frac{\partial}{\partial \theta_0} J(\theta_0, \theta_1)
temp1 := \theta_1 - \alpha \frac{\partial}{\partial \theta_1} J(\theta_0, \theta_1)
\theta_0 := temp0
\theta_1 := temp1

Gradient descent for linear regression
repeat until convergence {

    \[<span class="ql-right-eqno">   </span><span class="ql-left-eqno">   </span><img src="https://teach.sg/wp-content/ql-cache/quicklatex.com-776add333c9f68e7c5d7e1045a24c150_l3.png" height="109" width="270" class="ql-img-displayed-equation quicklatex-auto-format" alt="\begin{align*}   \theta_0 :=  \theta_0 - \alpha \frac{1}{m} \sum\limits_{i=1}^{m}(h_\theta(x_{i}) - y_{i}) \\   \theta_1 :=  \theta_1 - \alpha \frac{1}{m} \sum\limits_{i=1}^{m}\left((h_\theta(x_{i}) - y_{i}) x_{i}\right)    \end{align*}" title="Rendered by QuickLaTeX.com"/>\]

}

loading
×