https://www.notion.so/Optimization-Methods-f380a6e8e1ca4a8699cd63d0aa298290

For non-closed form functions: local search / hill climb

• w_t+1 = w_t + alpha * -gradientL(w_t)
• puts maximum weight on the points xi, yi, where w_t performing badly

Derivative: (scalar)