Normal equation
标准方程能够快速计算(在n相对较小的情况)出θ的值不需要迭代。
| Gradient Descent | Normal Equation |
|---|---|
| Need to choose alpha | No need to choose alpha |
| Needs many iterations | No need to iterate |
| O (kn^2 ) | O (n^3), need to calculate inverse of X^T*X |
| Works well when n is large | Slow if n is very large |
In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.
If X^T*X is noninvertible, the common causes might be having :
- Redundant features, where two features are very closely related (i.e. they are linearly dependent)
- Too many features (e.g. m ≤ n). In this case, delete some features or use “regularization” (to be explained in a later lesson).