Orthogonal Vectors and Subspaces

vector $x$ and vector $y$ are orthogonal

$x^Ty=0$

$||x||^2+||y||^2=||x+y||^2$

$x^Tx+y^Ty=(x+y)^T(x+y)=x^Tx+y^Ty+y^Tx+x^Ty=x^Tx+y^Ty+2x^Ty$

$x^Ty=0$

zero vector

• orthogonal to any other vector

subspace $S$ is orthogonal to subspace $T$

• means every vector in $S$ is orthogonal to every vector in $T$
• Example : (Not orthogonal)
  • Two Perpendicular plane in 3 dimensional space
• row space is orthogonal to null space
• Example : 3 dimension space
  • line and a plane ⇒ row space and a null space
  • line as row space and line as null space (X)
  • $A=\begin{bmatrix}1&& 2&& 5\\2&&4&&10\end{bmatrix}$
    • row space is a line
    • null space is a plane

       

      null space and row space are orthogonal complement in $R^n$

      • null space contains all vectors that are perpendicular to row space

 

 

 

•  

Solve $Ax=b$ when there is no solution

• noise in $b$ ⇒ separate noise and information from $b$
• How to solve it
  1. Throw away equations $m>n$
  2. Use $A^TA$
     • Square Matrix
     • Symmetric
     • When is it invertible?
       • $A=\begin{bmatrix}1&& 1\\ 1&&2\\2&&5 \end{bmatrix}$
         • $Ax=b$ is solvable when only $b$ is in the column space
       • $A=\begin{bmatrix}1&& 3\\ 1&&3\\1&&3 \end{bmatrix}$
         • $A^TA$ is not invertible
           • $rank(AB)\leq rank(B)$
           • $rank(A)=1,rank(A^T)=1$
     • $A^TA$ is invertible only when $A$ has independent columns