Linear Systems and Matrices [LA2]
In this series of posts, I`ll be writing about some basics of Linear Algebra [LA] so we can learn together. The book I`ll be using as the material is:
Cabral, M. & Goldfeld, P. (2012). Curso de Álgebra Linear: Fundamentos e Aplicações. Third Edition.
Last time we talked about Vectors, Linear Combination, Span, Basis, and Dimensions. Now, it’s time to know more about linear systems and matrices.
We will talk about:
- Matrices and Vectors in ℝⁿ
- Linear Systems
- Matrices and operations in linear systems
- Operations with Matrices
- Solving Linear Systems
- Matrix-vector product and Dot Product
Linear systems are all around us. We use linear systems, for example, to determine a, b ∈ ℝ such that the line
y = ax + b that cross the plane closest to the dots (x, y), such as:
This method is called Least Squares, and has the objective of finding the approximate solution of a system with 2 variables (a, b) and N equations:
1. Matrices and Vectors in ℝⁿ
A matrix A is an arrangement in a rectangle m x n (where m represents the rows and n the columns) of mn elements aij (i = 1, …, m and j = 1, …, n):
A vector can be thought of as a Matrix with one column. If we have a vector u= (a1, a2, …, an). We can represent the same vector u as
2. Linear System
We classify linear systems as with:
- No solution
- Unique solution
- Infinite solutions
We can think of a system in ℝ as a 1 x 1 system (one variable and one equation), such that ax = b.
In systems in ℝ² (the plane) each equation represent a line (r1 and r2).
and this system can be written as column vectors or matrices
In ℝ² a system with a unique solution is represented by the intersection of non-parallel lines:
In ℝ² a system with no solution can be represented as two lines that do not intersect.
In ℝ² a system with infinite solutions is represented by two lines that intersect at every point.
In ℝ³ we can have a lot to cover, but the idea is the same as in ℝ², some visual representations of ℝ³ results are:
3. Matrices and operations in linear systems
Consider the system with m equations and n variables x₁, x₂, …, xn:
We define the augmented matrix as
3.1 Diagonal Matrix
Some matrix A is diagonal if aij = 0 for all i ≠ j.
To solve a diagonal matrix is pretty easy because we already have the solution in front of us. For example, given the augmented matrix
The values of the variables are
That is, the solution is (5/3, -2, -2).
3.2 Triangular Matrix
3.2.1 Upper triangular matrix
Some matrix A is an upper triangular matrix if aij = 0 for all i > j.
When we have an upper triangular matrix, the solution is calculated with back substitution. In other words, we start with the last equation (where the variable already has a solution) and go upwards, substituting with the values of the variable we found below.
For example, solve the following matrix:
That is the same as the system:
Doing the back substitution, we will first find the x3 of the last equation. Then, we will substitute its value in the second equation to obtain x2. Finally, substituting x3 and x2 in the first equation we will find x1.
3.2.2 Lower triangular matrix
Some matrix A is an lower triangular matrix if aij = 0 for all i < j.
The logic to solve a lower triangular matrix is almost identical to the upper triangular, but we do the other way around with the substitution. That is, we begin from the first row and go substituting the variable value on the following rows.
4. Operations with Matrices
We are able to do some operations in a matrix, such as:
a) Change the order of the rows.
That is, we can change the place of row 2 with row 3 and we’ll be fine.
b) Multiply a row by a scalar k ≠ 0.
c) Substitute a row by its sum with a multiple of another row.
That is, Row1 can become Row1 + 3*Row2.
d) Discard of adding a row with only zeros.
5. Solving Linear Systems
First, we have to define what a pivot is:
a) Gaussian elimination algorithm:
I. p <- (number of non-null rows)
II. k <- 1
III. While k < p, repeat:
- Consider just the lines lk, lk+1, …, lp.
- Identify the non-null column that is more to the left.
- Change rows to obtain the non-null pivot.
- Delete the numbers under the pivot subtracting from lk+1, …,lp multiples of lk.
- p <- (number of non-null rows)
- k <- k + 1
b) Gauss-Jordan elimination algorithm:
I. Execute a)
II. Repeat, for k = p, p-1, …, 1:
- Divide lk by its pivot, making it 1.
- If k > 1, delete the numbers above the pivot subtracting from l1, …, lk-1 multiples of lk.
If you wish to see some examples of the application of those elimination methods, you can easily find it on the internet, here is an example.
6. Matrix-vector product and Dot Product
We can think of a matrix as a set of column vectors or row vectors. That is, if A is a matrix Mm x n, A is made by n column vectors:
Or A is made of m row vectors:
a) Matrix-vector product:
Let
and
. We define the product of matrix A by vector x (Ax ∈ ℝ), by
That is, the matrix-vector product is a linear combination of the columns of the matrix with coefficients given by the vector’s numbers.
For example, if we have
Then,
Or, in other words,
Given a matrix A, and vectors u, v ∈ ℝ and a scalar k, A(u + kv) = Au + kAv.
b) Dot Product:
Given vectors u = (u1, …, un), v = (v1, …, vn) ∈ ℝ, the dot product between u and v is defined by
If u.v = 0, that means the vectors u and v are orthogonal, which means they have a 90° angle between them.
We end this lesson here, I hope it was easy to follow. If you have anything to add or comment, feel free to contact me!
Contact
Stay in touch via LinkedIn ou via email rafavsbastos@gmail.com
I put all my codes on https://github.com/rafavsbastos
