RREF of $$$\left[\begin{array}{ccc}-5 & 1 & 3\\1 & -1 & 1\\3 & 1 & -5\end{array}\right]$$$

Find the reduced row echelon form of $$$\left[\begin{array}{ccc}-5 & 1 & 3\\1 & -1 & 1\\3 & 1 & -5\end{array}\right]$$$.

Divide row $$$1$$$ by $$$-5$$$: $$$R_{1} = - \frac{R_{1}}{5}$$$.

$$$\left[\begin{array}{ccc}1 & - \frac{1}{5} & - \frac{3}{5}\\1 & -1 & 1\\3 & 1 & -5\end{array}\right]$$$

Subtract row $$$1$$$ from row $$$2$$$: $$$R_{2} = R_{2} - R_{1}$$$.

$$$\left[\begin{array}{ccc}1 & - \frac{1}{5} & - \frac{3}{5}\\0 & - \frac{4}{5} & \frac{8}{5}\\3 & 1 & -5\end{array}\right]$$$

Subtract row $$$1$$$ multiplied by $$$3$$$ from row $$$3$$$: $$$R_{3} = R_{3} - 3 R_{1}$$$.

$$$\left[\begin{array}{ccc}1 & - \frac{1}{5} & - \frac{3}{5}\\0 & - \frac{4}{5} & \frac{8}{5}\\0 & \frac{8}{5} & - \frac{16}{5}\end{array}\right]$$$

Multiply row $$$2$$$ by $$$- \frac{5}{4}$$$: $$$R_{2} = - \frac{5 R_{2}}{4}$$$.

$$$\left[\begin{array}{ccc}1 & - \frac{1}{5} & - \frac{3}{5}\\0 & 1 & -2\\0 & \frac{8}{5} & - \frac{16}{5}\end{array}\right]$$$

Add row $$$2$$$ multiplied by $$$\frac{1}{5}$$$ to row $$$1$$$: $$$R_{1} = R_{1} + \frac{R_{2}}{5}$$$.

$$$\left[\begin{array}{ccc}1 & 0 & -1\\0 & 1 & -2\\0 & \frac{8}{5} & - \frac{16}{5}\end{array}\right]$$$

Subtract row $$$2$$$ multiplied by $$$\frac{8}{5}$$$ from row $$$3$$$: $$$R_{3} = R_{3} - \frac{8 R_{2}}{5}$$$.

$$$\left[\begin{array}{ccc}1 & 0 & -1\\0 & 1 & -2\\0 & 0 & 0\end{array}\right]$$$

Since the element at row $$$3$$$ and column $$$3$$$ (pivot element) equals $$$0$$$, we need to swap the rows.

Find the first nonzero element in column $$$3$$$ under the pivot entry.

As can be seen, there are no such entries.

The reduced row echelon form is $$$\left[\begin{array}{ccc}1 & 0 & -1\\0 & 1 & -2\\0 & 0 & 0\end{array}\right]$$$A.