#!/usr/bin/env python
# coding: utf-8

# # Least-squares

# In a least-squares, or linear regression, problem, we have measurements $A \in \mathcal{R}^{m \times n}$ and $b \in \mathcal{R}^m$ and seek a vector $x \in \mathcal{R}^{n}$ such that $Ax$ is close to $b$. Closeness is defined as the sum of the squared differences:
# $$ \sum_{i=1}^m (a_i^Tx - b_i)^2, $$
# also known as the $\ell_2$-norm squared, $\|Ax - b\|_2^2$.
# 
# For example, we might have a dataset of $m$ users, each represented by $n$ features. Each row $a_i^T$ of $A$ is the features for user $i$, while the corresponding entry $b_i$ of $b$ is the measurement we want to predict from $a_i^T$, such as ad spending. The prediction is given by $a_i^Tx$.
# 
# We find the optimal $x$ by solving the optimization problem
# $$  
#     \begin{array}{ll}
#     \mbox{minimize}   & \|Ax - b\|_2^2.
#     \end{array}
# $$
# Let $x^\star$ denote the optimal $x$. The quantity $r = Ax^\star - b$ is known as the residual. If $\|r\|_2 = 0$, we have a perfect fit.

# ## Example

# In the following code, we solve a least-squares problem with CVXPY.

# In[1]:


# Import packages.
import cvxpy as cp
import numpy as np

# Generate data.
m = 20
n = 15
np.random.seed(1)
A = np.random.randn(m, n)
b = np.random.randn(m)

# Define and solve the CVXPY problem.
x = cp.Variable(n)
cost = cp.sum_squares(A*x - b)
prob = cp.Problem(cp.Minimize(cost))
prob.solve()

# Print result.
print("\nThe optimal value is", prob.value)
print("The optimal x is")
print(x.value)
print("The norm of the residual is ", cp.norm(A*x - b, p=2).value)