# Image Approximation with Fourier and Wavelets¶


This numerical tour overviews the use of Fourier and wavelets for image approximation.

In [1]:
using PyPlot
using NtToolBox
arequire("NtToolBox")

WARNING: Base.UTF8String is deprecated, use String instead.
WARNING: Base.UTF8String is deprecated, use String instead.
WARNING: Base.UTF8String is deprecated, use String instead.
WARNING: Base.UTF8String is deprecated, use String instead.
WARNING: Base.UTF8String is deprecated, use String instead.
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
WARNING: Base.UTF8String is deprecated, use String instead.
likely near In[1]:4
in depwarn(::String, ::Symbol) at .\deprecated.jl:64
in #parse_file#6(::Array{Any,1}, ::Function, ::String) at C:\Users\Ayman\.julia\v0.5\Autoreload\src\files.jl:63
in execute_request(::ZMQ.Socket, ::IJulia.Msg) at C:\Users\Ayman\.julia\v0.5\IJulia\src\execute_request.jl:157
in eventloop(::ZMQ.Socket) at C:\Users\Ayman\.julia\v0.5\IJulia\src\eventloop.jl:8
WARNING: replacing module NtToolBox


Note: to measure the error of an image $f$ with its approximation $f_M$, we use the SNR measure, defined as

$$\text{SNR}(f,f_M) = -20\log_{10} \pa{ \frac{ \norm{f-f_M} }{ \norm{f} } },$$

which is a quantity expressed in decibels (dB). The higer the SNR, the better the quality.

First we load an image $f \in \RR^N$ of $N = N_0 \times N_0$ pixels.

In [2]:
n0 = 512

WARNING: Base.UTF8String is deprecated, use String instead.
likely near C:\Users\Ayman\.julia\v0.5\IJulia\src\kernel.jl:31
WARNING: Base.UTF8String is deprecated, use String instead.
likely near C:\Users\Ayman\.julia\v0.5\IJulia\src\kernel.jl:31
WARNING: Base.UTF8String is deprecated, use String instead.
likely near C:\Users\Ayman\.julia\v0.5\IJulia\src\kernel.jl:31
WARNING: Base.UTF8String is deprecated, use String instead.
likely near C:\Users\Ayman\.julia\v0.5\IJulia\src\kernel.jl:31
WARNING: Base.UTF8String is deprecated, use String instead.
likely near C:\Users\Ayman\.julia\v0.5\IJulia\src\kernel.jl:31
WARNING: Base.UTF8String is deprecated, use String instead.
likely near C:\Users\Ayman\.julia\v0.5\IJulia\src\kernel.jl:31


Display the original image.

In [3]:
figure(figsize = (5,5))
imageplot(f, "Image f")

Out[3]:
PyObject <matplotlib.text.Text object at 0x000000001E9D20B8>

Display a zoom in the middle.

In [4]:
figure(figsize = (5,5))
imageplot(f[Int(n0/2 - 32) : Int(n0/2 + 32), Int(n0/2 - 32) : Int(n0/2 + 32)], "Zoom")

Out[4]:
PyObject <matplotlib.text.Text object at 0x000000001E7DA240>

An image is a 2D array, it can be modified as a matrix.

In [5]:
figure(figsize = (8,8))
imageplot(-f, "-f", [1, 2, 1])
imageplot(f[end:-1:1, :], "Flipped", [1, 2, 2])

Out[5]:
PyObject <matplotlib.text.Text object at 0x000000001EF47898>

Blurring is achieved by computing a convolution $f \star h$ with a kernel $h$.

Compute the low pass kernel.

In [6]:
k = 9; #size of the kernel
h = ones(k, k)
h = h/sum(h); #normalize

Out[6]:
9×9 Array{Float64,2}:
0.0123457  0.0123457  0.0123457  …  0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457  …  0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457
0.0123457  0.0123457  0.0123457     0.0123457  0.0123457  0.0123457

Compute the convolution $f \star h$.

In [7]:
fh = conv2(Array{Float64, 2}(f), h);


Display.

In [8]:
figure(figsize = (5,5))
imageplot(fh, "Blurred image")

Out[8]:
PyObject <matplotlib.text.Text object at 0x000000001F218C50>

## Fourier Transform¶

The Fourier orthonormal basis is defined as $$\psi_m(k) = \frac{1}{\sqrt{N}}e^{\frac{2i\pi}{N_0} \dotp{m}{k} }$$ where $0 \leq k_1,k_2 < N_0$ are position indexes, and $0 \leq m_1,m_2 < N_0$ are frequency indexes.

The Fourier transform $\hat f$ is the projection of the image on this Fourier basis

$$\hat f(m) = \dotp{f}{\psi_m}.$$

The Fourier transform is computed in $O(N \log(N))$ operation using the FFT algorithm (Fast Fourier Transform). Note the normalization by $\sqrt{N}=N_0$ to make the transform orthonormal.

In [9]:
F = plan_fft(f)
F = (F*f)/n0;


We check this conservation of the energy.

In [10]:
println(@sprintf("Energy of Image:   %f", norm(f)))
println(@sprintf("Energy of Fourier:   %f", norm(F)))

Energy of Image:   262.554108
Energy of Fourier:   262.554138


Compute the logarithm of the Fourier magnitude $\log\left(\abs{\hat f(m)} + \epsilon\right)$, for some small $\epsilon$.

In [11]:
L = fftshift(log(abs(F) + 1e-1));


Display. Note that we use the function fftshift to put the 0 low frequency in the middle.

In [12]:
figure(figsize = (5,5))
imageplot(L, "Log(Fourier transform)")

Out[12]:
PyObject <matplotlib.text.Text object at 0x00000000244FA978>

## Linear Fourier Approximation¶

An approximation is obtained by retaining a certain set of index $I_M$

$$f_M = \sum_{ m \in I_M } \dotp{f}{\psi_m} \psi_m.$$

Linear approximation is obtained by retaining a fixed set $I_M$ of $M = \abs{I_M}$ coefficients. The important point is that $I_M$ does not depend on the image $f$ to be approximated.

For the Fourier transform, a low pass linear approximation is obtained by keeping only the frequencies within a square.

$$I_M = \enscond{m=(m_1,m_2)}{ -q/2 \leq m_1,m_2 < q/2 }$$

where $q = \sqrt{M}$.

This can be achieved by computing the Fourier transform, setting to zero the $N-M$ coefficients outside the square $I_M$ and then inverting the Fourier transform.

Number $M$ of kept coefficients.

In [13]:
M = Base.div(n0^2, 64);


Exercise 1

Perform the linear Fourier approximation with $M$ coefficients. Store the result in the variable $f_M$.

In [14]:
include("NtSolutions/introduction_4_fourier_wavelets/exo1.jl")

Out[14]:
PyObject <matplotlib.text.Text object at 0x0000000024A79908>
In [15]:
## Insert your code here.


Compare two 1D profile (lines of the image). This shows the strong ringing artifact of the linea approximation.

In [15]:
figure(figsize = (7, 6))

subplot(2, 1, 1)
plot(f[: , Base.div(n0, 2)])
xlim(0, n0)
title("f")

subplot(2, 1, 2)
plot(fM[: , Base.div(n0, 2)])
xlim(0, n0)
title("f_M")

show()