#!/usr/bin/env python # coding: utf-8 # # SparseEdges # # ## A bio-inspired sparse representation of edges in natural images # # Table of content # # * [What is the SparseEdges package?](#What-is-the-SparseEdges-package?) # * [Installing](#Installing) # * [testing one step](#testing-one-step) # # What is the SparseEdges package? # ================================ # # Our goal here is to build practical algorithms of sparse coding for computer vision. # # The code is available @ https://github.com/bicv/SparseEdges # # This class exploits the LogGabor package to provide with a sparse representation of edges in images. # # This algorithm was presented in the following paper: # # ```bibtex # @inbook{Perrinet15bicv, # title = {Sparse models}, # author = {Perrinet, Laurent U.}, # booktitle = {Biologically-inspired Computer Vision}, # chapter = {13}, # editor = {Keil, Matthias and Crist\'{o}bal, Gabriel and Perrinet, Laurent U.}, # publisher = {Wiley, New-York}, # year = {2015} # } # ``` # # # # This package gives a python implementation. # # Moreover, it gives additional tools to compute useful stistics in images; first- and second order statistics of co-occurences in images. # More information is available @ http://nbviewer.ipython.org/github/bicv/SparseEdges/blob/master/SparseEdges.ipynb # Tests for the packages are available @ http://nbviewer.ipython.org/github/bicv/SparseEdges/blob/master/test-SparseEdges.ipynb. # ## Installing # # To install the ``SparseEdges`` class, you would need the ``LogGabor`` class (multiscale filters) which itself depends on the ``SLIP`` class (the image processing tools). # # pip install git+https://github.com/bicv/SLIP.git # pip install git+https://github.com/bicv/LogGabor.git # pip install git+https://github.com/bicv/SparseEdges.git # # # But before, you need th usual depndencies, such as numpy, matplotlib, pyprind and imageio: # # pip install -U numpy matplotlib pyprind imageio # # [Back to top](#SparseEdges) # In[1]: get_ipython().run_line_magic('cd', '-q probe') from __future__ import division, print_function get_ipython().run_line_magic('matplotlib', 'inline') import os import matplotlib.pyplot as plt import numpy as np np.set_printoptions(precision=4)#, suppress=True) # ### Extracting edges on a sample image # # We will show here how we can simply reconstruct an example image with the list of extracted edges overlaid. # # First we define our object by loading default parameters from internet: # In[2]: from SparseEdges import SparseEdges mp = SparseEdges('https://raw.githubusercontent.com/bicv/SparseEdges/master/default_param.py') print(mp.pe) # At this point, we can change these parameters, by instance by using ``2048`` edges and a different value for the $\alpha$ value in matching pursuit: # In[3]: mp.pe.N = 16 mp.pe.MP_alpha = .9 # We can now load an image and make sure to set the framework to the appropriate size: # In[4]: # defining input image name = 'example' image = mp.imread('https://raw.githubusercontent.com/bicv/SparseEdges/master/database/lena256.png') mp.set_size(image) # ... then, we can initialize the algorithm and normalize the image: # In[5]: mp.init() image = mp.normalize(image, center=True) print(image.mean(), image.std()) # Then, it is easy to run matching pursuit on that image (or load a cached file with the results): # In[6]: print(os.path.join(mp.pe.matpath, name + '.npy')) # In[7]: matname = os.path.join(mp.pe.matpath, name + '.npy') try: edges = np.load(matname) except Exception: edges, C_res = mp.run_mp(image, verbose=True) np.save(matname, edges) # Let's summarize that in one script: # In[8]: get_ipython().run_cell_magic('writefile', 'experiment_example.py', '#! /usr/bin/env python\n# -*- coding: utf8 -*-\nfrom __future__ import division, print_function\n"""\n\nAn example MP run.\n\nTo run:\n$ python experiment_example.py \n\nTo remove cache:\n$ rm -fr **/example*\n\n"""\n__author__ = "(c) Laurent Perrinet INT - CNRS"\n\n\nimport numpy as np\nfrom SparseEdges import SparseEdges\nmp = SparseEdges(\'https://raw.githubusercontent.com/bicv/SparseEdges/master/default_param.py\')\nmp.N = 128\n\nimage = mp.imread(\'https://raw.githubusercontent.com/bicv/SparseEdges/master/database/lena256.png\')\n\nname = \'example\'\nwhite = mp.pipeline(image, do_whitening=True)\n\nimport os\nmatname = os.path.join(mp.pe.matpath, name + \'.npy\')\ntry:\n edges = np.load(matname)\nexcept Exception:\n edges, C_res = mp.run_mp(white, verbose=False)\n np.save(matname, edges) \n') # In[9]: get_ipython().run_line_magic('run', 'experiment_example.py') # Let's show the results of the sparse edge extraction with the edges overlaid on the original image: # In[10]: mp.pe.figsize_edges = 12 mp.pe.line_width = 3. mp.pe.scale = .5 fig, a = mp.show_edges(edges, image=mp.dewhitening(white), show_phase=False, show_mask=True) # Note the dashed circle which (as in Geisler, 2001) shows the limit after which we discard edges. Indeed, when computing statistics (our main goal) we wish to be not perturbed by the fact that images are rectangular. # # Let's show the results of the sparse edge extraction with the edges overlaid on the image reconstructed from the edges: # In[11]: image_rec = mp.reconstruct(edges) fig, a = mp.show_edges(edges, image=mp.dewhitening(image_rec), show_phase=False, show_mask=True) # # Check out [this blog post](https://laurentperrinet.github.io/sciblog/posts/2015-05-22-a-hitchhiker-guide-to-matching-pursuit.html) to learn more about Matching Pursuit. # # ## more examples # First, some tests are available as separate notebooks: # In[12]: from IPython.display import FileLink, FileLinks, Image FileLinks('../notebooks', recursive=False) # [Back to top](#SparseEdges) # ### Effect of parameters on edge extraction: image size # # In[13]: # TODO include figure # ### Effect of parameters on edge extraction: filter parameters # # # In[14]: # TODO include figure # As we test different parameters for the filters, we measured the gain in efficiency for the algorithm as the ratio of the code length to achieve $85\%$ of energy extraction relative to that for the default parameters (white bar). The average is computed on the same database of natural images and error bars denote the standard deviation of gain over the database. First, we studied the effect of the bandwidth of filters respectively in the $\textsf{(A)}$ spatial frequency and $\textsf{(B)}$ orientation spaces. The minimum is reached for the default parameters: this shows that default parameters provide an optimal compromise between the precision of filters in the frequency and position domains for this database. We may also compare pyramids with different number of filters. Indeed, efficiency (in bits) is equal to the number of selected filters times the coding cost for the address of each edge in the pyramid. # We plot here the average gain in efficiency which shows an optimal compromise respectively for respectively $\textsf{(C)}$ the number of orientations and $\textsf{(D)}$ the number of spatial frequencies (scales). Note first that with more than 12 directions, the gain remains stable. Note also that a dyadic scale ratio (that is of 2) is efficient but that other solutions ---such as using the golden section $\phi$--- prove to be significantly more efficient, though the average gain is relatively small (inferior to $5\%$). # ## some book keeping for the notebook # In[15]: get_ipython().run_line_magic('load_ext', 'watermark') get_ipython().run_line_magic('watermark', '') # In[16]: get_ipython().run_line_magic('load_ext', 'version_information') get_ipython().run_line_magic('version_information', 'numpy, scipy, matplotlib, sympy') # In[17]: get_ipython().run_line_magic('cd', '-q ..')