In research and development (R&D), interactive computing environments are a frequently employed alternative for data exploration, algorithm development and prototyping. In the last twelve years, a popular scientific computing environment flourished around the Python programming language. Most of this environment is part of (or built over) a software stack named the SciPy Stack. Combined to the OpenCV’s Python interface, this environment becomes an alternative for current computer vision R&D. This tutorial introduces such an environment and shows how it can address different steps on computer vision research, from initial data exploration to parallel computing implementations. Several code examples are presented, addressing problems from simple image processing to inference by machine learning.
About these notebooks
These notebooks are part of the tutorial SciPy and OpenCV as an interactive computing environment for computer vision to be presented at SIBGRAPI 2014. The tutorial will be also available as a full paper in Revista de Informática Teórica e Aplicada (RITA).
Computer vision practitioners need a computing environment that lets them explore data like images, video sequences, point clouds and feature vectors. Such an environment should help on the development and testing of new models and algorithms, and on the deployment of the results, either as a final software module or a scientific/technical publication. It should also provides an extensive list of tools, routines for image processing, machine learning, statistical inference, linear algebra, convex optimization and graph algorithms, just to name a few. Problems involving large sets of images can require high-performance computing (HPC) such that the environment should provide practical ways to parallelize and distribute computation. An ideal environment should also combine documentation and computation in a single bundle, promoting results reproducibility, but preventing the research pipeline to become more cumbersome.
In the last twelve years, a powerful scientific computing environment emerged from the Python programming community. This language is an attractive option for researchers: it is interpreted (a wanted property for interactive computing environments), dynamically typed, and presents a very concise and elegant syntax, resembling the pseudo-code found in computer science textbooks. But the tipping point for Python to become a major player in scientific computing was the advent of an efficient module for $n$-dimensional array representation and manipulation. The Numarray module was created by Greenfield et al. to address astronomical data analysis. In 2005, Numarray successor, NumPy, appeared and became the workhorse of the Python scientific computing. An active community composed by scientists and engineers flourished around Python and NumPy, represented today by the SciPy Stack and the SciPy Conferences.
In computer vision, the practitioner is interested in inferring the world state from images, that act as observations. The statistical relation between the world state and the observed images is defined by models. A particular model is defined by parameters, chosen by learning algorithms. Finally, the world state is estimated by inference algorithms.
This vision on computer vision is properly presented by Prince in his book and translates the state-of-the-art of contemporary research in the field, which is deeply associated to machine learning nowadays.
The development of these models and subsequent problems in learning and inference require a computing environment that allows proper tinkering with machine learning techniques. Visualization and exploration tools are necessary to address problems involving generalization, overfitting, dimensionality reduction and optimization. An interactive computing environment as IPython, enriched with tools from the SciPy Stack and the OpenCV library, can address these needs.
The present tutorial provides a short overview on this Python-based computing environment. Considering the large set of tools available and the space constraints, this tutorial does not pretend to be a complete reference or a broad review. It provides just a glimpse of the environment's capabilities to the computer vision community, briefly presenting and discussing some problems and code examples. These examples can guide the user's first steps and the provided references help on the next ones.
IPython notebooks presenting the complete code for the examples are available on
and on GitHub:
$ sudo apt-get install ipython ipython-qtconsole ipython-notebook $ sudo apt-get install python-scipy python-matplotlib $ sudo apt-get install python-sklearn python-skimage
As a general guideline, users should always consider the most recent installation instructions provided by the packages’ maintainers. To the date, Windows, Mac and Linux users could consider Continuum Analytics’s Anaconda distribution as one of the easiest ways to get all the environment presented in this tutorial. Linux users can also use the package manager of their systems to effortless download and install the components (as the
apt-get system in Debian/Ubuntu distributions). The pip tool is also a practical way to get the packages and keep them updated.