During this course we'll be doing nearly everything using the IPython notebook. For that reason, we'll start with a quick intro to the IPython environment and platform.
The basic IPython client: at the terminal, simply type ipython
:
$ ipython
Python 2.7.4 (default, Apr 19 2013, 18:28:01)
Type "copyright", "credits" or "license" for more information.
IPython 1.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: print "hello world"
hello world
When executing code in IPython, all valid Python syntax works as-is, but IPython provides a number of features designed to make the interactive experience more fluid and efficient.
In the notebook, to run a cell of code, hit Shift-Enter
. This executes the cell and puts the cursor in the next cell below, or makes a new one if you are at the end. Alternately, you can use:
Alt-Enter
to force the creation of a new cell unconditionally (useful when inserting new content in the middle of an existing notebook).Control-Enter
executes the cell and keeps the cursor in the same cell, useful for quick experimentation of snippets that you don't need to keep permanently.print("Hello")
Hello
?
?
and ??
¶Typing object_name?
will print all sorts of details about any object, including docstrings, function definition lines (for call arguments) and constructor details for classes.
import collections
collections.namedtuple?
collections.Counter??
*int*?
An IPython quick reference card:
%quickref
Tab completion, especially for attributes, is a convenient way to explore the structure of any object you’re dealing with. Simply type object_name.<TAB>
to view the object’s attributes. Besides Python objects and keywords, tab completion also works on file and directory names.
# type tab after the '.' below:
collections.
2+10
12
_+10
22
You can suppress the storage and rendering of output if you append ;
to the last cell (this comes in handy when plotting with matplotlib, for example):
10+20;
_
22
The output is stored in _N
and Out[N]
variables:
_10 == Out[10]
True
Out
{8: 22, 10: 22, 11: True, 7: 12}
And the last three have shorthands for convenience:
print('last output :', _)
print('second to last :', __)
print('third to last :', ___)
last output : True second to last : 22 third to last : 22
In[11]
'_10 == Out[10]'
_i
'In[11]'
_ii
'In[11]'
print('last input :', _i)
print('second to last :', _ii)
print('third to last :', _iii)
last input : _ii second to last : _i third to last : In[11]
%history
print("Hello") ? import collections collections.namedtuple? collections.Counter?? *int*? %quickref 2+10 _+10 10+20; _ _10 == Out[10] Out print('last output :', _) print('second to last :', __) print('third to last :', ___) In[11] _i _ii print('last input :', _i) print('second to last :', _ii) print('third to last :', _iii) %history
Note: the commands below work on Linux or Macs, but may behave differently on Windows, as the underlying OS is different. IPython's ability to access the OS is still the same, it's just the syntax that varies per OS.
!pwd
/Users/jakevdp/Opensource/OsloWorkshop2014/notebooks
files = !ls
print("My current directory's files:")
print(files)
My current directory's files: ['01.1_IPythonIntro.ipynb', '01.2_NumpyPandas.ipynb', '01.3_PandasBreakout.ipynb', '01.4_MatplotlibSeaborn.ipynb', '01.5_VisualizationBreakout.ipynb', '02.1_ModelFitting.ipynb', '02.2_ModelFittingBreakout.ipynb', '02.3_ScikitLearnIntro.ipynb', '02.4_MachineLearningBreakout.ipynb', '03.1-Classification-SVMs.ipynb', '03.2-Regression-Forests.ipynb', '03.3-Validation.ipynb', '03.4-Validation-Breakout.ipynb', '04.1-Dimensionality-PCA.ipynb', '04.2-Clustering-KMeans.ipynb', '04.3-Density-GMM.ipynb', '05.4-Unsupervised-Breakout.ipynb', 'BabyNames.ipynb', 'ExploreData.ipynb', 'FaceRecognition.ipynb', 'Index.ipynb', 'TextMining.ipynb', 'Untitled0.ipynb', '__pycache__', 'data', 'fig_code', 'images', 'solutions', 'tmp.ipynb']
!echo $files
[01.1_IPythonIntro.ipynb, 01.2_NumpyPandas.ipynb, 01.3_PandasBreakout.ipynb, 01.4_MatplotlibSeaborn.ipynb, 01.5_VisualizationBreakout.ipynb, 02.1_ModelFitting.ipynb, 02.2_ModelFittingBreakout.ipynb, 02.3_ScikitLearnIntro.ipynb, 02.4_MachineLearningBreakout.ipynb, 03.1-Classification-SVMs.ipynb, 03.2-Regression-Forests.ipynb, 03.3-Validation.ipynb, 03.4-Validation-Breakout.ipynb, 04.1-Dimensionality-PCA.ipynb, 04.2-Clustering-KMeans.ipynb, 04.3-Density-GMM.ipynb, 05.4-Unsupervised-Breakout.ipynb, BabyNames.ipynb, ExploreData.ipynb, FaceRecognition.ipynb, Index.ipynb, TextMining.ipynb, Untitled0.ipynb, __pycache__, data, fig_code, images, solutions, tmp.ipynb]
!echo {files[0].upper()}
01.1_IPYTHONINTRO.IPYNB
The IPyhton 'magic' functions are a set of commands, invoked by prepending one or two %
signs to their name, that live in a namespace separate from your normal Python variables and provide a more command-like interface. They take flags with --
and arguments without quotes, parentheses or commas. The motivation behind this system is two-fold:
To provide an orthogonal namespace for controlling IPython itself and exposing other system-oriented functionality.
To expose a calling mode that requires minimal verbosity and typing while working interactively. Thus the inspiration taken from the classic Unix shell style for commands.
%magic
Line vs cell magics:
%timeit range(10)
1000000 loops, best of 3: 246 ns per loop
%%timeit
range(10)
range(100)
1000000 loops, best of 3: 502 ns per loop
Line magics can be used even inside code blocks:
for i in range(5):
size = i*100
print('size:',size, end=' ')
%timeit range(size)
size: 0 10000000 loops, best of 3: 155 ns per loop size: 100 1000000 loops, best of 3: 250 ns per loop size: 200 1000000 loops, best of 3: 251 ns per loop size: 300 1000000 loops, best of 3: 279 ns per loop size: 400 1000000 loops, best of 3: 279 ns per loop
Magics can do anything they want with their input, so it doesn't have to be valid Python:
%%bash
echo "My shell is:" $SHELL
echo "User:" $USER
My shell is: /bin/bash User: jakevdp
Another interesting cell magic: create any file you want locally from the notebook:
%%file test.txt
This is a test file!
It can contain anything I want...
more...
Writing test.txt
!cat test.txt
This is a test file! It can contain anything I want... more...
Let's see what other magics are currently defined in the system:
%lsmagic
Available line magics: %alias %alias_magic %autocall %automagic %autosave %bookmark %cat %cd %clear %colors %config %connect_info %cp %debug %dhist %dirs %doctest_mode %ed %edit %env %gui %hist %history %install_default_config %install_ext %install_profiles %killbgscripts %ldir %less %lf %lk %ll %load %load_ext %loadpy %logoff %logon %logstart %logstate %logstop %ls %lsmagic %lx %macro %magic %man %matplotlib %mkdir %more %mv %notebook %page %pastebin %pdb %pdef %pdoc %pfile %pinfo %pinfo2 %popd %pprint %precision %profile %prun %psearch %psource %pushd %pwd %pycat %pylab %qtconsole %quickref %recall %rehashx %reload_ext %rep %rerun %reset %reset_selective %rm %rmdir %run %save %sc %store %sx %system %tb %time %timeit %unalias %unload_ext %who %who_ls %whos %xdel %xmode Available cell magics: %%! %%HTML %%SVG %%bash %%capture %%debug %%file %%html %%javascript %%latex %%perl %%prun %%pypy %%python %%python2 %%python3 %%ruby %%script %%sh %%svg %%sx %%system %%time %%timeit %%writefile Automagic is ON, % prefix IS NOT needed for line magics.
Not only can you input normal Python code, you can even paste straight from a Python or IPython shell session:
>>> # Fibonacci series:
... # the sum of two elements defines the next
... a, b = 0, 1
>>> while b < 10:
... print(b)
... a, b = b, a+b
1 1 2 3 5 8
In [1]: for i in range(5):
...: print(i)
...:
0 1 2 3 4
This imports numpy as np
and matplotlib's plotting routines as plt
, plus setting lots of other stuff for you to work interactivel very easily:
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.pyplot import gcf
x = np.linspace(0, 2*np.pi, 300)
y = np.sin(x**2)
plt.plot(x, y)
plt.title("A little chirp")
f = gcf() # let's keep the figure object around for later...
IPython Notebooks are just files (.ipynb
) on your file system
The Notebook server is aware of Notebooks in the directory (and in subdirectories) of the location where it is started.
If you cd to a Notebook directory and type:
ipython notebook
you will see the Notebooks in that directory in the dashboard
.ipynb
) on your file systemfrom IPython.nbformat import current
with open('01.1_IPythonIntro.ipynb') as f:
nb = current.read(f, 'json')
nb.worksheets[0].cells[0:5]
[{'metadata': {'slideshow': {'slide_type': 'slide'}}, 'cell_type': 'markdown', 'source': "# IPython: an environment for interactive computing\n\nDuring this course we'll be doing nearly everything using the IPython notebook. For that reason, we'll start with a quick intro to the IPython environment and platform."}, {'metadata': {'slideshow': {'slide_type': 'fragment'}}, 'cell_type': 'markdown', 'source': '## What is IPython?\n\n- Short for **I**nteractive **Python**\n- IPython is a platform for you to *interact* with your code and data\n- The *notebook*: a system for *literate computing*\n * The combination of narrative, code and results\n * Weave your scientific narratives together with your computational process\n- Tools for easy parallel computing\n * Interact with *many* processes'}, {'level': 1, 'cell_type': 'heading', 'metadata': {'slideshow': {'slide_type': 'slide'}}, 'source': 'IPython at the terminal'}, {'metadata': {'slideshow': {'slide_type': 'fragment'}}, 'cell_type': 'markdown', 'source': 'The basic IPython client: at the terminal, simply type `ipython`:\n\n $ ipython\n Python 2.7.4 (default, Apr 19 2013, 18:28:01) \n Type "copyright", "credits" or "license" for more information.\n \n IPython 1.0.0 -- An enhanced Interactive Python.\n ? -> Introduction and overview of IPython\'s features.\n %quickref -> Quick reference.\n help -> Python\'s own help system.\n object? -> Details about \'object\', use \'object??\' for extra details.\n \n In [1]: print "hello world"\n hello world\n'}, {'metadata': {'slideshow': {'slide_type': 'slide'}}, 'cell_type': 'markdown', 'source': '# Some tutorial help/resources :\n\n - The [IPython website](http://ipython.org)\n - The [IPython book](<a href="http://www.packtpub.com/learning-ipython-for-interactive-computing-and-data-visualization/book)\n - Search for "IPython in depth" tutorial on youtube and pyvideo, much longer, much deeper\n - Ask for help on [Stackoverflow, tag it "ipython"](http://stackoverflow.com/questions/tagged/ipython)\n - [Mailing list](http://mail.scipy.org/mailman/listinfo/ipython-dev)\n - File a [github issue](http://github.com/ipython/ipython)\n - [Twitter](https://twitter.com/IPythonDev)\n - [Reddit](http://www.reddit.com/r/IPython)\n - [Notebook Gallery](https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks)'}]
IPython Notebooks can also be exported to .py
files (see "File:Download As" menu item). You can tell the Notebook server to always save these .py
files alongside the .ipynb
files by starting the Notebook as:
ipython notebook --script
You can import Notebooks from the main Dashboard or simply by copying a Notebook into the Notebook directory.
Shift-Enter
to run a cellCtrl-Enter
to run a cell in placeCtrl-m ?
Ctrl-m h