Remote iPython Notebooks

Overview

The focus of this talk is on using an iPython Notebook instance hosted on a remote server. As we have seen, using iPython notebook locally is a fairly straight-forward process. Sometimes, however, it may be advantageous to set up an iPython notebook for use remotely, or to be accessed by multiple users. This requires installing iPython notebook on a centralized server and configuring it to accept remote connections.

image

We will also briefly discuss options for hosting a network-accessable iPython notebook.

Preparing a Notebook for remote access

There are several steps to creating and configuring a notebook server instance (adapted from official instructions provided at the Official iPython Website).

1. Create a profile that defines the notebook server instance.
2. Create a self-signed certificate to enable encrypted communications with the server.
3. Create a password hash to restrict access to the server instance.
4. Edit the profile and specify security settings.
image2a

1. Creating a profile for the server

Creating a profile for the server allows you to manage multiple iPython notebook instances on the server.

$ ipython profile create nbserver

On a linux/unix machine, this will create the ~/.ipython/profile_nbserver directory. In this directory, amongst all the files created by ipython, will be the ipython_notebook_config.py file. This is the file we will be working in for the configuration steps of this process.

2. Creating a self-signed certificate file

Unless you have an institutional security certificate, you will need to create a self-signed certificate file for use with your server. This will allow secure access.

$ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout mycert.pem -out mycert.pem

Caveats regarding the use of a self-signed certificate

Note that when using a self-signed certificate, your browser may complain that the sight is not necessarily valid. Because you created the certificate, it is safe to allow an exception for this server/certificate. If it is possible to instead use a valid signed certificate in place of a self-signed certificate, this is preferable.

3. Creating a password hash

In order to secure your notebook server, it is vital to set a password. This is done by defining a password hash in the notebook configuration file. You will generate the password hash from inside iPython.

$ ipython Python 2.7.3 | 64-bit | (default, Mar 25 2013, 15:52:02) Type "copyright", "credits" or "license" for more information. IPython 1.0.dev -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.
In [1]:
from IPython.lib import passwd

# Note: In practice, passwd() will prompt for a
# password. This is preferable, since otherwise
# your password will be saved inside your session
# history.

passwd("Test Password")
Out[1]:
'sha1:774d9926079f:e585086e10f2f00718a0d9c4ed1b652052d8d463'

4. Configuring the notebook profile

The notebook server is configured by editing the ipython_notebook_config.py file found inside the ~/.ipython/profile_nbserver directory created previously.

Open ipython_notebook_config.py in your favorite editor and add the following:

# Kernel config
c.IPKernelApp.pylab = 'inline'  # if you want plotting support always
# Notebook config
c.NotebookApp.certfile = u'/absolute/path/to/your/certificate/mycert.pem'
c.NotebookApp.ip = '*'
c.NotebookApp.open_browser = False
c.NotebookApp.password = u'sha1:bcd259ccf...your hashed password here'
# It's a good idea to put it on a known, fixed port
c.NotebookApp.port = 9999

Accessing the Notebook

In order to make the server available, you will start it with the following command:

$ ipython notebook --profile=nbserver

The Notebook server can now be accessed via browser, at https://your.host.com:9999.

image </center

Hosting Options

There are multiple options available for hosting an iPython notebook; these options balance convienence vs. cost.

  • Local hosting.
  • Self-Managed remote hosting.
  • Managed remote hosting.

1. Local Hosting

Local hosting refers to using a server that is under your direct control. This can be both easier and more frustrating than using a remote hosting service, whether managed or not.

  • Pros:

    • It is possible to use existing equipment.
    • It is self-managed, so no intermediary for configuration/adding new notebooks.
  • Cons:

    • Requires a little bit of technical expertise to set up.
    • Depending on the computing/network environment, remote access may not be entirely possible.
      • For example, the Unidata firewall blocks access to internal notebook servers from the guest wifi.

2. Self-Managed Cloud-based Services

  • Such as:
    • Rackspace
    • Amazon EC2
  • Pros:
    • Easy to set up and configure.
    • Affordable options available for smaller-scale distributions.
  • Cons:
    • Still self-managed.
    • Pricing can spike quickly depending on total use, bandwidth consumed, underlying processing power.
    • HPC and cluster computing requires manual configuration.

3. Managed Cloud-based Services

Pricing typically based on cores, memory, and total number of instances.

  • Such as:
  • Wakari: http://www.continuum.io/wakari
  • PiCloud: http://www.picloud.com
  • Pros:

    • Provides a remote environment, configured for big-data applications.
    • Managed by server provider.
    • Provides access to large HPC clusters (at a price).
  • Cons:

    • Anything besides basic service level requires a paid account.
    • Accounts can be shared, but cannot share a notebook between accounts.
    • Getting notebooks in to, out of these services can be tricky.
    • You may still need to configure custom packages manually.

Summary

  • Setting up a remotely-accessible iPython notebook is not difficult.
    • Select a host.
    • Create and Configureate a Notebook Server.
  • The choice of host is a trade-off between convienence and cost.