# ipythonPexpect example¶

Adam Lyon, Fermi National Accelerator Laboratory, lyon at fnal.gov, March 2013

Updated May 2015 - The input command and prompt are not repeated in the result cell unless you set "-a".

ipythonPexpect is an IPython notebook extension that interfaces with the pexpect python module by Noah Spurrier. See https://pypi.python.org/pypi/pexpect/ and http://www.noah.org/wiki/Pexpect for detailed information on pexpect.

pexpect is a module that spawns a process that starts some application program (like a shell, R, CERN's Root, even Python). You can then feed it commands, capture the output, and wait for the next prompt. Most any command line driven application that provides a consistent prompt can be used. We'll do some examples here.

There is also a feature to lock the notebook such that all subsequent commands that you give, even without the magic, will go to the pexpect session. This is very convenient for long sessions, but may be surprising to other readers. So be sure to explain what you are doing carefully.

First, we load the notebook extension.

In [1]:
%ipythonPexpect? for help

## bash¶

There are some prewritten spawn scripts to start bash, R, CERN's ROOT, and python. Let's try a bash shell (note that prompt is changed to a known string).

In [2]:
%pexpect_spawn_bash
Opened connection to /usr/bin/env bash
bash-3.2$PS1='bash> ' bash> We can now send commands to this bash session with %%P. In [3]: %%P date Tue May 12 00:39:20 CDT 2015 pexpect sessions are persistent, so the state is maintained between commands (unlike ! which starts a new shell on each call) In [4]: !export FOO="hi" In [5]: !echo$FOO

In [6]:
%%P
export FOO="hi"
In [7]:
%%P
echo $FOO hi If you want to see output in real time, you need to set the -a option to show the command and the output. In [8]: %%P -a for i in$(seq 1 5); do sleep 2; date; done
bash> for i in $(seq 1 5); do sleep 2; date; done Tue May 12 00:39:22 CDT 2015 Tue May 12 00:39:24 CDT 2015 Tue May 12 00:39:27 CDT 2015 Tue May 12 00:39:29 CDT 2015 Tue May 12 00:39:31 CDT 2015 bash> Note that blank lines and white space are seen In [9]: %%P -a echo "hi" bash> echo "hi" hi bash> bash> When done, close the session In [10]: %pexpect_close Closed connection to /usr/bin/env bash ## python¶ Let's try a python session In [11]: %pexpect_spawn_python Opened connection to /usr/bin/env python Python 2.7.9 |Anaconda 2.2.0 (x86_64)| (default, Dec 15 2014, 10:37:34) [GCC 4.2.1 (Apple Inc. build 5577)] on darwin Type "help", "copyright", "credits" or "license" for more information. Anaconda is brought to you by Continuum Analytics. Please check out: http://continuum.io/thanks and https://binstar.org >>> In [12]: %%P print 'hi' hi In [13]: %%P print 'hi' print 'bye' hi bye For the command below, note the two blank lines at the end (for some reason, nbviewer doesn't show it, but it's there in the actual notebook) In [14]: %%P for i in range(5): print i 0 1 2 3 4 In [15]: %%P print 'hi'+ \ ' adam' hi adam In [16]: %pexpect_close Closed connection to /usr/bin/env python ## R¶ Try R In [17]: %pexpect_spawn_R Opened connection to R R version 3.1.3 (2015-03-09) -- "Smooth Sidewalk" Copyright (C) 2015 The R Foundation for Statistical Computing Platform: x86_64-apple-darwin13.4.0 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > In [18]: %%P ls() character(0) In [19]: %%P a <- rnorm(20000) head(a) [1] 0.52805098 0.34819182 1.01972016 0.74633602 0.04287176 0.58574984 You can add an image if you know the output file name. In [20]: %%P -f rout.png png("rout.png") hist(a) dev.off() null device 1 Out[20]: You can return values to python (you may have to slice and dice the return string) with the -e (evaluate) option. In [21]: pya = %%P -e mean(a) [1] 0.009820224 In [22]: pya Out[22]: '[1] 0.009820224' In [23]: %pexpect_close Closed connection to R ## CERN ROOT¶ Note that ROOT is a special challenge because it does syntax highlighting and coloring at its command line. This leads to lots of ugly ANSI codes littering the output. The %pexpect_spawn_root magic writes a .rootrc file to the current directory that will turn Root to turn off this highlighting. In [24]: %pexpect_spawn_root A .rootrc file is automatically written to turn off root prompt syntax coloring. But it cannot be written because one already exists. Continuing assuming .rootrc turns off syntax coloring. ------ Opened connection to root -b ******************************************* * * * W E L C O M E to R O O T * * * * Version 5.34/25 12 January 2015 * * * * You are welcome to visit our Web site * * http://root.cern.ch * * * ******************************************* ROOT 5.34/25 (heads/v5-34-00-patches@v5-34-24-89-g05f58e1, Apr 17 2015, 12:16:00 on macosx64) CINT/ROOT C/C++ Interpreter version 5.18.00, July 2, 2010 Type ? for help. Commands must be C++ statements. Enclose multiple statements between { }. root [0] We're going to try the phase space tutorial In [25]: %%P gSystem.Load("libPhysics") (int)0 I'm getting tired of entering %%P. I'm going to lock the notebook in Root mode. Make sure you know what you are doing here. When I'm done, I have to %pexpect_unlock in order for the notebook to process IPython again. In [26]: %pexpect_lock WARNING: All future cell execution will be processed through pexpect! To return to IPython, issue %pexpect_unlock Continuing onward... In [27]: TLorentzVector target(0.0, 0.0, 0.0, 0.938) TLorentzVector beam(0.0, 0.0, 0.65, 0.65) TLorentzVector W = beam + target In [28]: Double_t masses[3] = { 0.938, 0.139, 0.139 } In [29]: masses (Double_t*)0x7fc81963d7a0 In [30]: TGenPhaseSpace event In [31]: event.SetDecay(W, 3, masses) (Bool_t)1 In [32]: TH2F *h2 = new TH2F("h2","h2", 50,1.1,1.8, 50,1.1,1.8); For some reason, if you don't put blocks of code within braces, you get syntax coloring again which confuses the code that removes the commands from the output. In [33]: { for (Int_t n=0;n<100000;n++) { Double_t weight = event.Generate(); TLorentzVector *pProton = event.GetDecay(0); TLorentzVector *pPip = event.GetDecay(1); TLorentzVector *pPim = event.GetDecay(2); TLorentzVector pPPip = *pProton + *pPip; TLorentzVector pPPim = *pProton + *pPim; h2->Fill(pPPip.M2() ,pPPim.M2() ,weight); } } In [34]: h2 (class TH2F*)0x7fc81bf25ae0 In [35]: h2->Draw() Info in <TCanvas::MakeDefCanvas>: created default TCanvas with name c1 Event though I'm in "locked" mode, I can still issue %%P if I need to give it options. In [36]: %%P -f root.png c1.SaveAs("root.png") Info in <TCanvas::Print>: png file root.png has been created Out[36]: In [37]: %pexpect_unlock Notebook will use IPython In [38]: %pexpect_close Closed connection to root -b ## Your own application¶ Use the %pexpect_spawn command and be sure to ask the system for help for the options. In [39]: %pexpect_spawn? Let's try to connect to ruby. Note that irb immediately shows the prompt with no preamble. But the prompt is pretty involved, so we will leave out the \r\n. In [40]: %pexpect_spawn -p 'irb.+> ' -c 'irb.+\* ' -e irb Opened connection to irb irb(main):001:0> In [41]: %%P 4 => 4 In [42]: %%P puts("hi") hi => nil In [43]: %%P 2 + 2 => 4 In [44]: %%P -a 2 + 2 irb(main):005:0> 2 + irb(main):006:0* 2 => 4 irb(main):007:0> In [45]: %%P$i = 0
$num = 5 until$i > $num do puts("Inside the loop i = #$i" )
\$i +=1;
end
=> 0
=> 5
Inside the loop i = 0
Inside the loop i = 1
Inside the loop i = 2
Inside the loop i = 3
Inside the loop i = 4
Inside the loop i = 5
=> nil
In [46]:
%pexpect_close
Closed connection to irb