Gordon Bean, April 2015
I want to create python modules directly in the notebook. Like %%file, I want to be able to see the contents of the module, rather than having an additional editor for maintaining the file.
However, rather than just using %%file, I want the code to be automatically imported into my namespace as a module, much like %%cython. New versions of the cell should override existing modules. Such functionality could be obtained using %%file, import, and importlib.reload, but would require two input cells.
So, here I introduce the %%module cell magic. It saves the cell contents in a temporary file (unique to the underlying python kernel) and imports that file to the __main__
namespace as a module.
from beans.magics import ModuleMagics
ip = get_ipython()
ip.register_magics(ModuleMagics)
%%module foobar
a = 1
b = 2
def baz(c, d):
return c + d
<module 'foobar' from '/tmp/.tmp-modules/kernel-24228926-336d-441b-bb58-f83c55cf7bf0/foobar.py'>
foobar.a, foobar.b
(1, 2)
foobar.baz(4,5)
9
The ultimate look under the hood is to examine the source code.
Here is the "cliff-notes" version:
/tmp/.tmp-modules/
.sys.path
, so they can be imported again by the same kernel (including in other modules created by %%module).__dict__
interfaces, any name will probably work, but can't be accessed using normal syntax - I advise avoiding pathology.sys.modules['__main__'].__dict__
). The ModuleMagics class can be instantiated with the keyword namespace
and a dictionary representing the namespace %%module modules will be loaded into.This is likely to change, but this is the current version:
## beans.magics
# Gordon Bean, April 2015
# To use these magics, you must register them with IPython
# e.g:
# from beans.magics import ModuleMagics
# ip = get_ipython()
# ip.register_magics(ModuleMagics)
from IPython.core.magic import Magics, magics_class, cell_magic
import os, sys, importlib
@magics_class
class ModuleMagics(Magics):
'''Magics for creating modules in IPython.'''
def __init__(self, shell=None, namespace=None):
if shell is None:
shell = get_ipython()
super(ModuleMagics, self).__init__(shell)
self.namespace = namespace
# Get the kernel id
self.kernelID = os.path.basename(shell.kernel.config['IPKernelApp']['connection_file'])[:-5]
# Create kernel-specific tmp-module directory
self.module_dir = os.path.join('/tmp/.tmp-modules', self.kernelID)
os.makedirs(self.module_dir, exist_ok=True)
def __del__(self):
# Remove module_dir from file system and sys.path
# I'm not sure this works - evidence so far says no...
tmpfiles = os.listdir(self.module_dir)
for file in tmpfiles:
os.remove(os.path.join(self.module_dir, file))
os.rmdir(self.module_dir)
sys.path.remove(self.module_dir)
@cell_magic
def module(self, line, cell):
'''Import the cell as a module.'''
# Parse module name
tokens = line.split()
name = tokens[0]
# Save to file
filename = os.path.join(self.module_dir, name + '.py')
with open(filename, 'w') as f:
f.write(cell)
# Import module
if self.module_dir not in sys.path:
sys.path.insert(0, self.module_dir)
namespace = self.namespace if self.namespace else sys.modules['__main__'].__dict__
if name in namespace:
# Always reload
del namespace[name]
module = importlib.import_module(name)
namespace[name] = module
return namespace[name]