Fortran magic's documentation¶

Fortran magic is an IPython extension that help to use fortran code in an interactive session.

It adds a %%fortran cell magic that compile and import the Fortran code in the cell, using F2py.

The contents of the cell are written to a .f90/.f file in the directory IPYTHONDIR/fortran using a filename with the hash of the code. This file is then compiled. The resulting module is imported and all of its symbols are injected into the user's namespace.

Warning: Starting with python 3.12, numpy switched to using meson instead of distutils. Unfortunately, at the moment, the backward compatibility of f2py based on meson is incomplete.

Author: Martín Gaitán gaitan@gmail.com
Homepage: https://github.com/mgaitan/fortran_magic
Twitter: @tin_nqn_
License: BSD

This software was originally sponsored by Phasety

Feedback, report of issues and pull requests are welcome!

Install or upgrade¶

You can install or upgrade via pip

    pip install -U fortran-magic

or install via conda-forge

    conda install -c conda-forge fortran-magic

Usage¶

Then you are ready to load the magic

In [1]:

%load_ext fortranmagic

To load it each time IPython starts, list it in your configuration file:

c.InteractiveShellApp.extensions = [
    'fortranmagic'
]

In [2]:

import os
import sys

import numpy as np

if sys.platform.startswith("win"):
        # Depends of system, python builds, and compilers compatibility.
        # See below.
    f_config = "--fcompiler=gnu95 --compiler=mingw32"
else:
        # For Unix, compilers are usually more compatible.
    f_config = ""

    # Disable only deprecated NumPy API warning without disable any APIs.
f_config += " --extra '-DNPY_NO_DEPRECATED_API=0'"

%fortran_config {f_config}

New default arguments for %fortran:
	 --extra '-DNPY_NO_DEPRECATED_API=0'

Basic example¶

Just mark the cell with %%fortran in the first line. The code will be highlighted accordingly and compiled when the cell is run

In [3]:

%%fortran

subroutine f1(x, y, z)
    real, intent(in) :: x,y
    real, intent(out) :: z

    z = sin(x+y)

end subroutine f1

In [4]:

%precision %g
f1(1.0, 2.1415)

Out[4]:

9.26574e-05

In [5]:

print(f1.__doc__)

z = f1(x,y)

Wrapper for ``f1``.

Parameters
----------
x : input float
y : input float

Returns
-------
z : float

In [6]:

print(f1.__source__)

subroutine f1(x, y, z)
    real, intent(in) :: x,y
    real, intent(out) :: z

    z = sin(x+y)

end subroutine f1

Verbosity¶

By default the magic only returns output when the compilation process fails. But you can increase the verbosity with the flag -v

In [7]:

%%fortran -v

module hi
  integer :: five = 5
end module   

Ok. The following fortran objects are ready to use: hi

In [8]:

assert hi.five == 5

In [9]:

%%fortran -vv

module hi
  integer :: five = 5
end module   

Running...
   /Users/leo/opt/anaconda3/envs/mtfm12/bin/python -m numpy.f2py -DNPY_NO_DEPRECATED_API=0 -m _fortran_magic_48cb49f6334c4d8b1b1991a488e863ad -c /Users/leo/.cache/ipython/fortranmagic/45b7e251/_fortran_magic_48cb49f6334c4d8b1b1991a488e863ad.f90

Ok. The following fortran objects are ready to use: hi

In [10]:

assert hi.five == 5

In [ ]:

%%fortran -vvv

module hi
  integer :: five = 5
end module   

In [12]:

assert hi.five == 5

Using f2py options¶

Almost all f2py's command line options are exposed to the %%fortran cell magic. See the docstring for detail. For example:

In [13]:

%%fortran -v --f77flags="-ffixed-form" --noarch
C
      SUBROUTINE ZADD(A,B,C,N)
C
      DOUBLE COMPLEX A(*)
      DOUBLE COMPLEX B(*)
      DOUBLE COMPLEX C(*)
      INTEGER N
      DO 20 J = 1, N
         C(J) = A(J)+B(J)
 20   CONTINUE
      END

Ok. The following fortran objects are ready to use: zadd

In [14]:

print(zadd.__doc__)

zadd(a,b,c,n)

Wrapper for ``zadd``.

Parameters
----------
a : input rank-1 array('D') with bounds (*)
b : input rank-1 array('D') with bounds (*)
c : input rank-1 array('D') with bounds (*)
n : input int

In [15]:

a = np.arange(10, dtype=np.cdouble)
b = a*complex(0, 1)
c = np.empty_like(a)
zadd(a, b, c, len(c))
print(c)

[0.+0.j 1.+1.j 2.+2.j 3.+3.j 4.+4.j 5.+5.j 6.+6.j 7.+7.j 8.+8.j 9.+9.j]

Linking resources¶

Use --link option. This is --link-<resource> in f2py command line

In [16]:

%%fortran --link lapack -vv

subroutine solve(A, b, x, n)
    ! solve the matrix equation A*x=b using LAPACK
    implicit none

    real*8, dimension(n,n), intent(in) :: A
    real*8, dimension(n), intent(in) :: b
    real*8, dimension(n), intent(out) :: x

    integer :: i, j, pivot(n), ok

    integer, intent(in) :: n
    x = b

    ! find the solution using the LAPACK routine SGESV
    call DGESV(n, 1, A, n, pivot, x, n, ok)
    
end subroutine

Running...
   /Users/leo/opt/anaconda3/envs/mtfm12/bin/python -m numpy.f2py --link-lapack -DNPY_NO_DEPRECATED_API=0 -m _fortran_magic_a8262a4529f201fe2f2dd196a64e14c8 -c /Users/leo/.cache/ipython/fortranmagic/45b7e251/_fortran_magic_a8262a4529f201fe2f2dd196a64e14c8.f90

Ok. The following fortran objects are ready to use: solve

In [17]:

A = np.array([[1, 2.5], [-3, 4]])
b = np.array([1, 2.5])

In [18]:

print(solve.__doc__)

x = solve(a,b,[n])

Wrapper for ``solve``.

Parameters
----------
a : input rank-2 array('d') with bounds (n,n)
b : input rank-1 array('d') with bounds (n)

Other Parameters
----------------
n : input int, optional
    Default: shape(a, 0)

Returns
-------
x : rank-1 array('d') with bounds (n)

Which is, by the way, the same than

In [19]:

x = np.linalg.solve(A, b)
x

Out[19]:

array([-0.19565217,  0.47826087])

In [20]:

np.testing.assert_allclose(x, solve(A, b))

Extra arguments¶

F2py could have many other arguments. You could append extra arguments with --extra. For example:

  %%fortran --extra '-L/path/to/open/ -lopenblas'

  %%fortran --extra '-D<define> -U<name>'

  %%fortran --extra '-DPREPEND_FORTRAN -DUPPERCASE_FORTRAN'

  %%fortran --extra '-DNPY_NO_DEPRECATED_API=0'

The option --extra could be given multiple times.

Compilers runtime compatibility¶

Incompatibility of compilers or runtime libraries that python was built with, and which are used to build the python extension, can lead to errors during build and/or errors in loading the resulting python extension module.

For example, at the moment, Visual Studio compiler and GNU Fortran (gfortran, formerly g95) are not compatible when used with numpy.f2py. GNU Fortran is compatible with the mingw32 compiler (32-bit or 64-bit), which is available in conda-forge or MSYS2.

Detailed description see:

Numpy issue Can't import module created by f2py "ImportError: DLL load failed" #16416 ;
Numpy documentation PR DOC: Windows and F2PY #20311 .

Save options¶

By default, %%fortran call to f2py without parameters (except, of course, the -m and -c needed to compile a new module). You can change this behaviour with %fortran_config. This line magic can be used in three different ways:

%fortran_config

    Show the current custom configuration

%fortran_config --defaults

    Delete the current configuration and back to defaults

%fortran_config <other options>

    Save (persitently) <other options> to use with %%fortran. The same arguments allowed for `%%fortran` are available

For example, to set the highest verbose level (-vvv) and gnu95 as default --fcompiler:

In [21]:

%fortran_config -vvv --fcompiler gnu95 {f_config}

New default arguments for %fortran:
	-vvv --fcompiler gnu95  --extra '-DNPY_NO_DEPRECATED_API=0'

Now the use of %%fortran will include -vvv --fcompiler gnu95 implicitly

In [ ]:

%%fortran

module hi
  integer :: five = 5
end module   

We can see whatever the default config has

In [23]:

%fortran_config

Current defaults arguments for %fortran:
	-vvv --fcompiler gnu95  --extra '-DNPY_NO_DEPRECATED_API=0'

You can override that global configuration for one specific cell. For example, %%fortran -v will change the the verbose level but still use --fcompiler gnu95

In [24]:

%%fortran -v

module hi
  integer :: five = 5
end module   

Ok. The following fortran objects are ready to use: hi

To clear the custom defaults and back to the defaults (no arguments) use:

In [25]:

%fortran_config --defaults

Deleted custom config. Back to default arguments for %%fortran

In [26]:

%fortran_config {f_config}

New default arguments for %fortran:
	 --extra '-DNPY_NO_DEPRECATED_API=0'

Help on f2py¶

F2py has some flag that output help. See the docstring of %f2py_help

In [27]:

%f2py_help --link blas

Use --dep for meson builds

In [ ]:

%f2py_help --fcompiler

In [ ]:

%f2py_help --compiler

Cache advanced¶

As described above, build is performed in a separate directory. Since name of temporary module is obtained as a hash of the source code, compilation keys and configuration, usually when working with a single notebook, this does not cause problems. However, in difficult cases: parallel work with several notebooks, significant dependence of the assembly on environment variables, etc., conflicts may arise over the names of temporary modules. For example, Windows locks writing to files of already loaded modules, Linux and macOS ignore changes when reloading the extension module into the same process, etc.

Identical modules are not rebuild, previously loaded instances are used.

In [30]:

original_f1 = f1

In [31]:

%%fortran

subroutine f1(x, y, z)
    real, intent(in) :: x,y
    real, intent(out) :: z

    z = sin(x+y)

end subroutine f1

The extension _fortran_magic_0fe80dae42b6629133a40ac6b13ad6b1 is already loaded. To reload it, use:
  %fortran_config --clean-cache

In [32]:

secondary_f1 = f1
assert original_f1.__source__ == secondary_f1.__source__
assert sys.version_info.major < 3 or original_f1.__hash__() == secondary_f1.__hash__()  # or .__code__ compare

`--add-hash=` flags changes hash and, accordingly, name of temporary module.

In [33]:

%%fortran --add-hash=1

subroutine f1(x, y, z)
    real, intent(in) :: x,y
    real, intent(out) :: z

    z = sin(x+y)

end subroutine f1

In [34]:

third_f1 = f1
assert original_f1.__source__ == third_f1.__source__
assert sys.version_info.major < 3 or original_f1.__hash__() != third_f1.__hash__()

Clearing cache directory.

In [35]:

%fortran_config -v --clean-cache

Clean cache: /Users/leo/.cache/ipython/fortranmagic/45b7e251
New cache: /Users/leo/.cache/ipython/fortranmagic/fb6121ce

At the moment, when using additional files or libraries, absolute paths should be used.

In [36]:

%%file tfone.f90

integer function tfone()
    tfone = 1
end

Overwriting tfone.f90

In [37]:

%%fortran --extra {os.path.abspath('.')}/tfone.f90

integer function tfdigits()
    real x
    tfdigits = digits(x)
end

In [38]:

assert tfone() == 1 and tfdigits() == 24

Final¶

Bugs? Ideas? Open an issue
Do you want to collaborate? Fork it! and send a pull-request

Cell tags¶

Tags	Descriptions
`random`	Tests don't check outpus tagged cells.
`random_long`	`Clear Outputs` before commit.
`fast`	Will be tested on `pytest -m 'fast'`
`slow`	Don't tested on `pytest -m 'not slow'` (pytest-astropy plugin: `pytest --run-slow`)
`skip`, `skip_darwin`, `skip_linux`, `skip_win32`	Skip cell (tests don't compute)
`xfail`, `xfail_darwin`, `xfail_linux`, `xfail_win32`	Cell compute, but failed.