Even though computers are often considered deterministic, computational software is a rapidly evolving and changing landscape. Libraries are constantly adding new features and fixing issues.
Image source: http://www.michaelogawa.com/research/storylines/
Even libraries with the strictest backwards-compatibility policies can change in significant ways.
Image source: http://www.bonkersworld.net/backwards-compatibility/
A reproducible computational environment has a sufficiently consistent state for the computational task at hand.
For example, this can consist of
Does not include:
Image source: https://www.youtube.com/watch?v=g1LgVfV5_ZQ
Image source: http://time-az.com/images/2014/02/20140203carjam.jpg
Linux container systems , like Docker, are new type of tool to easily build, ship, and run reproducible, binary applications.
It is "good enough" for a reproducible computational environment.
In this talk, we will introduce Docker from the perspective a scientific research software engineer. We will
Docker is an open-source engine that automates the deployment of any application as a lightweight, portable, self-sufficient container that will run virtually anywhere.
Docker works with images that consume minimal disk space, versioned, archiveable, and shareable. Executing applications in these images does not require dedicated resources and is high performance.
It works with containers as opposed to virtual machines (VM's).
%time !docker run --rm busybox sh -c 'echo "Hello Docker World!"'
Hello Docker World! CPU times: user 4 ms, sys: 4 ms, total: 8 ms Wall time: 1.23 s
Union file systems, or UnionFS, are file systems that operate by creating layers, making them very lightweight and fast while saving disk space.
Docker can make use of several union file system variants including:
docker <subcommand>
docker push
, docker pull
, docker tag
docker export
will create a archiveable tarball of an image's filesystem.Here's what you need:
!docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE busybox buildroot-2014.02 8c2e06607696 11 weeks ago 2.433 MB busybox latest 8c2e06607696 11 weeks ago 2.433 MB odise/busybox-python latest 649988b8bf0e 4 months ago 20.26 MB
!docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
!docker run -d busybox sh -c 'sleep 3'
3a6bf9d61548ae36bdc0bdb5a87aec17a8056517709c61e2df989aa0a37b7f32
!docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
!docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3a6bf9d61548 busybox:buildroot-2014.02 "sh -c 'sleep 3'" 14 seconds ago Exited (0) 11 seconds ago goofy_almeida a3761241bf97 busybox:buildroot-2014.02 "sh -c 'sleep 3'" 53 minutes ago Exited (0) 53 minutes ago reverent_stallman
!ls $PWD/images/
BackwardsCompatibility.png DockerVM.jpg BuildInstructions1.png Eww.jpg BuildInstructions2.png FilesystemsGeneric.png BuildInstructions3.png itkka.png BuildInstructions4.png Jenkins.png CarJam.jpg Jupyter.png Chroot.png LayerCake.jpg ConfusedCat.jpg Liar.png Debian.png MakerwareScreenshot.png DockerDeploy.jpg MakerwareVTK.png DockerFilesystemsBusybox.png MakerwareWebsite.png DockerFilesystems.svg MasonJar.jpg DockerHub.png ModulesModulesModules.png DockerLogo.png PythonStoryline.svg
!docker run --rm --volume $PWD/images:/images busybox \
sh -c 'ls /images'
BackwardsCompatibility.png BuildInstructions1.png BuildInstructions2.png BuildInstructions3.png BuildInstructions4.png CarJam.jpg Chroot.png ConfusedCat.jpg Debian.png DockerDeploy.jpg DockerFilesystems.svg DockerFilesystemsBusybox.png DockerHub.png DockerLogo.png DockerVM.jpg Eww.jpg FilesystemsGeneric.png Jenkins.png Jupyter.png LayerCake.jpg Liar.png MakerwareScreenshot.png MakerwareVTK.png MakerwareWebsite.png MasonJar.jpg ModulesModulesModules.png PythonStoryline.svg itkka.png
A portable Docker image will only assume standard CPU/memory/disk/network resources are available. If local USB devices and video card devices are used the images will not be runnable anywhere.