Convert ns-ALEX Becker-Hickl SPC/SET files to Photon-HDF5


Summary

This [Jupyter notebook](https://jupyter.org/) will guide you through the conversion of a ns-ALEX data file from **SPC/SET** to [Photon-HDF5](http://photon-hdf5.org) format. For more info on how to edit a jupyter notebook refer to [this example](http://nbviewer.jupyter.org/github/jupyter/notebook/blob/master/docs/source/examples/Notebook/Notebook%20Basics.ipynb#The-Notebook-dashboard).

If you are running this notebook online please note that this is a demo service and the file size of input data files is limited to 35MB.

Please send feedback and report any problems to the [Photon-HDF5 google group](https://groups.google.com/forum/#!forum/photon-hdf5).

1. How to run it?

The notebook is composed by "text cells", such as this paragraph, and "code cells" containing the code to be executed (and identifyied by an In [ ] prompt). To execute a code cell, select it and press SHIFT+ENTER. To modify an cell, click on it to enter "edit mode" (indicated by a green frame), than type.

You can run this notebook directly online (for demo purposes), or you can run it on your on desktop. For a local installation please refer to:


Please run each each code cell using SHIFT+ENTER.

2. Prepare the data file

2.1 Upload the data file

Note: Skip to section 2.2 if you are running the notebook locally.

To start, we need to upload the file we want to convert to Photon-HDF5. You can use one of the example data files freely available on on figshare.

To upload the file switch to the "Home" tab in your browser, click the upload button and select the data file. Wait until the upload completes than switch back to this notebook.

2.2 Select the file

Specify the file name of the input data file in the following cell:

In [ ]:
filename = 'dsdna_d7_d17_50_50_1.spc'

The next cell will check if the filename location is correct:

In [ ]:
import os
try: 
    with open(filename): pass
    print('Data file found, you can proceed.')
except IOError:
    print('ATTENTION: Data file not found, please check the filename.\n'
          '           (current value "%s")' % filename)

In case of file not found, please double check the file name and that the file has been uploaded.

3. Load the data

We start by loading the software:

In [ ]:
%matplotlib inline
import numpy as np
import phconvert as phc
print('phconvert version: ' + phc.__version__)

Then we load the input file:

In [ ]:
d, meta = phc.loader.nsalex_bh(filename,
                               donor = 4,
                               acceptor = 6,
                               laser_repetition_rate = 40e6,
                               tcspc_range = 60e-9,
                               timestamps_unit = 60e-9,
                               alex_period_donor = (1800, 3300),
                               alex_period_acceptor = (270, 1500),
                               excitation_wavelengths = (532e-9, 635e-9),
                               detection_wavelengths = (580e-9, 680e-9),
                               allow_missing_set = False)

And we plot the nanotimes histogram:

In [ ]:
phc.plotter.alternation_hist(d)

The previous plot is the nanotimes histogram for the donor and acceptor channel separately. The shaded areas marks the donor (green) and acceptor (red) excitation periods.

If the histogram looks wrong in some aspects (no photons, wrong detectors assignment, wrong period selection) please go back to the previous cell which loads the file and change the parameters until the histogram looks correct.

You may also find useful to see how many different detectors are present and their number of photons. This information is shown in the next cell:

In [ ]:
detectors = d['photon_data']['detectors']

print("Detector    Counts")
print("--------   --------")
for det, count in zip(*np.unique(detectors, return_counts=True)):
    print("%8d   %8d" % (det, count))

4. Metadata

In the next few cells, we specify some metadata that will be stored in the Photon-HDF5 file. Please modify these fields to reflect the content of the data file:

In [ ]:
author = 'John Doe'
author_affiliation = 'Research Institution'
description = 'A demonstrative smFRET-nsALEX measurement.'
sample_name = '50-50 mixture of two FRET samples'
dye_names = 'ATTO550, ATTO647N'
buffer_name = 'TE50'

5. Conversion


Once you finished editing the the previous sections you can proceed with the actual conversion. To do that, click on the menu *Cells* -> *Run All Below*.

After the execution go to **Section 6** to download the Photon-HDF5 file.

The cells below contain the code to convert the input file to Photon-HDF5.

5.1 Add metadata

In [ ]:
d['description'] = description

d['sample'] = dict(
    sample_name=sample_name,
    dye_names=dye_names,
    buffer_name=buffer_name,
    num_dyes = len(dye_names.split(',')))

d['identity'] = dict(
    author=author,
    author_affiliation=author_affiliation)

5.2 Save to Photon-HDF5

This command saves the new file to disk. If the input data does not follows the Photon-HDF5 specification it returns an error (Invalid_PhotonHDF5) printing what violates the specs.

In [ ]:
phc.hdf5.save_photon_hdf5(d, overwrite=True)

You can check it's content by using an HDF5 viewer such as HDFView.

6. Load Photon-HDF5

We can load the newly created Photon-HDF5 file to check its content:

In [ ]:
from pprint import pprint
In [ ]:
filename = d['_data_file'].filename
In [ ]:
h5data = phc.hdf5.load_photon_hdf5(filename)
In [ ]:
phc.hdf5.dict_from_group(h5data.identity)
In [ ]:
phc.hdf5.dict_from_group(h5data.setup)
In [ ]:
pprint(phc.hdf5.dict_from_group(h5data.photon_data))
In [ ]:
h5data._v_file.close()