#!/usr/bin/env python
# coding: utf-8

# # Py-ART I/O in detail
# 
# One of Py-ART most useful features is the ability to read in radar data from a number of formats.  
# As of version 1.5.0, Py-ART has full support for reading the following formats:
# 
# * NetCDF files formatted to meet the [CfRadial](http://www.ral.ucar.edu/projects/titan/docs/radial_formats/cfradial.html) conventions.
# 
# * Sigmet/IRIS, for example raw files from the ARM [XSAPR](http://www.arm.gov/instruments/xsapr) radars.
# 
# * MDV, for example raw files from the ARM [CSAPR](http://www.arm.gov/instruments/csapr) radars.
# 
# * Universal Format (UF) files.
# 
# * NEXRAD level 2 and 3 files available from [NCDC](http://www.ncdc.noaa.gov/).
# 
# * NEXRAD level 2 files converted using [Unidata's NetCDF-Java library](http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/).
# 
# * Files from the [CSU-CHILL](http://www.chill.colostate.edu/w/CSU_CHILL) radar. 
# 
# * Any format supported by [NASA's TRMM RSL](http://trmm-fc.gsfc.nasa.gov/trmm_gv/software/rsl/) library.
# 
# 
# Additionally Py-ART has partial support for reading the following formats:
# 
# * HDF5 files following the WMO's ODIM_H5 standard.
# * HDF5 produced by GAMIC radars.
# * NetCDF files from:
#     * [ARM's KAZR instruments](http://www.arm.gov/instruments/kazr), a zenith pointing radar.
#     * EDGE software.
#     * NASA's D3R radar during GCPEx.
#     * NSSL's NOXP radar during IPHEX.
# 
# * Any format supported by or [RadX](http://www.ral.ucar.edu/projects/titan/docs/radial_formats/radx.html).
# 
# 
# 
# Py-ART is continually being improved with support to read in additional format.  If your favorite format is missing, talk to us, we might be able to help you add support for it.

# ---
# Import matplotlib and Py-ART

# In[1]:


get_ipython().run_line_magic('matplotlib', 'inline')
import matplotlib
matplotlib.rcParams['figure.figsize'] = [12.0, 9.0]
import pyart


# ## Basic reading of radar files in Py-ART
# 
# Lets read in some radar files.  First an MDV file from an ARM XSAPR radar in the Southern Great Plains

# In[2]:


radar = pyart.io.read_mdv('data/110635.mdv')


# Plotting the reflectivity data.

# In[3]:


display = pyart.graph.RadarDisplay(radar)
display.plot('reflectivity', vmin=-16, vmax=80, cmap='pyart_NWSRef')


# ---
# Similarly a Sigmet/IRIS file from the ARM XSAPR radar in the SGP can be read.

# In[4]:


radar = pyart.io.read_sigmet('data/XSW110520113537.RAW7HHL')


# In[5]:


display = pyart.graph.RadarDisplay(radar)
display.plot('reflectivity', vmin=-32, vmax=80, cmap='pyart_NWSRef')


# ---
# And the same for a NEXRAD Level 2 file from the radar in Upton, NY.  Notice that the file is compressed using GZip.  Py-ART will uncompress file transparently during the read.

# In[6]:


radar = pyart.io.read_nexrad_archive('data/KOKX20110828_072107_V03.gz')


# In[7]:


display = pyart.graph.RadarDisplay(radar)
display.plot('reflectivity', vmin=0, vmax=80, cmap='pyart_NWSRef')


# ---
# Py-ART has partial support for reading files from ARM's KAZR radar.  Since the support is partial use the function in the pyart.aux_io namespace.

# In[8]:


radar = pyart.aux_io.read_kazr('data/sgpkazrhiC1.a1.20110503.000001.cdf')


# In[9]:


display = pyart.graph.RadarDisplay(radar)
display.plot('reflectivity_xpol', vmin=-70, vmax=0)


# ---
# Remembering to use the correct `pyart.io.read_filetype` function for the file you are reading can be hard, so Py-ART provides the [`pyart.io.read`](http://arm-doe.github.io/pyart-docs-travis/user_reference/generated/pyart.io.read.html#pyart.io.read) function that will examine the file, guess the type and call the correct read function.  

# In[10]:


radar = pyart.io.read('data/110635.mdv')
# try the pyart.io.read function with othe other files used thus far..


# The `read` function will not read data from file which Py-ART only partially supports.  To read these files you **must** use the `pyart.aux_io.read_filetype` functions.
# 
# All of these read functions return a **Radar** instance, which will be discussed in detail in the next notebook.  
# 
# As we saw in the last notebook the `info` method of the **Radar** class can provide details on the data stored in the object.

# In[11]:


radar.info()


# This feature is very useful to quickly detemine what data is contained in a file.  For this reason the command line  program, **radar_info**, will also provide this information.

# In[12]:


# ! indicate that a command will be execute from a shell not Juypter
# The results are printed here. 
get_ipython().system('radar_info -s data/110635.mdv')


# ---
# ## Advanced reading with Py-ART
# 
# Optional arguments provided to the `read` function allow control over what and how data is read from the file.  Details on these parameters are outline the the documenation of the [pyart.io.read](http://arm-doe.github.io/pyart-docs-travis/user_reference/generated/pyart.io.read.html#pyart.io.read) function.
# 
# A text version of this documenation can be access in Jupyter by addding a '?' to the function

# In[13]:


get_ipython().run_line_magic('pinfo', 'pyart.io.read')


# ---
# ### Controlling how Py-ART determines field names.
# 
# Many of the parameters of this function control how the fields are named. By default Py-ART uses settings in a configuration file to map between the fields in the raw file and a standard set of names.  This can be controlled at many levels.
# 
# The _file_field_names_ parameter is useful to completely disable this mapping.

# In[14]:


radar = pyart.io.read('data/XSW110520113537.RAW7HHL', file_field_names=False)
for field_name in radar.fields.keys():
    print field_name


# In[15]:


radar = pyart.io.read('data/XSW110520113537.RAW7HHL', file_field_names=True)
for field_name in radar.fields.keys():
    print field_name


# ---
# The _field_names_ parameter can be used to provide a custom mapping from the fields in the file to custom field names. 

# In[16]:


custom_mapping = {
    'DBZ2': 'refl',
    'VEL2': 'vel',
    'WIDTH2': 'width',
    'SQI2': 'sqi',
    'PHIDP2': None,
}

radar = pyart.io.read('data/XSW110520113537.RAW7HHL', field_names=custom_mapping)
for field_name in radar.fields.keys():
    print field_name


# To override the default name mappings and other configuartion options for an entire session use the [pyart.load_config](http://arm-doe.github.io/pyart-docs-travis/user_reference/generated/pyart.load_config.html#pyart.load_config) function at the top of a Python script/session.  Setting the **PYART_CONFIG** environmental variable to a configuration file will load this configuration everytime Py-ART is imported.

# ---
# The _exclude_fields_ parameter will prevent the specified fields from being read.

# In[17]:


radar = pyart.io.read('data/XSW110520113537.RAW7HHL')
for field_name in radar.fields.keys():
    print field_name


# In[18]:


radar = pyart.io.read('data/XSW110520113537.RAW7HHL', 
                      exclude_fields=['normalized_coherent_power'])
for field_name in radar.fields.keys():
    print field_name


# In[19]:


custom_mapping = {
    'DBZ2': 'refl',
    'VEL2': 'vel',
    'WIDTH2': 'width',
    'SQI2': 'sqi',
    'PHIDP2': None,
}

radar = pyart.io.read('data/XSW110520113537.RAW7HHL', 
                      field_names=custom_mapping, exclude_fields=['sqi'])
for field_name in radar.fields.keys():
    print field_name


# ---
# Setting the _delay_field_loading_ argument to `True` will delay the loading of all field data from the file.  For certain file formats this can significantly reduce memory usage and the time required to read a file.  Not all formats support this parameter and will issue a warning.

# In[20]:


radar = pyart.io.read('data/sgpwsacrcwrhiC1.a1.20120820.204016.nc', delay_field_loading=True)


# In[21]:


radar = pyart.io.read('data/XSW110520113537.RAW7HHL', delay_field_loading=True)


# ## Writing out files

# Py-ART can write out as NetCDF files which follow the Cf/Radial convention using the [`pyart.io.write_cfradial`](http://arm-doe.github.io/pyart-docs-travis/user_reference/generated/pyart.io.write_cfradial.html#pyart.io.write_cfradial) function.

# In[22]:


radar = pyart.io.read('data/XSW110520113537.RAW7HHL')


# In[23]:


pyart.io.write_cfradial('converted_sigmet_file.nc', radar)


# In[24]:


get_ipython().system('ncdump -h converted_sigmet_file.nc')


# ---
# A command line tool to perform this conversion is also included with Py-ART.

# In[25]:


get_ipython().system('anytocfradial data/110635.mdv converted_mdv_file.nc')


# In[26]:


get_ipython().system('ncdump -h converted_mdv_file.nc')