This notebook explores the data model used for pointing radar data in the Python ARM Radar Toolkit (Py-ART). We will do this by loading a radar file from CF-Radial from ARM's X-Band system in the North Slope of Alaska Barrow site.
If you are not running this notebook in the short course VM, go fetch the data from HERE and place it in a data directory in the same directory as this notebook.
import pyart
from matplotlib import pyplot as plt
%matplotlib inline
filename = 'data/nsaxsaprrhiC1.a1.20131203.141936.nc'
Read the CF-Radial file into Py-ART's data model for pointing gated data
radar = pyart.io.read(filename)
Lets investigate what is at the top level with a dir() command
dir(radar)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_dic_info', 'add_field', 'add_field_like', 'altitude', 'altitude_agl', 'antenna_transition', 'azimuth', 'elevation', 'extract_sweeps', 'fields', 'fixed_angle', 'info', 'instrument_parameters', 'latitude', 'longitude', 'metadata', 'ngates', 'nrays', 'nsweeps', 'radar_calibration', 'range', 'scan_rate', 'scan_type', 'sweep_end_ray_index', 'sweep_mode', 'sweep_number', 'sweep_start_ray_index', 'target_scan_rate', 'time']
Anything in the data model which contains array-like data is a dictionary with metadata and the actual data contained in the 'data' key, for example the array which contains information about the elevation angle of the sensor.
radar.azimuth.keys()
[u'comment', u'long_name', u'standard_name', u'units', 'data', u'axis']
radar.azimuth['standard_name']
u'beam_azimuth_angle'
radar.azimuth['data']
array([ 96.88842773, 96.88842773, 96.88842773, ..., 186.43798828, 186.43798828, 186.43798828], dtype=float32)
f = plt.figure(figsize=[15,8])
plt.plot(radar.time['data'], radar.azimuth['data'] )
plt.xlabel(radar.time['standard_name'] + ' (' + radar.time['units'] + ')')
plt.ylabel(radar.azimuth['standard_name'] + ' (' + radar.azimuth['units'] + ')')
<matplotlib.text.Text at 0x7f39948e1b90>
So all the pointing data is contained in the base object, the azimuth and elevation of the antenna/sensor plus the range and time axes
print radar.range['data'].min(), radar.range['data'].max(), radar.range['units']
f = plt.figure(figsize=[15,8])
plt.plot(radar.time['data'], radar.elevation['data'] )
plt.xlabel(radar.time['standard_name'] + ' (' + radar.time['units'] + ')')
plt.ylabel(radar.elevation['standard_name'] + ' (' + radar.elevation['units'] + ')')
0.0 39950.0 meters
<matplotlib.text.Text at 0x7f399492ca10>
there is also a swag of metadata contained within, well.. the metadata dictionary
for mykey in radar.metadata.keys():
print mykey, ': ', radar.metadata[mykey]
comment : Data in this file has not be calibrated, corrected, or had any quality control performed, use with caution. instrument_type : radar site_id : nsa references : http://radar.arm.gov/ARM_Radars/XSAPR.html volume_number : 0 process_version : $ title : Atmospheric Radiation Measurement (ARM) program X-band Scanning ARM Precipitation Radar (XSAPR) raw moments command_line : xsapr_ingest -s nsa -f C1 source : Atmospheric Radiation Measurement (ARM) program X-band Scanning ARM Precipitation Radar (XSAPR) instrument_name : xsapr-nsar1 platform_is_mobile : false data_level : a1 datastream : nsaxsaprrhiC1.a1 facility_id : C1 institution : United States Department of Energy - Atmospheric Radiation Measurement (ARM) program n_gates_vary : false primary_axis : axis_z dod_version : xsaprrhi-a1-1.2 Conventions : CF/Radial instrument_parameters platform_id : xsaprrhi platform_type : fixed location_description : North Slope of Alaska (NSA), Barrow, Alaska input_source : /data/home/dev/ingest/xsapr/DATA/data/collection/nsa/nsaxsaprC1.00/BRW131203141935.RAWWEHR field_names : total_power, reflectivity_horizontal, mean_doppler_velocity, doppler_spectrum_width, differential_reflectivity, specific_differential_phase, cross_correlation_ratio, normalized_coherent_power, differential_phase original_container : sigmet history : created by user jhelmus on machine jade at 2014-01-13 16:18:54, using $
Now the final top level bit of information, the data model we use follows CF-Radial morphology and hence has a set of "helper" fields to format out the radar coverage pattern. That is, to seperate sweeps.
radar.sweep_end_ray_index['data']
f = plt.figure(figsize=[15,8])
for i in range(len(radar.sweep_end_ray_index['data'])):
start_index = radar.sweep_start_ray_index['data'][i]
end_index = radar.sweep_end_ray_index['data'][i]
plt.plot(radar.time['data'][start_index:end_index],
radar.elevation['data'][start_index:end_index],
label = 'Sweep number '+ str(radar.sweep_number['data'][i]))
plt.legend()
plt.xlabel(radar.time['standard_name'] + ' (' + radar.time['units'] + ')')
plt.ylabel(radar.elevation['standard_name'] + ' (' + radar.elevation['units'] + ')')
<matplotlib.text.Text at 0x7f39946c3150>
Now to the actual data, or what ARM would call Primary Measurements. This is all stored in the field field of the radar object and is a dictionary of dictionaries. Best shown by example:
print radar.fields.keys()
print ""
for mykey in radar.fields.keys():
print mykey,':', radar.fields[mykey]['standard_name'] + ' (' + radar.fields[mykey]['units'] + ')'
[u'reflectivity_horizontal', u'cross_correlation_ratio', u'normalized_coherent_power', u'total_power', u'mean_doppler_velocity', u'doppler_spectrum_width', u'differential_reflectivity', u'specific_differential_phase', u'differential_phase'] reflectivity_horizontal : equivalent_reflectivity_factor (dBZ) cross_correlation_ratio : cross_correlation_ratio_hv (unitless) normalized_coherent_power : normalized_coherent_power (unitless) total_power : equivalent_reflectivity_factor (dBZ) mean_doppler_velocity : radial_velocity_of_scatters_away_from_instrument (meters_per_second) doppler_spectrum_width : doppler_spectrum_width (meters_per_second) differential_reflectivity : log_differential_reflectivity_hv (dB) specific_differential_phase : specific_differential_phase_hv (degrees_per_km) differential_phase : differential_phase_hv (degrees)
As far as CF-Radial ingest and write is concerned the variable names correspond to the variable names, the non-array data to the variable attributes and the 'data' key to the array.. lets look at some data
f = plt.figure(figsize=[15,8])
my_pc = plt.pcolormesh(radar.range['data'], radar.time['data'],
radar.fields['reflectivity_horizontal']['data'])
plt.xlabel(radar.range['standard_name'] + ' (' + radar.range['units'] + ')')
plt.ylabel(radar.time['standard_name'] + ' (' + radar.time['units'] + ')')
cb = plt.colorbar(mappable = my_pc)
cb.set_label(radar.fields['reflectivity_horizontal']['standard_name'] +\
' (' + radar.fields['reflectivity_horizontal']['units'] + ')')
And of course we can use our sweep indicators to isolate a single sweep
f = plt.figure(figsize=[15,8])
start_index = radar.sweep_start_ray_index['data'][0]
end_index = radar.sweep_end_ray_index['data'][0]
my_pc = plt.pcolormesh(radar.range['data'], radar.time['data'][start_index:end_index],
radar.fields['reflectivity_horizontal']['data'][start_index:end_index, :])
plt.xlabel(radar.range['standard_name'] + ' (' + radar.range['units'] + ')')
plt.ylabel(radar.time['standard_name'] + ' (' + radar.time['units'] + ')')
cb = plt.colorbar(mappable = my_pc)
cb.set_label(radar.fields['reflectivity_horizontal']['standard_name'] +\
' (' + radar.fields['reflectivity_horizontal']['units'] + ')')
We can get a quick overview of what is contained in the radar object using the info method.
radar.info('compact') # see what happens with 'standard' or 'full'
altitude: <ndarray of type: float64 and shape: (1,)> altitude_agl: None antenna_transition: None azimuth: <ndarray of type: float32 and shape: (1196,)> elevation: <ndarray of type: float32 and shape: (1196,)> fields: reflectivity_horizontal: <ndarray of type: float32 and shape: (1196, 800)> cross_correlation_ratio: <ndarray of type: float32 and shape: (1196, 800)> normalized_coherent_power: <ndarray of type: float32 and shape: (1196, 800)> total_power: <ndarray of type: float32 and shape: (1196, 800)> mean_doppler_velocity: <ndarray of type: float32 and shape: (1196, 800)> doppler_spectrum_width: <ndarray of type: float32 and shape: (1196, 800)> differential_reflectivity: <ndarray of type: float32 and shape: (1196, 800)> specific_differential_phase: <ndarray of type: float32 and shape: (1196, 800)> differential_phase: <ndarray of type: float32 and shape: (1196, 800)> fixed_angle: <ndarray of type: float32 and shape: (2,)> instrument_parameters: prt: <ndarray of type: float32 and shape: (1196,)> unambiguous_range: <ndarray of type: float32 and shape: (1196,)> prt_mode: <ndarray of type: |S1 and shape: (2, 32)> nyquist_velocity: <ndarray of type: float32 and shape: (1196,)> latitude: <ndarray of type: float64 and shape: (1,)> longitude: <ndarray of type: float64 and shape: (1,)> nsweeps: 2 ngates: 800 nrays: 1196 radar_calibration: range: <ndarray of type: float32 and shape: (800,)> scan_rate: None scan_type: rhi sweep_end_ray_index: <ndarray of type: int32 and shape: (2,)> sweep_mode: <ndarray of type: |S1 and shape: (2, 32)> sweep_number: <ndarray of type: int32 and shape: (2,)> sweep_start_ray_index: <ndarray of type: int32 and shape: (2,)> target_scan_rate: None time: <ndarray of type: float64 and shape: (1196,)> metadata: comment: Data in this file has not be calibrated, corrected, or had any quality control performed, use with caution. instrument_type: radar site_id: nsa references: http://radar.arm.gov/ARM_Radars/XSAPR.html volume_number: 0 process_version: $ title: Atmospheric Radiation Measurement (ARM) program X-band Scanning ARM Precipitation Radar (XSAPR) raw moments command_line: xsapr_ingest -s nsa -f C1 source: Atmospheric Radiation Measurement (ARM) program X-band Scanning ARM Precipitation Radar (XSAPR) instrument_name: xsapr-nsar1 platform_is_mobile: false data_level: a1 datastream: nsaxsaprrhiC1.a1 facility_id: C1 institution: United States Department of Energy - Atmospheric Radiation Measurement (ARM) program n_gates_vary: false primary_axis: axis_z dod_version: xsaprrhi-a1-1.2 Conventions: CF/Radial instrument_parameters platform_id: xsaprrhi platform_type: fixed location_description: North Slope of Alaska (NSA), Barrow, Alaska input_source: /data/home/dev/ingest/xsapr/DATA/data/collection/nsa/nsaxsaprC1.00/BRW131203141935.RAWWEHR field_names: total_power, reflectivity_horizontal, mean_doppler_velocity, doppler_spectrum_width, differential_reflectivity, specific_differential_phase, cross_correlation_ratio, normalized_coherent_power, differential_phase original_container: sigmet history: created by user jhelmus on machine jade at 2014-01-13 16:18:54, using $
This functionality is also available from the command line using the radar_info command.
!radar_info --compact data/nsaxsaprrhiC1.a1.20131203.141936.nc
altitude: <ndarray of type: float64 and shape: (1,)> altitude_agl: None antenna_transition: None azimuth: <ndarray of type: float32 and shape: (1196,)> elevation: <ndarray of type: float32 and shape: (1196,)> fields: reflectivity_horizontal: <ndarray of type: float32 and shape: (1196, 800)> cross_correlation_ratio: <ndarray of type: float32 and shape: (1196, 800)> normalized_coherent_power: <ndarray of type: float32 and shape: (1196, 800)> total_power: <ndarray of type: float32 and shape: (1196, 800)> mean_doppler_velocity: <ndarray of type: float32 and shape: (1196, 800)> doppler_spectrum_width: <ndarray of type: float32 and shape: (1196, 800)> differential_reflectivity: <ndarray of type: float32 and shape: (1196, 800)> specific_differential_phase: <ndarray of type: float32 and shape: (1196, 800)> differential_phase: <ndarray of type: float32 and shape: (1196, 800)> fixed_angle: <ndarray of type: float32 and shape: (2,)> instrument_parameters: prt: <ndarray of type: float32 and shape: (1196,)> unambiguous_range: <ndarray of type: float32 and shape: (1196,)> prt_mode: <ndarray of type: |S1 and shape: (2, 32)> nyquist_velocity: <ndarray of type: float32 and shape: (1196,)> latitude: <ndarray of type: float64 and shape: (1,)> longitude: <ndarray of type: float64 and shape: (1,)> nsweeps: 2 ngates: 800 nrays: 1196 radar_calibration: range: <ndarray of type: float32 and shape: (800,)> scan_rate: None scan_type: rhi sweep_end_ray_index: <ndarray of type: int32 and shape: (2,)> sweep_mode: <ndarray of type: |S1 and shape: (2, 32)> sweep_number: <ndarray of type: int32 and shape: (2,)> sweep_start_ray_index: <ndarray of type: int32 and shape: (2,)> target_scan_rate: None time: <ndarray of type: float64 and shape: (1196,)> metadata: comment: Data in this file has not be calibrated, corrected, or had any quality control performed, use with caution. instrument_type: radar site_id: nsa references: http://radar.arm.gov/ARM_Radars/XSAPR.html volume_number: 0 process_version: $ title: Atmospheric Radiation Measurement (ARM) program X-band Scanning ARM Precipitation Radar (XSAPR) raw moments command_line: xsapr_ingest -s nsa -f C1 source: Atmospheric Radiation Measurement (ARM) program X-band Scanning ARM Precipitation Radar (XSAPR) instrument_name: xsapr-nsar1 platform_is_mobile: false data_level: a1 datastream: nsaxsaprrhiC1.a1 facility_id: C1 institution: United States Department of Energy - Atmospheric Radiation Measurement (ARM) program n_gates_vary: false primary_axis: axis_z dod_version: xsaprrhi-a1-1.2 Conventions: CF/Radial instrument_parameters platform_id: xsaprrhi platform_type: fixed location_description: North Slope of Alaska (NSA), Barrow, Alaska input_source: /data/home/dev/ingest/xsapr/DATA/data/collection/nsa/nsaxsaprC1.00/BRW131203141935.RAWWEHR field_names: total_power, reflectivity_horizontal, mean_doppler_velocity, doppler_spectrum_width, differential_reflectivity, specific_differential_phase, cross_correlation_ratio, normalized_coherent_power, differential_phase original_container: sigmet history: created by user jhelmus on machine jade at 2014-01-13 16:18:54, using $
Thus concludes the intro! Py-ART, of course, can do all this for you including pretty PPIs etc.. but this gives an introduction to the data model we use. Questions? Comments? Science Lead: Scott Collis Development lead: Jonathan Helmus.