In this last notebook we saw that Py-ART reads data from files into Radar instances, which is Py-ART's data model for radar volumes. In this notebook we will examine this class in detail, looking at what is contained within and how they can be used to build custom processing methods.
The layout of data in the Radar class is derived from the CfRadial specification. For those familar with this specification thinking of the Radar class as an in memory version of a CfRadial file is a good mental model.
First, the standand imports
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['figure.figsize'] = [12.0, 9.0]
import pyart
We will use the volume from an ARM XSAPR radar that was examined in the last notebook.
radar = pyart.io.read('data/XSW110520113537.RAW7HHL')
This created a Radar instance which is the class Py-ART uses to store and process radar volume data. Python's built in help system can be used to provide information about this object.
type(radar)
pyart.core.radar.Radar
help(radar)
Help on Radar in module pyart.core.radar object: class Radar(__builtin__.object) | A class for storing antenna coordinate radar data. | | The structure of the Radar class is based on the CF/Radial Data file | format. Global attributes and variables (section 4.1 and 4.3) are | represented as a dictionary in the metadata attribute. Other required and | optional variables are represented as dictionaries in a attribute with the | same name as the variable in the CF/Radial standard. When a optional | attribute not present the attribute has a value of None. The data for a | given variable is stored in the dictionary under the 'data' key. Moment | field data is stored as a dictionary of dictionaries in the fields | attribute. Sub-convention variables are stored as a dictionary of | dictionaries under the meta_group attribute. | | Refer to the attribute section for information on the parameters. | | Attributes | ---------- | time : dict | Time at the center of each ray. | range : dict | Range to the center of each gate (bin). | fields : dict of dicts | Moment fields. | metadata : dict | Metadata describing the instrument and data. | scan_type : str | Type of scan, one of 'ppi', 'rhi', 'sector' or 'other'. If the scan | volume contains multiple sweep modes this should be 'other'. | latitude : dict | Latitude of the instrument. | longitude : dict | Longitude of the instrument. | altitude : dict | Altitude of the instrument, above sea level. | altitude_agl : dict or None | Altitude of the instrument above ground level. If not provided this | attribute is set to None, indicating this parameter not available. | sweep_number : dict | The number of the sweep in the volume scan, 0-based. | sweep_mode : dict | Sweep mode for each mode in the volume scan. | fixed_angle : dict | Target angle for thr sweep. Azimuth angle in RHI modes, elevation | angle in all other modes. | sweep_start_ray_index : dict | Index of the first ray in each sweep relative to the start of the | volume, 0-based. | sweep_end_ray_index : dict | Index of the last ray in each sweep relative to the start of the | volume, 0-based. | rays_per_sweep : dict | Number of rays in each sweep. This is a read only attribute, | attempting to set the attribute will raise a AttributeError and any | changes to the dictionary keys will be lost when the attribute is | accessed again. | target_scan_rate : dict or None | Intended scan rate for each sweep. If not provided this attribute is | set to None, indicating this parameter is not available. | rays_are_indexed : dict or None | Indication of whether ray angles are indexed to a regular grid in | each sweep. If not provided this attribute is set to None, indicating | ray angle spacing is not determined. | ray_angle_res : dict or None | If rays_are_indexed is not None, this provides the angular resolution | of the grid. If not provided or available this attribute is set to | None. | azimuth : dict | Azimuth of antenna, relative to true North. | elevation : dict | Elevation of antenna, relative to the horizontal plane. | scan_rate : dict or None | Actual antenna scan rate. If not provided this attribute is set to | None, indicating this parameter is not available. | antenna_transition : dict or None | Flag indicating if the antenna is in transition, 1 = yes, 0 = no. | If not provided this attribute is set to None, indicating this | parameter is not available. | rotation : dict or None | The rotation angle of the antenna. The angle about the aircraft | longitudinal axis for a vertically scanning radar. | tilt : dict or None | The tilt angle with respect to the plane orthogonal (Z-axis) to | aircraft longitudinal axis. | roll : dict or None | The roll angle of platform, for aircraft right wing down is positive. | drift : dict or None | Drift angle of antenna, the angle between heading and track. | heading : dict or None | Heading (compass) angle, clockwise from north. | pitch : dict or None | Pitch angle of antenna, for aircraft nose up is positive. | georefs_applied : dict or None | Indicates whether the variables have had georeference calculation | applied. Leading to Earth-centric azimuth and elevation angles. | instrument_parameters : dict of dicts or None | Instrument parameters, if not provided this attribute is set to None, | indicating these parameters are not avaiable. This dictionary also | includes variables in the radar_parameters CF/Radial subconvention. | radar_calibration : dict of dicts or None | Instrument calibration parameters. If not provided this attribute is | set to None, indicating these parameters are not available | ngates : int | Number of gates (bins) in the volume. | nrays : int | Number of rays in the volume. | nsweeps : int | Number of sweep in the volume. | | Methods defined here: | | __init__(self, time, _range, fields, metadata, scan_type, latitude, longitude, altitude, sweep_number, sweep_mode, fixed_angle, sweep_start_ray_index, sweep_end_ray_index, azimuth, elevation, altitude_agl=None, target_scan_rate=None, rays_are_indexed=None, ray_angle_res=None, scan_rate=None, antenna_transition=None, instrument_parameters=None, radar_calibration=None, rotation=None, tilt=None, roll=None, drift=None, heading=None, pitch=None, georefs_applied=None) | | add_field(self, field_name, dic, replace_existing=False) | Add a field to the object. | | Parameters | ---------- | field_name : str | Name of the field to add to the dictionary of fields. | dic : dict | Dictionary contain field data and metadata. | replace_existing : bool | True to replace the existing field with key field_name if it | exists, loosing any existing data. False will raise a ValueError | when the field already exists. | | add_field_like(self, existing_field_name, field_name, data, replace_existing=False) | Add a field to the object with metadata from a existing field. | | Parameters | ---------- | existing_field_name : str | Name of an existing field to take metadata from when adding | the new field to the object. | field_name : str | Name of the field to add to the dictionary of fields. | data : array | Field data. | replace_existing : bool | True to replace the existing field with key field_name if it | exists, loosing any existing data. False will raise a ValueError | when the field already exists. | | check_field_exists(self, field_name) | Check that a field exists in the fields dictionary. | | If the field does not exist raise a KeyError. | | Parameters | ---------- | field_name : str | Name of field to check. | | extract_sweeps(self, sweeps) | Create a new radar contains only the data from select sweeps. | | Parameters | ---------- | sweeps : array_like | Sweeps (0-based) to include in new Radar object. | | Returns | ------- | radar : Radar | Radar object which contains a copy of data from the selected | sweeps. | | get_azimuth(self, sweep, copy=False) | Return an array of azimuth angles for a given sweep. | | Parameters | ---------- | sweep : int | Sweep number to retrieve data for, 0 based. | copy : bool, optional | True to return a copy of the azimuths. False, the default, returns | a view of the azimuths (when possible), changing this data will | change the data in the underlying Radar object. | | Returns | ------- | azimuths : array | Array containing the azimuth angles for a given sweep. | | get_elevation(self, sweep, copy=False) | Return an array of elevation angles for a given sweep. | | Parameters | ---------- | sweep : int | Sweep number to retrieve data for, 0 based. | copy : bool, optional | True to return a copy of the elevations. False, the default, | returns a view of the elevations (when possible), changing this | data will change the data in the underlying Radar object. | | Returns | ------- | azimuths : array | Array containing the elevation angles for a given sweep. | | get_end(self, sweep) | Return the ending ray for a given sweep. | | get_field(self, sweep, field_name, copy=False) | Return the field data for a given sweep. | | Parameters | ---------- | sweep : int | Sweep number to retrieve data for, 0 based. | field_name : str | Name of the field from which data should be retrieved. | copy : bool, optional | True to return a copy of the data. False, the default, returns | a view of the data (when possible), changing this data will | change the data in the underlying Radar object. | | Returns | ------- | data : array | Array containing data for the requested sweep and field. | | get_nyquist_vel(self, sweep, check_uniform=True) | Return the Nyquist velocity in meters per second for a given sweep. | | Raises a LookupError if the Nyquist velocity is not available, an | Exception is raised if the velocities are not uniform in the sweep | unless check_uniform is set to False. | | Parameters | ---------- | sweep : int | Sweep number to retrieve data for, 0 based. | check_uniform : bool | True to check to perform a check on the Nyquist velocities that | they are uniform in the sweep, False will skip this check and | return the velocity of the first ray in the sweep. | | Returns | ------- | nyquist_velocity : float | Array containing the Nyquist velocity in m/s for a given sweep. | | get_slice(self, sweep) | Return a slice for selecting rays for a given sweep. | | get_start(self, sweep) | Return the starting ray index for a given sweep. | | get_start_end(self, sweep) | Return the starting and ending ray for a given sweep. | | info(self, level='standard', out=<IPython.kernel.zmq.iostream.OutStream object>) | Print information on radar. | | Parameters | ---------- | level : {'compact', 'standard', 'full', 'c', 's', 'f'} | Level of information on radar object to print, compact is | minimal information, standard more and full everything. | out : file-like | Stream to direct output to, default is to print information | to standard out (the screen). | | iter_azimuth(self) | Return an iterator which returns sweep azimuth data. | | iter_elevation(self) | Return an iterator which returns sweep elevation data. | | iter_end(self) | Return an iterator over the sweep end indices. | | iter_field(self, field_name) | Return an iterator which returns sweep field data. | | iter_slice(self) | Return an iterator which returns sweep slice objects. | | iter_start(self) | Return an iterator over the sweep start indices. | | iter_start_end(self) | Return an iterator over the sweep start and end indices. | | ---------------------------------------------------------------------- | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined) | | rays_per_sweep
We can also use Jupyter's help system.
radar?
This tells us about the Radar class in general. Use the info
method to describe the data in the XSAPR file we have just read.
radar.info('compact')
altitude: <ndarray of type: float64 and shape: (1,)> altitude_agl: None antenna_transition: None azimuth: <ndarray of type: float32 and shape: (4278,)> elevation: <ndarray of type: float32 and shape: (4278,)> fields: radar_echo_classification: <ndarray of type: float32 and shape: (4278, 534)> corrected_reflectivity: <ndarray of type: float32 and shape: (4278, 534)> differential_phase: <ndarray of type: float32 and shape: (4278, 534)> cross_correlation_ratio: <ndarray of type: float32 and shape: (4278, 534)> normalized_coherent_power: <ndarray of type: float32 and shape: (4278, 534)> spectrum_width: <ndarray of type: float32 and shape: (4278, 534)> total_power: <ndarray of type: float32 and shape: (4278, 534)> reflectivity: <ndarray of type: float32 and shape: (4278, 534)> differential_reflectivity: <ndarray of type: float32 and shape: (4278, 534)> specific_differential_phase: <ndarray of type: float32 and shape: (4278, 534)> velocity: <ndarray of type: float32 and shape: (4278, 534)> corrected_differential_reflectivity: <ndarray of type: float32 and shape: (4278, 534)> fixed_angle: <ndarray of type: float32 and shape: (6,)> instrument_parameters: prt_mode: <ndarray of type: |S5 and shape: (6,)> unambiguous_range: <ndarray of type: float32 and shape: (4278,)> pulse_width: <ndarray of type: float32 and shape: (4278,)> prt: <ndarray of type: float32 and shape: (4278,)> nyquist_velocity: <ndarray of type: float32 and shape: (4278,)> radar_beam_width_h: <ndarray of type: float32 and shape: (1,)> radar_beam_width_v: <ndarray of type: float32 and shape: (1,)> latitude: <ndarray of type: float64 and shape: (1,)> longitude: <ndarray of type: float64 and shape: (1,)> nsweeps: 6 ngates: 534 nrays: 4278 radar_calibration: None range: <ndarray of type: float32 and shape: (534,)> scan_rate: None scan_type: rhi sweep_end_ray_index: <ndarray of type: int32 and shape: (6,)> sweep_mode: <ndarray of type: |S3 and shape: (6,)> sweep_number: <ndarray of type: int32 and shape: (6,)> sweep_start_ray_index: <ndarray of type: int32 and shape: (6,)> target_scan_rate: None time: <ndarray of type: float64 and shape: (4278,)> metadata: comment: sigmet_task_name: MC3E_CLD_RHI title: sigmet_extended_header: false Conventions: CF/Radial instrument_parameters source: version: 1.3 references: time_ordered: none instrument_name: xsapr-sgpr2 rays_missing: 0 original_container: sigmet institution: history:
Notice that from the above infomation we are looking at a volume consisting of Range Height Indicator or RHI scans. We can verify this by examining the scan_type
attribute of the class.
radar.scan_type
'rhi'
The information on the elevation angles is stored in the elevation attribute as a Python dictionary. The numerical values are stored as a NumPy array in the 'data' key.
type(radar.elevation)
dict
radar.elevation.keys()
['comment', 'long_name', 'standard_name', 'units', 'data', 'axis']
print radar.elevation['long_name']
print radar.elevation['standard_name']
print radar.elevation['units']
elevation_angle_from_horizontal_plane beam_elevation_angle degrees
radar.elevation['data']
array([ 1.03271484, 1.28814697, 1.54907227, ..., 178.409729 , 178.6706543 , 178.93707275], dtype=float32)
We can plot the elevation angles for all the sweeps in the volume.
plt.plot(radar.elevation['data'])
[<matplotlib.lines.Line2D at 0x115f999d0>]
By counting the local extrema in the graph, this volume appears to contain 6 sweeps. We can check to see that this agrees with the number of sweeps reported by the file:
radar.nsweeps
6
The number of rays and gates in each ray is also available.
print radar.nrays
print radar.ngates
4278 534
Other information about the radar volume is stored in dictionary attributes, with the numerical data stored under the 'data' key.
radar.azimuth
{'axis': 'radial_azimuth_coordinate', 'comment': 'Azimuth of antenna relative to true north', 'data': array([ 37.24639893, 37.24639893, 37.24639893, ..., 187.25372314, 187.25372314, 187.25372314], dtype=float32), 'long_name': 'azimuth_angle_from_true_north', 'standard_name': 'beam_azimuth_angle', 'units': 'degrees'}
radar.latitude # also try longitude and latitude
{'data': array([ 36.49103003]), 'long_name': 'Latitude', 'standard_name': 'Latitude', 'units': 'degrees_north'}
radar.time
{'calendar': 'gregorian', 'comment': 'Coordinate variable for time. Time at the center of each ray, in fractional seconds since the global variable time_coverage_start', 'data': array([ 29., 29., 29., ..., 187., 187., 187.]), 'long_name': 'time_in_seconds_since_volume_start', 'standard_name': 'time', 'units': 'seconds since 2011-05-20T11:35:37Z'}
radar.time # try radar.range for the locations of the gates (bins)
{'calendar': 'gregorian', 'comment': 'Coordinate variable for time. Time at the center of each ray, in fractional seconds since the global variable time_coverage_start', 'data': array([ 29., 29., 29., ..., 187., 187., 187.]), 'long_name': 'time_in_seconds_since_volume_start', 'standard_name': 'time', 'units': 'seconds since 2011-05-20T11:35:37Z'}
Some data may not be available in the original file or not applicable to the volume. When this data is missing the attribute is set to None
radar.target_scan_rate
radar.target_scan_rate is None
True
# XSAPR is stationary so geoference data does not apply
print radar.rotation is None
print radar.tilt is None
print radar.roll is None
print radar.drift is None
True True True True
The radar fields or moments are stored in the fields attribute as a dictionary of dictionaries.
radar.fields.keys()
['radar_echo_classification', 'corrected_reflectivity', 'differential_phase', 'cross_correlation_ratio', 'normalized_coherent_power', 'spectrum_width', 'total_power', 'reflectivity', 'differential_reflectivity', 'specific_differential_phase', 'velocity', 'corrected_differential_reflectivity']
type(radar.fields['reflectivity'])
dict
radar.fields['reflectivity'].keys()
['_FillValue', 'coordinates', 'long_name', 'standard_name', 'units', 'data']
print radar.fields['reflectivity']['standard_name']
print radar.fields['reflectivity']['units']
print radar.fields['reflectivity']['coordinates']
equivalent_reflectivity_factor dBZ elevation azimuth range
Field data is stored as a 2D array with dimensions of rays and gates. Masked points indicate that the gate was either not collected or below the detection threshold.
print type(radar.fields['reflectivity']['data'])
print radar.fields['reflectivity']['data'].shape
print radar.fields['reflectivity']['data'].dtype
<class 'numpy.ma.core.MaskedArray'> (4278, 534) float32
Instrument parameters are also stored as a dictionary of dictionaries
radar.instrument_parameters.keys()
['prt_mode', 'unambiguous_range', 'pulse_width', 'prt', 'nyquist_velocity', 'radar_beam_width_h', 'radar_beam_width_v']
radar.instrument_parameters['nyquist_velocity']
{'comments': 'Unambiguous velocity', 'data': array([ 12.94250011, 12.94250011, 12.94250011, ..., 12.94250011, 12.94250011, 12.94250011], dtype=float32), 'long_name': 'Nyquist velocity', 'meta_group': 'instrument_parameters', 'units': 'meters_per_second'}
Often we want to work with a single sweep and therefore need to extact out the data from a particular sweep. The Radar class stores sweep specific information about the volume.
radar.fixed_angle
{'data': array([ 37.29858398, 67.30224609, 97.30041504, 127.29858398, 157.30224609, 187.30041504], dtype=float32), 'long_name': 'Target angle for sweep', 'standard_name': 'target_fixed_angle', 'units': 'degrees'}
radar.sweep_number
{'data': array([0, 1, 2, 3, 4, 5], dtype=int32), 'long_name': 'Sweep number', 'standard_name': 'sweep_number', 'units': 'count'}
radar.sweep_mode
{'comment': 'Options are: "sector", "coplane", "rhi", "vertical_pointing", "idle", "azimuth_surveillance", "elevation_surveillance", "sunscan", "pointing", "manual_ppi", "manual_rhi"', 'data': array(['rhi', 'rhi', 'rhi', 'rhi', 'rhi', 'rhi'], dtype='|S3'), 'long_name': 'Sweep mode', 'standard_name': 'sweep_mode', 'units': 'unitless'}
The indices which delinate a particular sweep are stored in the sweep_start_ray_index and sweep_end_ray_index attributes of the radar object.
radar.sweep_start_ray_index
{'data': array([ 0, 713, 1426, 2139, 2852, 3565], dtype=int32), 'long_name': 'Index of first ray in sweep, 0-based', 'units': 'count'}
radar.sweep_end_ray_index
{'data': array([ 712, 1425, 2138, 2851, 3564, 4277], dtype=int32), 'long_name': 'Index of last ray in sweep, 0-based', 'units': 'count'}
The rays_per_sweep attribute provides a count of the number of rays in each sweep.
radar.rays_per_sweep
{'data': array([713, 713, 713, 713, 713, 713], dtype=int32), 'long_name': 'Number of rays in each sweep', 'units': 'count'}
The first sweep starts at ray 0 and the last ray in the sweep is ray 712. We can plot the elevation angles from just this sweep.
plt.plot(radar.elevation['data'][0:713])
[<matplotlib.lines.Line2D at 0x1155aebd0>]
These limits can also be detemined for a specific sweep using the get_start
, get_end
, or get_start_end
methods.
print radar.get_start(0)
print radar.get_end(0)
print radar.get_start_end(0)
0 712 (0, 712)
Using these values to access sweep data can be cumbersome and care must be taken to avoid off-by-one errors. For convenience the Radar object provides a get_slice
method which returns a slice object that can be used to select out the rays belonging to a particular sweep.
radar.get_slice?
Using this feature to plot the elevation angles from the first two sweeps in the volume.
sweep_0_slice = radar.get_slice(0)
plt.plot(radar.elevation['data'][sweep_0_slice])
[<matplotlib.lines.Line2D at 0x119b61410>]
sweep_1_slice = radar.get_slice(1)
plt.plot(radar.elevation['data'][sweep_1_slice])
[<matplotlib.lines.Line2D at 0x11a0dd750>]
The radar class offers another method which further simplified this procedure, get_elevation
.
help(radar.get_elevation)
Help on method get_elevation in module pyart.core.radar: get_elevation(self, sweep, copy=False) method of pyart.core.radar.Radar instance Return an array of elevation angles for a given sweep. Parameters ---------- sweep : int Sweep number to retrieve data for, 0 based. copy : bool, optional True to return a copy of the elevations. False, the default, returns a view of the elevations (when possible), changing this data will change the data in the underlying Radar object. Returns ------- azimuths : array Array containing the elevation angles for a given sweep.
plt.plot(radar.get_elevation(0))
[<matplotlib.lines.Line2D at 0x10222cad0>]
The azimuth angles can extracted in the same manner using the get_azimuth
method.
Slice objects can also select out field data form a particular sweep.
Plotting data from the entire volume in a b-scan representation.
plt.imshow(radar.fields['reflectivity']['data'], aspect=0.5, origin='bottom')
plt.xlabel('range gate')
plt.ylabel('ray number')
<matplotlib.text.Text at 0x1155d2950>
And just the data from the first sweep.
sweep_0_slice = radar.get_slice(0)
plt.imshow(radar.fields['reflectivity']['data'][sweep_0_slice], aspect=0.5, origin='bottom')
plt.xlabel('range gate')
plt.ylabel('ray number')
<matplotlib.text.Text at 0x119ef5290>
The get_field
method can simplify this even further.
help(radar.get_field)
Help on method get_field in module pyart.core.radar: get_field(self, sweep, field_name, copy=False) method of pyart.core.radar.Radar instance Return the field data for a given sweep. Parameters ---------- sweep : int Sweep number to retrieve data for, 0 based. field_name : str Name of the field from which data should be retrieved. copy : bool, optional True to return a copy of the data. False, the default, returns a view of the data (when possible), changing this data will change the data in the underlying Radar object. Returns ------- data : array Array containing data for the requested sweep and field.
refl_sweep_data = radar.get_field(sweep=0, field_name='reflectivity')
plt.imshow(refl_sweep_data, aspect=0.5, origin='bottom')
plt.xlabel('range gate')
plt.ylabel('ray number')
<matplotlib.text.Text at 0x108171450>
A final method for accessing a specific sweep is to use the extract_sweeps
method to create a new Radar instance which contains only a single sweep.
print radar.nrays
print radar.nsweeps
4278 6
radar2 = radar.extract_sweeps([0])
print radar2.nrays
print radar2.nsweeps
713 1
plt.imshow(radar2.fields['reflectivity']['data'], aspect=0.5, origin='bottom')
plt.xlabel('range gate')
plt.ylabel('ray number')
<matplotlib.text.Text at 0x1190e42d0>
Another common task is to loop over all sweeps in the volume. The Radar class contains a number of helpful methods to accomplish this task.
for start in radar.iter_start():
print start
0 713 1426 2139 2852 3565
for end in radar.iter_end():
print end
712 1425 2138 2851 3564 4277
for start, end in radar.iter_start_end():
print start, end
0 712 713 1425 1426 2138 2139 2851 2852 3564 3565 4277
fig = plt.figure()
ax = fig.add_subplot(111)
for i, sweep_slice in enumerate(radar.iter_slice()):
ax.plot(radar.elevation['data'][sweep_slice] + i * 20)
fig = plt.figure()
ax = fig.add_subplot(111)
for i, elev_data in enumerate(radar.iter_elevation()):
ax.plot(elev_data + i * 20)
fig = plt.figure()
for i, refl_sweep_data in enumerate(radar.iter_field('reflectivity')):
ax = fig.add_subplot(1, radar.nsweeps, i+1)
ax.imshow(refl_sweep_data)