Use CSW to find ROMS data at COMT pycsw

In [1]:
from owslib.csw import CatalogueServiceWeb
from owslib import fes
import numpy as np

Find model results at COMT pycsw

In [2]:
endpoint = 'http://geoport.whoi.edu/csw'   # NODC/UAF Geoportal: granule level
#endpoint = 'http://geoport.whoi.edu:8000'
csw = CatalogueServiceWeb(endpoint,timeout=60)
print csw.version
2.0.2
In [3]:
csw
Out[3]:
<owslib.csw.CatalogueServiceWeb at 0x7fb155dad9d0>
In [4]:
import owslib
In [5]:
csw.get_operation_by_name('GetRecords').constraints
Out[5]:
[Constraint: SupportedISOQueryables - ['apiso:DistanceValue', 'apiso:Abstract', 'apiso:RevisionDate', 'apiso:Subject', 'apiso:KeywordType', 'apiso:Title', 'apiso:CRS', 'apiso:PublicationDate', 'apiso:Type', 'apiso:AlternateTitle', 'apiso:BoundingBox', 'apiso:AnyText', 'apiso:ParentIdentifier', 'apiso:Modified', 'apiso:Operation', 'apiso:Format', 'apiso:TempExtent_end', 'apiso:DistanceUOM', 'apiso:OrganisationName', 'apiso:ServiceType', 'apiso:TempExtent_begin', 'apiso:ResourceLanguage', 'apiso:ServiceTypeVersion', 'apiso:OperatesOn', 'apiso:Denominator', 'apiso:HasSecurityConstraints', 'apiso:OperatesOnIdentifier', 'apiso:GeographicDescriptionCode', 'apiso:Language', 'apiso:Identifier', 'apiso:OperatesOnName', 'apiso:TopicCategory', 'apiso:CreationDate', 'apiso:CouplingType'],
 Constraint: AdditionalQueryables - ['apiso:Lineage', 'apiso:Classification', 'apiso:Creator', 'apiso:Relation', 'apiso:OtherConstraints', 'apiso:SpecificationTitle', 'apiso:ResponsiblePartyRole', 'apiso:SpecificationDateType', 'apiso:Degree', 'apiso:Contributor', 'apiso:ConditionApplyingToAccessAndUse', 'apiso:SpecificationDate', 'apiso:AccessConstraints', 'apiso:Publisher'],
 Constraint: SupportedDublinCoreQueryables - ['dc:contributor', 'dc:source', 'dc:language', 'dc:title', 'dc:subject', 'dc:creator', 'dc:type', 'ows:BoundingBox', 'dct:modified', 'dct:abstract', 'dc:relation', 'dc:date', 'dc:identifier', 'dc:publisher', 'dc:format', 'csw:AnyText', 'dc:rights']]
In [6]:
val = 'CMG_Portal'
filter1 = fes.PropertyIsLike(propertyname='apiso:AnyText',literal=('*%s*' % val),
                        escapeChar='\\',wildCard='*',singleChar='?')
filter_list = [ filter1 ]
In [7]:
csw.getrecords2(constraints=filter_list,maxrecords=100,esn='full')
print len(csw.records.keys())
for rec in list(csw.records.keys()):
    print csw.records[rec].title 
    
7
COAWST Hindcast:Barnegat Bay:ADCIRC tides,Real rivers,Plume,Lowpass Espresso bdry,NAM,new bathy
COAWST Forecast System : USGS : US East Coast and Gulf of Mexico (Experimental)
ASGS NCFS Sandy Advisory 27
USGS-CMG-COAWST Model: Hurricane Sandy, NYB05 Nest
USGS-CMG-COAWST Model: Hurricane Sandy, SHF05 Nest
USGS-CMG-COAWST Model: Hurricane Sandy, USE27 Nest
COAWST Hindcast:MVCO/CBlast 2007:ripples with SWAN-40m res
In [8]:
choice=np.random.choice(list(csw.records.keys()))
print(csw.records[choice].title)
csw.records[choice].references
USGS-CMG-COAWST Model: Hurricane Sandy, SHF05 Nest
Out[8]:
[{'scheme': 'WWW:LINK',
  'url': 'http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_SHF05.ncml.html'},
 {'scheme': 'WWW:LINK',
  'url': 'http://www.ncdc.noaa.gov/oa/wct/wct-jnlp-beta.php?singlefile=http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_SHF05.ncml'},
 {'scheme': 'OPeNDAP:OPeNDAP',
  'url': 'http://geoport-dev.whoi.edu/thredds/dodsC/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_SHF05.ncml'},
 {'scheme': 'OGC:WMS',
  'url': 'http://geoport-dev.whoi.edu/thredds/wms/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_SHF05.ncml?service=WMS&version=1.3.0&request=GetCapabilities'},
 {'scheme': 'UNIDATA:NCSS',
  'url': 'http://geoport-dev.whoi.edu/thredds/ncss/grid/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_SHF05.ncml/dataset.html'},
 {'scheme': 'file',
  'url': 'http://geoport-dev.whoi.edu/thredds/fileServer/clay/usgs/users/jcwarner/Projects/Sandy/triple_nest/00_dir_SHF05.ncml'}]

Add bounding box constraint. To specify lon,lat order for bbox (which we want to do so that we can use the same bbox with either geoportal server or pycsw requests), we need to request the bounding box specifying the CRS84 coordinate reference system. The CRS84 option is available in pycsw 1.1.10+. The ability to specify the crs in the bounding box request is available in owslib 0.8.12+. For more info on the bounding box problem and how it was solved, see this pycsw issue, this geoportal server issue, and this owslib issue

In [9]:
bbox = [-87.40, 34.25, -63.70, 66.70]
bbox_filter = fes.BBox(bbox,crs='urn:ogc:def:crs:OGC:1.3:CRS84')
filter_list = [fes.And([filter1, bbox_filter])]
csw.getrecords2(constraints=filter_list, maxrecords=1000)
In [10]:
print(len(csw.records.keys()))
for rec in list(csw.records.keys()):
    print('title:'+csw.records[rec].title) 
    print('identifier:'+csw.records[rec].identifier)
    print('modified:'+csw.records[rec].modified)
    print(' ')
7
title:COAWST Hindcast:Barnegat Bay:ADCIRC tides,Real rivers,Plume,Lowpass Espresso bdry,NAM,new bathy
identifier:gov.usgs.cmgp:COAWST.Barnegat_Bay.spring2012
modified:2015-07-06
 
title:COAWST Forecast System : USGS : US East Coast and Gulf of Mexico (Experimental)
identifier:gov.usgs.cmgp:COAWST.USEAST.Forecast
modified:2015-07-06
 
title:ASGS NCFS Sandy Advisory 27
identifier:RENCI_ADCIRC.ASGS.NCFS.Sandy.27
modified:2015-05-05
 
title:USGS-CMG-COAWST Model: Hurricane Sandy, NYB05 Nest
identifier:gov.usgs.cmg:USGS_COAWST_Sandy_NYB05_sim6
modified:2015-07-06
 
title:USGS-CMG-COAWST Model: Hurricane Sandy, SHF05 Nest
identifier:gov.usgs.cmg:USGS_COAWST_Sandy_SHF05_sim6
modified:2015-07-06
 
title:USGS-CMG-COAWST Model: Hurricane Sandy, USE27 Nest
identifier:gov.usgs.cmg:USGS_COAWST_Sandy_USE27_sim6
modified:2015-07-06
 
title:COAWST Hindcast:MVCO/CBlast 2007:ripples with SWAN-40m res
identifier:gov.usgs.cmgp:COAWST.MVCO.CBLAST.spatial_7_ar0fd
modified:2015-07-09
 
In [11]:
foo=csw.records[rec]
In [12]:
print foo.xml
<csw:SummaryRecord xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:ows100="http://www.opengis.net/ows"><dc:identifier>gov.usgs.cmgp:COAWST.MVCO.CBLAST.spatial_7_ar0fd</dc:identifier><dc:title>COAWST Hindcast:MVCO/CBlast 2007:ripples with SWAN-40m res</dc:title><dc:type>dataset</dc:type><dc:subject>Oceans; Ocean Temperature; Potential Temperature</dc:subject><dc:subject>Oceans; Salinity/Density; Salinity</dc:subject><dc:subject>Oceans; Sea Surface Topography; Sea Surface Height</dc:subject><dc:subject>Oceans; Ocean Circulation; Ocean Currents</dc:subject><dc:subject>CMG_Portal</dc:subject><dc:subject>USGS Coastal and Marine Geology Program</dc:subject><dc:subject>sea_floor_depth</dc:subject><dc:subject>water_surface_height_above_reference_datum</dc:subject><dc:subject>barotropic_x_sea_water_velocity</dc:subject><dc:subject>barotropic_y_sea_water_velocity</dc:subject><dc:subject>x_sea_water_velocity</dc:subject><dc:subject>y_sea_water_velocity</dc:subject><dc:subject>sea_water_potential_temperature</dc:subject><dc:subject>sea_water_salinity</dc:subject><dc:subject>sea_surface_wave_significant_height</dc:subject><dc:subject>ocean_s_coordinate</dc:subject><dc:subject>ocean_s_coordinate</dc:subject><dc:subject>time</dc:subject><dct:references scheme="WWW:LINK">http://geoport-dev.whoi.edu/thredds/dodsC/usgs/data0/mvco_ce/mvco_output/spatial_7_ar0fd/00_dir_roms.ncml.html</dct:references><dct:references scheme="WWW:LINK">http://www.ncdc.noaa.gov/oa/wct/wct-jnlp-beta.php?singlefile=http://geoport-dev.whoi.edu/thredds/dodsC/usgs/data0/mvco_ce/mvco_output/spatial_7_ar0fd/00_dir_roms.ncml</dct:references><dct:references scheme="OPeNDAP:OPeNDAP">http://geoport-dev.whoi.edu/thredds/dodsC/usgs/data0/mvco_ce/mvco_output/spatial_7_ar0fd/00_dir_roms.ncml</dct:references><dct:references scheme="OGC:WMS">http://geoport-dev.whoi.edu/thredds/wms/usgs/data0/mvco_ce/mvco_output/spatial_7_ar0fd/00_dir_roms.ncml?service=WMS&amp;version=1.3.0&amp;request=GetCapabilities</dct:references><dct:references scheme="UNIDATA:NCSS">http://geoport-dev.whoi.edu/thredds/ncss/usgs/data0/mvco_ce/mvco_output/spatial_7_ar0fd/00_dir_roms.ncml/dataset.html</dct:references><dct:references scheme="file">http://geoport-dev.whoi.edu/thredds/fileServer/usgs/data0/mvco_ce/mvco_output/spatial_7_ar0fd/00_dir_roms.ncml</dct:references><dc:relation /><dct:modified>2015-07-09</dct:modified><dct:abstract>COAWST simulation of CBLAST 2007 region south of Marthas Vineyard, including the Martha's Vineyard Coastal Observatory Region. This simulation is CASE07_AR0FD.  Ripple formulation with SWAN-40m resolution wave forcing.</dct:abstract><ows100:BoundingBox crs="urn:x-ogc:def:crs:EPSG:6.11:4326" dimensions="2"><ows100:LowerCorner>41.29 -70.61</ows100:LowerCorner><ows100:UpperCorner>41.35 -70.5</ows100:UpperCorner></ows100:BoundingBox></csw:SummaryRecord>
In [12]: