Using xarray to read EBAS data

Using xarray to read EBAS data

See more at http://ebas.nilu.no/ and https://ebas.nilu.no/thredds/

The EBAS database collects observational data on atmospheric chemical composition and physical properties from a variety of national and international research projects and monitoring programs, such as ACTRIS, AMAP, EMEP, GAW and HELCOM, as well as for the Norwegian monitoring programs funded by the Norwegian Environment Agency, the Ministry of Climate and Environment and NILU – Norwegian Institute for Air Research.

See all files available: https://thredds.nilu.no/thredds/catalog/ebas/catalog.html

# You need to first install threddsclient:
# pip install threddsclient
import threddsclient
import xarray as xr
# Find url addresses for files on EBAS

all_opendap_urls = threddsclient.opendap_urls(
'https://thredds.nilu.no/thredds/catalog/ebas/catalog.xml')

# get all data urls for one station, e.g., Zeppelin NO0042G
opendap_urls = [x for x in all_opendap_urls if 'NO0042G' in x]
# get all scattering data urls
opendap_urls = [x for x in opendap_urls if 'nephelometer' in x]

opendap_urls
['https://thredds.nilu.no/thredds/dodsC/ebas/NO0042G.20100101000000.20150216111241.nephelometer..pm10.4y.1h.SE02L_TSI_3563_ZEP_dry.SE02L_scat_coef.lev2.nc',
 'https://thredds.nilu.no/thredds/dodsC/ebas/NO0042G.20080708135939.20181213000000.nephelometer..aerosol_humidified.3mo.6h.CH02L_TSI_3563_ZEP_ref+TSI_3563_ZEP_wet.CH02L_hygro_tandem_neph_CorrData.lev2.nc']
# read multiple files
# the problem with this example is, that the files are from different instruments

dsmf = xr.open_mfdataset(opendap_urls)
dsmf
<xarray.Dataset>
Dimensions:                                                         (Location: 1, RH: 13, RH_base: 1, RH_base_max: 1, RH_humidified: 1, Wavelength: 3, aerosol_light_backscattering_coefficient_amean_qc_flags: 2, aerosol_light_backscattering_coefficient_perc8413_qc_flags: 2, aerosol_light_backscattering_coefficient_prec1587_qc_flags: 2, aerosol_light_backscattering_coefficient_qc_flags: 1, aerosol_light_backscattering_enhancement_factor_qc_flags: 1, aerosol_light_scattering_coefficient_amean_qc_flags: 2, aerosol_light_scattering_coefficient_perc8413_qc_flags: 2, aerosol_light_scattering_coefficient_prec1587_qc_flags: 2, aerosol_light_scattering_coefficient_qc_flags: 1, aerosol_light_scattering_enhancement_factor_qc_flags: 1, metadata_time: 5, pressure_qc_flags: 2, relative_humidity_qc_flags: 2, tbnds: 2, temperature_qc_flags: 2, time: 35451)
Coordinates:
  * metadata_time                                                   (metadata_time) datetime64[ns] ...
  * time                                                            (time) datetime64[ns] ...
  * Wavelength                                                      (Wavelength) float64 ...
  * RH_base                                                         (RH_base) float64 ...
  * RH_humidified                                                   (RH_humidified) float64 ...
  * RH_base_max                                                     (RH_base_max) float64 ...
  * RH                                                              (RH) float64 ...
  * Location                                                        (Location) |S64 ...
Dimensions without coordinates: aerosol_light_backscattering_coefficient_amean_qc_flags, aerosol_light_backscattering_coefficient_perc8413_qc_flags, aerosol_light_backscattering_coefficient_prec1587_qc_flags, aerosol_light_backscattering_coefficient_qc_flags, aerosol_light_backscattering_enhancement_factor_qc_flags, aerosol_light_scattering_coefficient_amean_qc_flags, aerosol_light_scattering_coefficient_perc8413_qc_flags, aerosol_light_scattering_coefficient_prec1587_qc_flags, aerosol_light_scattering_coefficient_qc_flags, aerosol_light_scattering_enhancement_factor_qc_flags, pressure_qc_flags, relative_humidity_qc_flags, tbnds, temperature_qc_flags
Data variables: (12/41)
    time_bnds                                                       (time, tbnds) datetime64[ns] dask.array<chunksize=(35451, 2), meta=np.ndarray>
    metadata_time_bnds                                              (metadata_time, tbnds) datetime64[ns] dask.array<chunksize=(5, 2), meta=np.ndarray>
    aerosol_light_scattering_enhancement_factor_qc                  (Wavelength, RH_base, RH_humidified, aerosol_light_scattering_enhancement_factor_qc_flags, time) float64 dask.array<chunksize=(3, 1, 1, 1, 35451), meta=np.ndarray>
    aerosol_light_backscattering_enhancement_factor_ebasmetadata    (Wavelength, RH_base_max, RH_humidified, metadata_time) object dask.array<chunksize=(3, 1, 1, 5), meta=np.ndarray>
    aerosol_light_scattering_enhancement_factor_ebasmetadata        (Wavelength, RH_base, RH_humidified, metadata_time) object dask.array<chunksize=(3, 1, 1, 5), meta=np.ndarray>
    aerosol_light_backscattering_coefficient_qc                     (Wavelength, RH, aerosol_light_backscattering_coefficient_qc_flags, time) float64 dask.array<chunksize=(3, 13, 1, 35451), meta=np.ndarray>
    ...                                                              ...
    aerosol_light_scattering_coefficient_prec1587                   (Wavelength, time) float64 dask.array<chunksize=(3, 35451), meta=np.ndarray>
    pressure                                                        (Location, time) float64 dask.array<chunksize=(1, 35451), meta=np.ndarray>
    aerosol_light_backscattering_coefficient_amean                  (Wavelength, time) float64 dask.array<chunksize=(3, 35451), meta=np.ndarray>
    aerosol_light_scattering_coefficient_perc8413                   (Wavelength, time) float64 dask.array<chunksize=(3, 35451), meta=np.ndarray>
    relative_humidity                                               (Location, time) float64 dask.array<chunksize=(1, 35451), meta=np.ndarray>
    aerosol_light_backscattering_coefficient_prec1587               (Wavelength, time) float64 dask.array<chunksize=(3, 35451), meta=np.ndarray>
Attributes: (12/46)
    Conventions:                   CF-1.7, ACDD-1.3
    featureType:                   timeSeries
    title:                         Ground based in situ observations of nephe...
    keywords:                      NO0042G, Zeppelin mountain (Ny-Ålesund), p...
    id:                            NO0042G.20100101000000.20150216111241.neph...
    naming_authority:              EBAS
    ...                            ...
    geospatial_lat_units:          degrees_north
    geospatial_lon_units:          degrees_east
    comment:                       {\n    "Data definition": "EBAS_1.1", \n  ...
    standard_name_vocabulary:      CF-1.7, ACDD-1.3
    history:                       None
    creator_url:                   ebas.nilu.no
# read one file

ds = xr.open_dataset(
'https://thredds.nilu.no/thredds/dodsC/ebas/'
'CA0420G.20040101000000.20201112154612.nephelometer.'
'.pm10.16y.1h.CA01L_TSI_3563_ALT_pm10.CA01L_scat_coef.lev2.nc')
ds
<xarray.Dataset>
Dimensions:                                                         (Location: 1, Wavelength: 3, aerosol_light_backscattering_coefficient_amean_qc_flags: 1, aerosol_light_backscattering_coefficient_perc8413_qc_flags: 1, aerosol_light_backscattering_coefficient_prec1587_qc_flags: 1, aerosol_light_scattering_coefficient_amean_qc_flags: 1, aerosol_light_scattering_coefficient_perc8413_qc_flags: 1, aerosol_light_scattering_coefficient_prec1587_qc_flags: 1, metadata_time: 16, pressure_qc_flags: 1, relative_humidity_qc_flags: 1, tbnds: 2, temperature_qc_flags: 1, time: 140256)
Coordinates:
  * time                                                            (time) datetime64[ns] ...
  * metadata_time                                                   (metadata_time) datetime64[ns] ...
  * Wavelength                                                      (Wavelength) float64 ...
  * Location                                                        (Location) |S64 ...
Dimensions without coordinates: aerosol_light_backscattering_coefficient_amean_qc_flags, aerosol_light_backscattering_coefficient_perc8413_qc_flags, aerosol_light_backscattering_coefficient_prec1587_qc_flags, aerosol_light_scattering_coefficient_amean_qc_flags, aerosol_light_scattering_coefficient_perc8413_qc_flags, aerosol_light_scattering_coefficient_prec1587_qc_flags, pressure_qc_flags, relative_humidity_qc_flags, tbnds, temperature_qc_flags
Data variables: (12/29)
    time_bnds                                                       (time, tbnds) datetime64[ns] ...
    metadata_time_bnds                                              (metadata_time, tbnds) datetime64[ns] ...
    aerosol_light_scattering_coefficient_amean_ebasmetadata         (Wavelength, metadata_time) |S64 ...
    temperature_qc                                                  (Location, temperature_qc_flags, time) float64 ...
    relative_humidity_qc                                            (Location, relative_humidity_qc_flags, time) float64 ...
    aerosol_light_scattering_coefficient_prec1587_ebasmetadata      (Wavelength, metadata_time) |S64 ...
    ...                                                              ...
    aerosol_light_scattering_coefficient_prec1587                   (Wavelength, time) float64 ...
    pressure                                                        (Location, time) float64 ...
    aerosol_light_backscattering_coefficient_amean                  (Wavelength, time) float64 ...
    aerosol_light_scattering_coefficient_perc8413                   (Wavelength, time) float64 ...
    relative_humidity                                               (Location, time) float64 ...
    aerosol_light_backscattering_coefficient_prec1587               (Wavelength, time) float64 ...
Attributes: (12/52)
    Conventions:                   CF-1.8, ACDD-1.3
    featureType:                   timeSeries
    title:                         Ground based in situ observations of nephe...
    keywords:                      NOAA-ESRL, pm10, aerosol_light_backscatter...
    id:                            CA0420G.20040101000000.20201112154612.neph...
    naming_authority:              EBAS
    ...                            ...
    geospatial_lat_units:          degrees_north
    geospatial_lon_units:          degrees_east
    comment:                       {\n    "Data definition": "EBAS_1.1", \n  ...
    standard_name_vocabulary:      CF-1.7, ACDD-1.3
    history:                       None
    creator_url:                   ebas.nilu.no