import pandas as pd
import xarray as xr
from pprint import pprint

# Load Catalog
a=pd.read_csv('/data/keeling/a/cristi/a/esm_data/cmip6_catalog.csv')

# Search for files to be read

            # variable in ESGF
path=a.loc[(a['variable_id']=='hur')&
           # experiment
           (a['experiment']=='AMIP')&
           # modelname
           (a['source_id'].str.contains('GFDL'))&
           # realization
           (a['variant_label'].str.contains('r1i1p1'))]['path'].to_list()

if path == []:
    print('Your search did not match any files on Keeling. Please proceed to download.')
else:
    pprint(path)

['/data/cristi/a/cristi/esm_data/cmip6/GFDL-CM4/amip/r1i1p1f1/hur_Amon_GFDL-CM4_amip_r1i1p1f1_gr1_197901-198412.nc',
 '/data/cristi/a/cristi/esm_data/cmip6/GFDL-CM4/amip/r1i1p1f1/hur_Amon_GFDL-CM4_amip_r1i1p1f1_gr1_198501-199012.nc',
 '/data/cristi/a/cristi/esm_data/cmip6/GFDL-CM4/amip/r1i1p1f1/hur_Amon_GFDL-CM4_amip_r1i1p1f1_gr1_199101-199612.nc',
 '/data/cristi/a/cristi/esm_data/cmip6/GFDL-CM4/amip/r1i1p1f1/hur_Amon_GFDL-CM4_amip_r1i1p1f1_gr1_199701-200212.nc',
 '/data/cristi/a/cristi/esm_data/cmip6/GFDL-CM4/amip/r1i1p1f1/hur_Amon_GFDL-CM4_amip_r1i1p1f1_gr1_200301-200812.nc',
 '/data/cristi/a/cristi/esm_data/cmip6/GFDL-CM4/amip/r1i1p1f1/hur_Amon_GFDL-CM4_amip_r1i1p1f1_gr1_200901-201412.nc']

# Read the files in Xarray
ds = xr.open_mfdataset(path,combine="by_coords",use_cftime=True)
ds

<xarray.Dataset>
Dimensions:    (bnds: 2, time: 432, plev: 19, lat: 180, lon: 288)
Coordinates:
  * bnds       (bnds) float64 1.0 2.0
  * lat        (lat) float64 -89.5 -88.5 -87.5 -86.5 ... 86.5 87.5 88.5 89.5
  * lon        (lon) float64 0.625 1.875 3.125 4.375 ... 355.6 356.9 358.1 359.4
  * plev       (plev) float64 1e+05 9.25e+04 8.5e+04 7e+04 ... 1e+03 500.0 100.0
  * time       (time) object 1979-01-16 12:00:00 ... 2014-12-16 12:00:00
Data variables:
    hur        (time, plev, lat, lon) float32 dask.array<chunksize=(72, 19, 180, 288), meta=np.ndarray>
    lat_bnds   (time, lat, bnds) float64 dask.array<chunksize=(72, 180, 2), meta=np.ndarray>
    lon_bnds   (time, lon, bnds) float64 dask.array<chunksize=(72, 288, 2), meta=np.ndarray>
    time_bnds  (time, bnds) object dask.array<chunksize=(72, 2), meta=np.ndarray>
Attributes: (12/44)
    external_variables:    areacella
    history:               File was processed by fremetar (GFDL analog of CMO...
    table_id:              Amon
    activity_id:           CMIP
    branch_method:         no parent
    branch_time_in_child:  0.0
    ...                    ...
    title:                 NOAA GFDL GFDL-CM4 model output prepared for CMIP6...
    tracking_id:           hdl:21.14100/7c27e5b3-c207-42a3-a01a-40d264a01935
    variable_id:           hur
    variant_info:          N/A
    references:            see further_info_url attribute
    variant_label:         r1i1p1f1

xarray.Dataset

• Dimensions:
  • bnds: 2
  • time: 432
  • plev: 19
  • lat: 180
  • lon: 288
• Coordinates: (5)
  • bnds
    (bnds)
    float64
    1.0 2.0

    long_name :

    vertex number

    array([1., 2.])

  • lat
    (lat)
    float64
    -89.5 -88.5 -87.5 ... 88.5 89.5

    long_name :

    latitude

    units :

    degrees_north

    axis :

    Y

    bounds :

    lat_bnds

    standard_name :

    latitude

    cell_methods :

    time: point

    array([-89.5, -88.5, -87.5, -86.5, -85.5, -84.5, -83.5, -82.5, -81.5, -80.5,
           -79.5, -78.5, -77.5, -76.5, -75.5, -74.5, -73.5, -72.5, -71.5, -70.5,
           -69.5, -68.5, -67.5, -66.5, -65.5, -64.5, -63.5, -62.5, -61.5, -60.5,
           -59.5, -58.5, -57.5, -56.5, -55.5, -54.5, -53.5, -52.5, -51.5, -50.5,
           -49.5, -48.5, -47.5, -46.5, -45.5, -44.5, -43.5, -42.5, -41.5, -40.5,
           -39.5, -38.5, -37.5, -36.5, -35.5, -34.5, -33.5, -32.5, -31.5, -30.5,
           -29.5, -28.5, -27.5, -26.5, -25.5, -24.5, -23.5, -22.5, -21.5, -20.5,
           -19.5, -18.5, -17.5, -16.5, -15.5, -14.5, -13.5, -12.5, -11.5, -10.5,
            -9.5,  -8.5,  -7.5,  -6.5,  -5.5,  -4.5,  -3.5,  -2.5,  -1.5,  -0.5,
             0.5,   1.5,   2.5,   3.5,   4.5,   5.5,   6.5,   7.5,   8.5,   9.5,
            10.5,  11.5,  12.5,  13.5,  14.5,  15.5,  16.5,  17.5,  18.5,  19.5,
            20.5,  21.5,  22.5,  23.5,  24.5,  25.5,  26.5,  27.5,  28.5,  29.5,
            30.5,  31.5,  32.5,  33.5,  34.5,  35.5,  36.5,  37.5,  38.5,  39.5,
            40.5,  41.5,  42.5,  43.5,  44.5,  45.5,  46.5,  47.5,  48.5,  49.5,
            50.5,  51.5,  52.5,  53.5,  54.5,  55.5,  56.5,  57.5,  58.5,  59.5,
            60.5,  61.5,  62.5,  63.5,  64.5,  65.5,  66.5,  67.5,  68.5,  69.5,
            70.5,  71.5,  72.5,  73.5,  74.5,  75.5,  76.5,  77.5,  78.5,  79.5,
            80.5,  81.5,  82.5,  83.5,  84.5,  85.5,  86.5,  87.5,  88.5,  89.5])

  • lon
    (lon)
    float64
    0.625 1.875 3.125 ... 358.1 359.4

    long_name :

    longitude

    units :

    degrees_east

    axis :

    X

    bounds :

    lon_bnds

    standard_name :

    longitude

    cell_methods :

    time: point

    array([  0.625,   1.875,   3.125, ..., 356.875, 358.125, 359.375])

  • plev
    (plev)
    float64
    1e+05 9.25e+04 ... 500.0 100.0

    long_name :

    pressure

    units :

    Pa

    axis :

    Z

    positive :

    down

    standard_name :

    air_pressure

    description :

    There are 19 requested levels. If the upper levels are above the model top they should be omitted.

    array([100000.,  92500.,  85000.,  70000.,  60000.,  50000.,  40000.,  30000.,
            25000.,  20000.,  15000.,  10000.,   7000.,   5000.,   3000.,   2000.,
             1000.,    500.,    100.])

  • time
    (time)
    object
    1979-01-16 12:00:00 ... 2014-12-...

    long_name :

    time

    axis :

    T

    calendar_type :

    noleap

    bounds :

    time_bnds

    standard_name :

    time

    description :

    Temporal mean

    array([cftime.DatetimeNoLeap(1979, 1, 16, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1979, 2, 15, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1979, 3, 16, 12, 0, 0, 0, has_year_zero=True),
           ...,
           cftime.DatetimeNoLeap(2014, 10, 16, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 11, 16, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 12, 16, 12, 0, 0, 0, has_year_zero=True)],
          dtype=object)

• Data variables: (4)
  • hur
    (time, plev, lat, lon)
    float32
    dask.array<chunksize=(72, 19, 180, 288), meta=np.ndarray>

    long_name :

    Relative Humidity

    units :

    %

    cell_methods :

    time: mean

    cell_measures :

    area: areacella

    standard_name :

    relative_humidity

    interp_method :

    conserve_order2

    original_name :

    hur

    Array Chunk Bytes 1.59 GiB 270.53 MiB Shape (432, 19, 180, 288) (72, 19, 180, 288) Count 18 Tasks 6 Chunks Type float32 numpy.ndarray

  • lat_bnds
    (time, lat, bnds)
    float64
    dask.array<chunksize=(72, 180, 2), meta=np.ndarray>

    long_name :

    latitude bounds

    units :

    degrees_north

    axis :

    Y

    Array Chunk Bytes 1.19 MiB 202.50 kiB Shape (432, 180, 2) (72, 180, 2) Count 24 Tasks 6 Chunks Type float64 numpy.ndarray

  • lon_bnds
    (time, lon, bnds)
    float64
    dask.array<chunksize=(72, 288, 2), meta=np.ndarray>

    long_name :

    longitude bounds

    units :

    degrees_east

    axis :

    X

    Array Chunk Bytes 1.90 MiB 324.00 kiB Shape (432, 288, 2) (72, 288, 2) Count 24 Tasks 6 Chunks Type float64 numpy.ndarray

  • time_bnds
    (time, bnds)
    object
    dask.array<chunksize=(72, 2), meta=np.ndarray>

    long_name :

    time axis boundaries

    Array Chunk Bytes 6.75 kiB 1.12 kiB Shape (432, 2) (72, 2) Count 18 Tasks 6 Chunks Type object numpy.ndarray

• Indexes: (5)
  • bnds
    PandasIndex

    PandasIndex(Float64Index([1.0, 2.0], dtype='float64', name='bnds'))

  • lat
    PandasIndex

    PandasIndex(Float64Index([-89.5, -88.5, -87.5, -86.5, -85.5, -84.5, -83.5, -82.5, -81.5,
                  -80.5,
                  ...
                   80.5,  81.5,  82.5,  83.5,  84.5,  85.5,  86.5,  87.5,  88.5,
                   89.5],
                 dtype='float64', name='lat', length=180))

  • lon
    PandasIndex

    PandasIndex(Float64Index([             0.625, 1.8749999999999998,              3.125,
                               4.375,              5.625,              6.875,
                               8.125,              9.375,             10.625,
                              11.875,
                  ...
                             348.125,            349.375,            350.625,
                             351.875,            353.125,            354.375,
                             355.625,            356.875,            358.125,
                             359.375],
                 dtype='float64', name='lon', length=288))

  • plev
    PandasIndex

    PandasIndex(Float64Index([100000.0,  92500.0,  85000.0,  70000.0,  60000.0,  50000.0,
                   40000.0,  30000.0,  25000.0,  20000.0,  15000.0,  10000.0,
                    7000.0,   5000.0,   3000.0,   2000.0,   1000.0,    500.0,
                     100.0],
                 dtype='float64', name='plev'))

  • time
    PandasIndex

    PandasIndex(CFTimeIndex([1979-01-16 12:00:00, 1979-02-15 00:00:00, 1979-03-16 12:00:00,
                 1979-04-16 00:00:00, 1979-05-16 12:00:00, 1979-06-16 00:00:00,
                 1979-07-16 12:00:00, 1979-08-16 12:00:00, 1979-09-16 00:00:00,
                 1979-10-16 12:00:00,
                 ...
                 2014-03-16 12:00:00, 2014-04-16 00:00:00, 2014-05-16 12:00:00,
                 2014-06-16 00:00:00, 2014-07-16 12:00:00, 2014-08-16 12:00:00,
                 2014-09-16 00:00:00, 2014-10-16 12:00:00, 2014-11-16 00:00:00,
                 2014-12-16 12:00:00],
                dtype='object', length=432, calendar='noleap', freq='None'))

• Attributes: (44)

  external_variables :

  areacella

  history :

  File was processed by fremetar (GFDL analog of CMOR). TripleID: [exper_id_RpBPUUl3mi,realiz_id_cR5lGHtt5s,run_id_G9rrXIDztr]

  table_id :

  Amon

  activity_id :

  CMIP

  branch_method :

  no parent

  branch_time_in_child :

  0.0

  comment :

  <null ref>

  contact :

  gfdl.climate.model.info@noaa.gov

  Conventions :

  CF-1.7 CMIP-6.0 UGRID-1.0

  creation_date :

  2019-03-26T17:50:29Z

  data_specs_version :

  01.00.27

  experiment :

  AMIP

  experiment_id :

  amip

  forcing_index :

  1

  frequency :

  monC

  further_info_url :

  https://furtherinfo.es-doc.org/CMIP6.NOAA-GFDL.GFDL-CM4.amip.none.r1i1p1f1

  grid :

  atmos data regridded from Cubed-sphere (c96) to 180,288; interpolation method: conserve_order2

  grid_label :

  gr1

  initialization_index :

  1

  institution :

  National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory, Princeton, NJ 08540, USA

  institution_id :

  NOAA-GFDL

  license :

  CMIP6 model data produced by NOAA-GFDL is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses/). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  nominal_resolution :

  100 km

  parent_activity_id :

  no parent

  parent_experiment_id :

  no parent

  parent_mip_era :

  no parent

  parent_source_id :

  no parent

  parent_variant_label :

  no parent

  physics_index :

  1

  product :

  model-output

  realization_index :

  1

  realm :

  atmos

  source :

  GFDL-CM4 (2018): aerosol: interactive atmos: GFDL-AM4.0.1 (Cubed-sphere (c96) - 1 degree nominal horizontal resolution; 360 x 180 longitude/latitude; 33 levels; top level 1 hPa) atmosChem: fast chemistry, aerosol only land: GFDL-LM4.0.1 (1 degree nominal horizontal resolution; 360 x 180 longitude/latitude; 20 levels; bot level 10m); land:Veg:unnamed (dynamic vegetation, dynamic land use); land:Hydro:unnamed (soil water and ice, multi-layer snow, rivers and lakes) landIce: GFDL-LM4.0.1 ocean: GFDL-OM4p25 (GFDL-MOM6, tripolar - nominal 0.25 deg; 1440 x 1080 longitude/latitude; 75 levels; top grid cell 0-2 m) ocnBgchem: GFDL-BLINGv2 seaIce: GFDL-SIM4p25 (GFDL-SIS2.0, tripolar - nominal 0.25 deg; 1440 x 1080 longitude/latitude; 5 layers; 5 thickness categories) (GFDL ID: 2019_0229)

  source_id :

  GFDL-CM4

  source_type :

  AGCM

  sub_experiment :

  none

  sub_experiment_id :

  none

  title :

  NOAA GFDL GFDL-CM4 model output prepared for CMIP6 AMIP

  tracking_id :

  hdl:21.14100/7c27e5b3-c207-42a3-a01a-40d264a01935

  variable_id :

  hur

  variant_info :

  N/A

  references :

  see further_info_url attribute

  variant_label :

  r1i1p1f1