xarray_to_cdf

cdflib.xarray.xarray_to_cdf(xarray_dataset, file_name, unixtime_to_cdftt2000=False, datetime_to_cdftt2000=False, datetime64_to_cdftt2000=False, istp=True, terminate_on_warning=False, auto_fix_depends=True, record_dimensions=[], compression=0, from_unixtime=False, from_datetime=False)[source][source]

This function converts XArray Dataset objects into CDF files.

Parameters:

xarray_dataset (xarray.Dataset) – The XArray Dataset object that you’d like to convert into a CDF file
file_name (str) – The path to the place the newly created CDF file
datetime_to_cdftt2000 (bool, optional) – Whether or not to convert variables named “epoch” or “epoch_X” to CDF_TT2000 from datetime objects
datetime64_to_cdftt2000 (bool, optional) – Whether or not to convert variables named “epoch” or “epoch_X” to CDF_TT2000 from the numpy datetime64
unixtime_to_cdftt2000 (bool, optional) – Whether or not to convert variables named “epoch” or “epoch_X” to CDF_TT2000 from unixtime
istp (bool, optional) – Whether or not to do checks on the Dataset object to attempt to enforce CDF compliance
terminate_on_warning (bool, optional) – Whether or not to throw an error when given warnings or to continue trying to make the file
auto_fix_depends (bool, optional) – Whether or not to automatically add dependencies
record_dimensions (list of str, optional) – If the code cannot determine which dimensions should be made into CDF records, you may provide a list of them here
compression (int, optional) – The level of compression to gzip the data in the variables. Default is no compression, standard is 6.
from_datetime (bool, optional, deprecated) – Same as the datetime_to_cdftt2000 option
from_unixtime (bool, optional, deprecated) – Same as the unixtime_to_cdftt2000 option

Returns:

None, but generates a CDF file

Return type:

None

Example CDF file from scratch:

>>> # Import the needed libraries
>>> from cdflib.xarray import xarray_to_cdf
>>> import xarray as xr
>>> import os
>>> import urllib.request

>>> # Create some fake data
>>> var_data = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
>>> var_dims = ['epoch', 'direction']
>>> data = xr.Variable(var_dims, var_data)

>>> # Create fake epoch data
>>> epoch_data = [1, 2, 3]
>>> epoch_dims = ['epoch']
>>> epoch = xr.Variable(epoch_dims, epoch_data)

>>> # Combine the two into an xarray Dataset and export as CDF (this will print out many ISTP warnings)
>>> ds = xr.Dataset(data_vars={'data': data, 'epoch': epoch})
>>> xarray_to_cdf(ds, 'hello.cdf')

>>> # Add some global attributes
>>> global_attributes = {'Project': 'Hail Mary',
>>>                      'Source_name': 'Thin Air',
>>>                      'Discipline': 'None',
>>>                      'Data_type': 'counts',
>>>                      'Descriptor': 'Midichlorians in unicorn blood',
>>>                      'Data_version': '3.14',
>>>                      'Logical_file_id': 'SEVENTEEN',
>>>                      'PI_name': 'Darth Vader',
>>>                      'PI_affiliation': 'Dark Side',
>>>                      'TEXT': 'AHHHHH',
>>>                      'Instrument_type': 'Banjo',
>>>                      'Mission_group': 'Impossible',
>>>                      'Logical_source': ':)',
>>>                      'Logical_source_description': ':('}

>>> # Lets add a new coordinate variable for the "direction"
>>> dir_data = [1, 2, 3]
>>> dir_dims = ['direction']
>>> direction = xr.Variable(dir_dims, dir_data)

>>> # Recreate the Dataset with this new objects, and recreate the CDF
>>> ds = xr.Dataset(data_vars={'data': data, 'epoch': epoch, 'direction':direction}, attrs=global_attributes)
>>> os.remove('hello.cdf')
>>> xarray_to_cdf(ds, 'hello.cdf')

Example netCDF -> CDF conversion:

>>> # Download a netCDF file (if needed)
>>> fname = 'dn_magn-l2-hires_g17_d20211219_v1-0-1.nc'
>>> url = ("https://lasp.colorado.edu/maven/sdc/public/data/sdc/web/cdflib_testing/dn_magn-l2-hires_g17_d20211219_v1-0-1.nc")
>>> if not os.path.exists(fname):
>>>     urllib.request.urlretrieve(url, fname)

>>> # Load in the dataset, and set VAR_TYPES attributes (the most important attribute as far as this code is concerned)
>>> goes_r_mag = xr.load_dataset("dn_magn-l2-hires_g17_d20211219_v1-0-1.nc")
>>> for var in goes_r_mag:
>>>     goes_r_mag[var].attrs['VAR_TYPE'] = 'data'
>>> goes_r_mag['coordinate'].attrs['VAR_TYPE'] = 'support_data'
>>> goes_r_mag['time'].attrs['VAR_TYPE'] = 'support_data'
>>> goes_r_mag['time_orbit'].attrs['VAR_TYPE'] = 'support_data'

>>> # Create the CDF file
>>> xarray_to_cdf(goes_r_mag, 'hello.cdf')

Processing Steps:

Determines the list of dimensions that represent time-varying dimensions. These ultimately become the “records” of the CDF file
- If it is named “epoch” or “epoch_N”, it is considered time-varying
- If a variable points to another variable with a DEPEND_0 attribute, it is considered time-varying
- If a variable has an attribute of VAR_TYPE equal to “data”, it is time-varying
- If a variable has an attribute of VAR_TYPE equal to “support_data” and it is 2 dimensional, it is time-varying
Determine a list of “dimension” variables within the Dataset object
- These are all coordinates in the dataset that are not time-varying
- Additionally, variables that a DEPEND_N attribute points to are also considered dimensions
Optionally, if ISTP=true, automatically add in DEPEND_0/1/2/etc attributes as necessary
Optionally, if ISTP=true, check all variable attributes and global attributes are present
Convert all data into either CDF_INT8, CDF_DOUBLE, CDF_UINT4, or CDF_CHAR
Optionally, convert variables with the name “epoch” or “epoch_N” to CDF_TT2000
Write all variables and global attributes to the CDF file!

ISTP Warnings:

If ISTP=true, these are some of the common things it will check:

Missing or invalid VAR_TYPE variable attributes
DEPEND_N missing from variables
DEPEND_N/LABL_PTR/UNIT_PTR/FORM_PTR are pointing to missing variables
Missing required global attributes
Missing an “epoch” dimension
DEPEND_N attribute pointing to a variable with oncompatible dimensions