Glossary¶
ARM (Atmospheric Radiation Measurement)¶
A DOE program that operates and provides data from in situ and remote sensing observations stratigically located to improve the understanding and representation, in climate and earth system models, of clouds and aerosols as well as their interactions and coupling with the Earth’s surface.
Bin Average¶
Bin Average is a transform method that maps a coordinate variable with a smaller interval onto a new coordinate grid with a larger interval by averaging the smaller values.
Base_platform¶
The core name associated with a datastream excluding site and facility. For ARM data it represents the VAP or ingest that produced the datastream.
BOD¶
Birth of a Datastream.
CDI¶
Definition needed.
CDS (Common Data Structure)¶
Internal data structures used by the ARMADI library.
Dataset¶
A collection of ARM data contained in an ARM standard netCDF datafile(s).
Datastream¶
A dataset consisting of dimensions, variables, and attributes. A datastream is described in terms of a site, base_platform, and facility.
DOD¶
Data Object Design.
DSDB¶
a VAP’s general characteristics, inputs, outputs, and transformations are stored in the Data System Database (DSDB).
DSPL¶
The ADI processes utilize the Data System Process Library (libdsproc3) to retrieve data from the input files and perform the necessary manipulations to create the output data products defined in the PCM.
IDL¶
The Interactive Data Language (IDL) is a vectorized, interactive programming language used for processing large datasets. IDL is one of the languages (along with Python and C) that can be used for creating and customizing a VAP.
Interpolate¶
Interpolate is a transform method that maps a coordinate variable with a larger interval onto a new coordinate grid with a smaller interval by interpolating.
netCDF¶
Network Common Data Form.
PCM (Process Control Manager)¶
The graphic interface through which ARMADI processes, variable retrieval specifications, and output datastreams are defined.
QC¶
Quality Check.
Serial 1D Transformation¶
The method for transforming multidimensional arrays is designed, by default, to transform slices of the data one dimension at a time. This method is referred to in this document as Serial 1D transformation. For example, for a time series of temperature profiles by height, time is transformed first (i.e., all the lowest level temperatures over our chosen time window, then all the second lowest level temperatures, etc.), and then height is transformed (the first temperature profile, then the second profile, etc.). This allows the use of different transform methods and parameters for each dimension, For example,we might want to interpolate in height but subsample the time dimension.
Right now, we do not have a way to do multidimensional transformations directly; you have to do one dimension and then the next, even if you’d rather do them together. At this time all transformations are performed using the serial 1D method. Multidimensional transformation, meaning the transformation of two or more dimensions using a single call to the core transform function, is being explored and may be supported in the future.
Subsample¶
Subsample is a transform method that can be used to transform any coordinate dimension onto a new grid, assigning values based on a nearest neighbor approach.
Transformation¶
We use the word transformation to describe the process of mapping data from one coordinate grid onto another. This can involve interpolation, averaging, subsampling, or any other method one can devise for regridding the data. Transformation includes filling gaps and replacing bad data using the available QC. A number of methods are used to fill/fix data and include interpolating over them or not including them in the average, etc. Thus, a valid transform is to map a set of data onto its own grid to fill gaps and estimate values for samples marked as Bad.
Transform Driver Function¶
The driver function is the main function that performs the transformation on a variable. It is not called directly, but by one of the supported transform method functions.
Transform Interface Functions¶
Interface functions are the bridge between the core functions and the driver function. The driver function loops over the appropriate (one dimensional) slices of the input data, and calls the interface function for the transformation in that dimension. The interface function sets up all the arrays and parameters needed and used by the core functions, and takes the output of the core functions and converts it back into ADI’s internal data structures and objects. It is the interface function that knows how to find the information it needs from ADI, from variable attributes to numbers of elements to transform parameters. Thus, while the core function should be written in a generic way, coding an interface function requires some understanding of how ADI works and how the driver function will be calling it. A developer could easily provide his own core function to do a kind of transformation, but a developer working within the CDI environment will have to write an interface function to make it work with CDI.
Transform Method Functions¶
Generic functions that perform a transformation based on a particular approach. Methods supported includes bin average, interpolate, and subsample functions. These functions apply the mathematics necessary to perform the transform. Their work with regular C arrays and data structures and do not use ADI objects or other constructions. As a result, these functions can be written and used by processes outside of CDI.
Transform Metrics¶
Transform metrics are additional output data variables that provide details about the specific data points used in the transformation. The canonical example is the standard deviation of the points used in an average.
Transform Parameters¶
Transform parameters are variables that allow customization and information passing between different ADI functions and modules. They are often set by the user via the PCM or the configuration file, but can also be set internally through the appropriate cds function calls.
VAP¶
An ARM Value Added Product (VAP) is a process that reads in netCDF data, adds some scientific value to that data, and produces netCDF files that adhere to ARM’s data standards.