halodrops.processor

Module Contents

Classes

Sonde

Class identifying a sonde and containing its metadata

Data

_no_default

API

halodrops.processor._no_default

‘object(…)’

class halodrops.processor.Sonde

Class identifying a sonde and containing its metadata

A Sonde identifies an instrument that has been deployed. This means that pre-initialization sondes do not exist for this class.

Every Sonde mandatorily has a serial_id which is unique. Therefore, all instances with the same serial_id are to be considered as having the same metadata and data.

Optionally, the sonde also has metadata attributes, which can be broadly classified into:

• campaign and flight information

• location and time information of launch

• performance of the instrument and sensors

• other information such as reconditioning status, signal strength, etc.

sort_index: numpy.datetime64

‘field(…)’

serial_id: str

None

_: dataclasses.KW_ONLY

None

launch_time: Optional[Any]

None

__post_init__()

Initializes the ‘qc’ attribute as an empty object and sets the ‘sort_index’ attribute based on ‘launch_time’.

The ‘sort_index’ attribute is only applicable when ‘launch_time’ is available. If ‘launch_time’ is None, ‘sort_index’ will not be set.

add_flight_id(flight_id: str) → None

Sets attribute of flight ID

Parameters

flight_id : str The flight ID of the flight during which the sonde was launched

add_platform_id(platform_id: str) → None

Sets attribute of platform ID

Parameters

platform_id : str The platform ID of the flight during which the sonde was launched

add_spatial_coordinates_at_launch(launch_coordinates: List) → None

Sets attributes of spatial coordinates at launch

Expected units for altitude, latitude and longitude are meter above sea level, degree north and degree east, respectively.

Parameters

launch_coordinates : List List must be provided in the order of [launch_alt,launch_lat,launch_lon]

add_launch_detect(launch_detect_bool: bool) → None

Sets bool attribute of whether launch was detected

Parameters

launch_detect_bool : bool True if launch detected, else False

add_afile(path_to_afile: str) → None

Sets attribute with path to A-file of the sonde

Parameters

path_to_afile : str Path to the sonde’s A-file

add_level_dir()

run_aspen(path_to_postaspenfile: str = None) → None

Runs aspen and sets attribute with path to post-ASPEN file of the sonde

If the A-file path is known for the sonde, i.e. if the attribute path_to_afile exists, then the function will attempt to look for a post-ASPEN file of the same date-time as in the A-file’s name. Sometimes, the post-ASPEN file might not exist (e.g. because launch was not detected), and in such cases, ASPEN will run in a docker image and create the file.

If the A-file path is not known for the sonde, the function will expect the argument path_to_postaspenfile to be not empty.

Parameters

path_to_postaspenfile : str, optional The path to the post-ASPEN file. If not provided, the function will attempt to construct the path from the afile attribute.

Attributes Set

postaspenfile : str The path to the post-ASPEN file. This attribute is set if the file exists at the constructed or provided path.

add_aspen_ds() → None

Sets attribute with an xarray Dataset read from post-ASPEN file

The function will first check if the serial ID of the instance and that obtained from the global attributes of the post-ASPEN file match. If they don’t, function will print out an error.

If the postaspenfile attribute doesn’t exist, function will print out an error

filter_no_launch_detect() → None

Filter out sondes that did not detect a launch

The function will check if the launch_detect attribute exists and if it is False. If the attribute doesn’t exist, the function will raise an error. If the attribute exists and is False, the function will print a no-launch detected message. If the attribute exists and is True, the function will return the object.

This function serves as a checkpoint for filtering out sondes that did not detect a launch before running functions that will require aspen_ds, e.g. the QC functions.

Parameters

None

Returns

self : Sonde object The Sonde object itself, if the launch was detected, else None

Raises

ValueError If the launch_detect attribute does not exist.

detect_floater(gpsalt_threshold: float = 25, consecutive_time_steps: int = 3, skip: bool = False)

Detects if a sonde is a floater.

Parameters

gpsalt_threshold : float, optional The gpsalt altitude below which the sonde will check for time periods when gpsalt and pres have not changed. Default is 25. skip : bool, optional If True, the function will return the object without performing any operations. Default is False.

Returns

self The object itself with the new is_floater attribute added based on the function parameters.

crop_aspen_ds_to_landing_time()

Crops the aspen_ds to the time period before landing.

Parameters

None

Returns

self The object itself with the new cropped_aspen_ds attribute added if the sonde is a floater.

profile_fullness(variable_dict={'u_wind': 4, 'v_wind': 4, 'rh': 2, 'tdry': 2, 'pres': 2}, time_dimension='time', timestamp_frequency=4, fullness_threshold=0.75, add_fullness_fraction_attribute=True, skip=False)

Calculates the profile coverage for a given set of variables, considering their sampling frequency. If the sonde is a floater, the function will take the cropped_aspen_ds attribute (calculated with the crop_aspen_ds_to_landing_time method) as the dataset to calculate the profile coverage.

This function assumes that the time_dimension coordinates are spaced over 0.25 seconds, implying a timestamp_frequency of 4 hertz. This is applicable for ASPEN-processed QC and PQC files, specifically for RD41.

For each variable in the variable_dict, the function calculates the fullness fraction. If the fullness fraction is less than the fullness_threshold, it sets an attribute in self.qc named “profile_fullness_{variable}” to False. Otherwise, it sets this attribute to True.

If add_fullness_fraction_attribute is True, the function also sets an attribute in self named “profile_fullness_fraction_{variable}” to the calculated fullness fraction.

Parameters

variable_dict : dict, optional Dictionary containing the variables in self.aspen_ds and their respective sampling frequencies. The function will estimate the weighted profile-coverage for these variables. Default is {‘u_wind’:4,’v_wind’:4,’rh’:2,’tdry’:2,’pres’:2}. time_dimension : str, optional The independent dimension of the profile. Default is “time”. timestamp_frequency : numeric, optional The sampling frequency of time_dimension in hertz. Default is 4. fullness_threshold : float or str, optional The threshold for the fullness fraction. If the calculated fullness fraction is less than this threshold, the profile is considered not full. Default is 0.8. add_fullness_fraction_attribute : bool or str, optional If True, the function will add the fullness fraction as an attribute to the object. Default is True. If provided as string, it should be possible to convert it to a boolean value with the helper get_bool function. skip : bool, optional If True, the function will return the object without performing any operations. Default is False.

Returns

self The object itself, possibly with new attributes added based on the function parameters.

near_surface_coverage(variables=['u_wind', 'v_wind', 'rh', 'tdry', 'pres'], alt_bounds=[0, 1000], alt_dimension_name='alt', count_threshold=50, add_near_surface_count_attribute=True, skip=False)

Calculates the fraction of non-null values in specified variables near the surface.

Parameters

variables : list, optional The variables to consider for the calculation. Defaults to [“u_wind”,”v_wind”,”rh”,”tdry”,”pres”]. alt_bounds : list, optional The lower and upper bounds of altitude in meters to consider for the calculation. Defaults to [0,1000]. alt_dimension_name : str, optional The name of the altitude dimension. Defaults to “alt”. If the sonde is a floater, this will be set to “gpsalt” regardless of user-provided value. count_threshold : int, optional The minimum count of non-null values required for a variable to be considered as having near surface coverage. Defaults to 50. add_near_surface_count_attribute : bool, optional If True, adds the count of non-null values as an attribute for every variable to the object. Defaults to True. skip : bool, optional If True, skips the calculation and returns the object as is. Defaults to False.

Returns

self The object with updated attributes.

Raises

ValueError If the attribute aspen_ds does not exist. The add_aspen_ds method should be run first.

filter_qc_fail(filter_flags=None)

Filters the sonde based on a list of QC flags. If any of the flags are False, the sonde will be filtered out from creating L2. If the sonde passes all the QC checks, the attributes listed in filter_flags will be removed from the sonde object.

Parameters

filter_flags : str or list, optional Comma-separated string or list of QC-related attribute names to be checked. Each item can be a specific attribute name or a prefix to include all attributes starting with that prefix. You can also provide ‘all_except_’ to filter all flags except those starting with ‘’. If ‘all_except_’ is provided, it should be the only value in filter_flags. If not provided, no sondes will be filtered.

Returns

self : object The sonde object itself, with the attributes listed in filter_flags removed if it passes all the QC checks.

Raises

ValueError If a flag in filter_flags does not exist as an attribute of the sonde object, or if ‘all_except_’ is provided in filter_flags along with other values. Please ensure that the flag names provided in ‘filter_flags’ correspond to existing QC attributes. If you’re using a prefix to filter attributes, make sure the prefix is correct. Check your skipped QC functions or your provided list of filter flags.

create_interim_l2_ds()

Creates an interim L2 dataset from the aspen_ds or cropped_aspen_ds attribute.

Parameters

None

Returns

self : object Returns the sonde object with the interim L2 dataset added as an attribute.

convert_to_si(variables=['rh', 'pres', 'tdry'], skip=False)

Converts variables to SI units.

Parameters

variables : list or str, optional The variables to convert to SI units. If a string is provided, it should be a comma-separated list of variables. The default variables are ‘rh’, ‘pres’, and ‘tdry’.

skip : bool, optional If set to True, the function will skip the conversion process but will still ensure that the ‘_interim_l2_ds’ attribute is set. If ‘_interim_l2_ds’ is not already an attribute of the object, it will be set to ‘aspen_ds’. Default is False.

Returns

self : object Returns the sonde object with the specified variables in aspen_ds converted to SI units. If ‘skip’ is set to True, it returns the sonde object with ‘_interim_l2_ds’ set to ‘aspen_ds’ if it wasn’t already present.

get_l2_variables(l2_variables: dict = hh.l2_variables)

Gets the variables from aspen_ds to create L2.

Parameters

l2_variables : dict or str, optional A dictionary where the keys are the variables in aspen_ds to keep for L2. If dictionary items contain a ‘rename_to’ key, the variable will be renamed to the value of ‘rename_to’. If dictionary items contain a ‘attributes’ key, the variable will be assigned the attributes in the value of ‘attributes’. The default is the l2_variables dictionary from the helper module.

Returns

self : object Returns the sonde object with only the specified variables (renamed if dictionary has ‘rename_to’ key and attributes added if dictionary has ‘attributes’ key) in _interim_l2_ds attribute. If ‘_interim_l2_ds’ is not already an attribute of the object, it will first be set to ‘aspen_ds’ before reducing to the variables and renaming.

add_sonde_id_variable(variable_name='sonde_id')

Adds a variable and related attributes to the sonde object with the Sonde object (self)’s serial_id attribute.

Parameters

variable_name : str, optional The name of the variable to be added. Default is ‘sonde_id’.

Returns

self : object Returns the sonde object with a variable containing serial_id. Name of the variable provided by ‘variable_name’.

get_flight_attributes(l2_flight_attributes_map: dict = hh.l2_flight_attributes_map) → None

Gets flight attributes from the A-file and adds them to the sonde object.

Parameters

l2_flight_attributes_map : dict or str, optional A dictionary where the keys are the flight attributes in the A-file and the values are the corresponding (renamed) attribute names to be used for the L2 file. The default is the l2_flight_attributes_map dictionary from the helper module.

Returns

self : object Returns the sonde object with the flight attributes added as attributes.

get_other_global_attributes()

add_global_attributes_to_interim_l2_ds()

Adds global attributes to _interim_l2_ds.

Parameters

None

Returns

self : object Returns the sonde object with global attributes added to _interim_l2_ds.

add_compression_and_encoding_properties(encoding_variables: dict = hh.encoding_variables, default_variable_compression_properties: dict = hh.variable_compression_properties)

Adds compression and encoding properties to _interim_l2_ds.

Parameters

comp : dict or str, optional A dictionary containing the compression properties to be used for the L2 file. The default is the comp dictionary from the helper module.

Returns

self : object Returns the sonde object with compression and encoding properties added to _interim_l2_ds.

get_l2_filename(l2_filename: str = None, l2_filename_template: str = None)

Gets the L2 filename from the template provided.

Parameters

l2_filename : str, optional The L2 filename. The default is the l2_filename_template from the helper module.

Returns

self : object Returns the sonde object with the L2 filename added as an attribute.

write_l2(l2_dir: str = None)

Writes the L2 file to the specified directory.

Parameters

l2_dir : str, optional The directory to write the L2 file to. The default is the directory of the A-file with ‘0’ replaced by ‘2’.

Returns

self : object Returns the sonde object with the L2 file written to the specified directory using the l2_filename attribute to set the name.

add_l2_ds(l2_dir: str = None)

Adds the L2 dataset as an attribute to the sonde object.

Parameters

l2_dir : str, optional The directory to read the L2 file from. The default is the directory of the A-file with ‘0’ replaced by ‘2’.

Returns

self : object Returns the sonde object with the L2 dataset added as an attribute.

add_q_and_theta_to_l2_ds()

Adds potential temperature and specific humidity to the L2 dataset.

Parameters

None

Returns

self : object Returns the sonde object with potential temperature and specific humidity added to the L2 dataset.