8.6.1.6. pymodaq_data.h5modules.saving module

Created the 15/11/2022

@author: Sebastien Weber

class pymodaq_data.h5modules.saving.DataType(*values)

Bases: BaseEnum

axis = 'Axis'

bkg = 'Bkg'

data = 'Data'

data_enlargeable = 'EnlData'

error = 'ErrorBar'

external_h5 = 'ExtData'

live_scan = 'Live'

strings = 'Strings'

class pymodaq_data.h5modules.saving.FileType(*values)

Bases: BaseEnum

actuator = 1

axis = 2

detector = 0

scan = 3

class pymodaq_data.h5modules.saving.H5SaverLowLevel(save_type='scan', backend=None)

Bases: H5Backend

Object containing basic methods in order to structure and interact with a h5file compatible with the h5browser

See also

H5Browser

h5_file

object used to save all datas and metadas

Type:

pytables hdf5 file

h5_file_path

The file path

Type:

str or Path

Attributes:

h5_file

raw_group

Methods

add_act_group(where[, title, ...])	Add a new group of type detector .
add_array(where, name, data_type[, ...])	save data arrays on the hdf5 file together with metadata :param where: node where to save the array :type where: Union[GROUP, str] :param name: name of the array in the hdf5 file :type name: str :param data_type: mandatory so that the h5Browser can interpret correctly the array :type data_type: DataType :param data_shape: the shape of the array to save, mandatory if array_to_save is None :type data_shape: tuple :param data_dimension: The data's dimension :type data_dimension: DataDim :param scan_shape: the shape of the scan dimensions :type scan_shape: tuple :param title: the title attribute of the array node :type title: str :param array_to_save: data to be saved in the array. If None, array_type and data_shape should be specified in order to init correctly the memory :type array_to_save: ndarray :param array_type: eg np.float, np.int32 ... :type array_type: dtype :type fill_value: float or int :param fill_value: value to be used to fill the array if array_to_save is None :type fill_value: float or int :param enlargeable: if False, data are saved as a CARRAY, otherwise as a EARRAY (for ragged data, see add_string_array) :type enlargeable: bool :type metadata: dict :param metadata: dictionnary whose keys will be saved as the array attributes :type metadata: dict :type add_scan_dim: if True, the scan axes dimension (scan_shape iterable) is prepended to the array shape on the hdf5 :param add_scan_dim: In that case, the array is usually initialized as zero and further populated :type add_scan_dim: if True, the scan axes dimension (scan_shape iterable) is prepended to the array shape on the hdf5.
add_ch_group(where[, title, ...])	Add a new group of type channel .
add_data_group(where, data_dim[, title, ...])	Creates a group node at given location in the tree
add_det_group(where[, title, ...])	Add a new group of type detector .
add_generic_group([where, title, ...])	Add a new group of type given by the input argument group_type
add_incremental_group(group_type, where[, ...])	Add a node in the h5 file tree of the group type with an increment in the given name :param group_type: one of the possible values of group_types :type group_type: Union[str, GroupType, Enum] :type where: str or node :param where: parent node where to create the new group :type where: str or node :type title: str :param title: node title :type title: str :type settings_as_xml: str :param settings_as_xml: XML string containing Parameter representation :type settings_as_xml: str :type metadata: dict :param metadata: extra metadata to be saved with this new group node :type metadata: dict
add_move_group(where[, title, ...])	Add a new group of type actuator .
add_scan_group([where, title, ...])	Add a new group of type scan
from_file(path[, save_type, new_file, metadata])	Create and initialise an H5SaverLowLevel from a file path.
get_groups(where, group_type)	Get all groups hanging from a Group and of a certain type
get_node_from_attribute_match(where, ...)	Get a Node starting from a given node (Group) matching a pair of node attribute name and value
get_node_from_title(where, title)	Get a Node starting from a given node (Group) matching the given title
get_set_group(where, name[, title])	Get the group located at where if it exists otherwise creates it
get_set_logger([where])	Retrieve or create (if absent) a logger enlargeable array to store logs Get attributed to the class attribute logger_array :param where: location within the tree where to save or retrieve the array :type where: Node
init_file(file_name[, raw_group_name, ...])	Initializes a new h5 file,
set_swmr_flush_interval(interval)	Set how often to flush data for SWMR readers.
tick_flush()	Increment the write counter and flush if the interval is reached.

add_log
add_string_array
get_last_group
save_file

classmethod from_file(path, save_type='scan', new_file=False, metadata=None)

Create and initialise an H5SaverLowLevel from a file path.

Convenience factory that combines the constructor and init_file() call into a single expression. :param path: Full path to the HDF5 file. :type path: Union[Path, str] :param save_type: Type label stored in the file (default 'scan'). :type save_type: SaveType :param new_file: If True a new file is created, otherwise the existing file is

opened for appending.

Parameters:

metadata (dict) – Extra attributes written to the raw-data group on creation.

Returns:

A fully initialised instance ready for reading/writing.

Return type:

H5SaverLowLevel

add_act_group(where, title='', settings_as_xml='', metadata=None)

Add a new group of type detector .. seealso:: add_incremental_group

add_array(where, name, data_type, array_to_save=None, data_shape=None, array_type=None, fill_value=None, data_dimension=None, scan_shape=(), add_scan_dim=False, enlargeable=False, title='', metadata={})

save data arrays on the hdf5 file together with metadata :param where: node where to save the array :type where: Union[GROUP, str] :param name: name of the array in the hdf5 file :type name: str :param data_type: mandatory so that the h5Browser can interpret correctly the array :type data_type: DataType :param data_shape: the shape of the array to save, mandatory if array_to_save is None :type data_shape: tuple :param data_dimension: The data’s dimension :type data_dimension: DataDim :param scan_shape: the shape of the scan dimensions :type scan_shape: tuple :param title: the title attribute of the array node :type title: str :param array_to_save: data to be saved in the array. If None, array_type and data_shape should be specified in order to init

correctly the memory

Parameters:

• array_type (dtype) – eg np.float, np.int32 …

• fill_value (float or int) – value to be used to fill the array if array_to_save is None

• enlargeable (bool) – if False, data are saved as a CARRAY, otherwise as a EARRAY (for ragged data, see add_string_array)

• metadata (dict) – dictionnary whose keys will be saved as the array attributes

• add_scan_dim (if True, the scan axes dimension (scan_shape iterable) is prepended to the array shape on the hdf5) – In that case, the array is usually initialized as zero and further populated

Return type:

array (CARRAY or EARRAY)

See also

add_data, add_string_array

add_ch_group(where, title='', settings_as_xml='', metadata=None)

Add a new group of type channel .. seealso:: add_incremental_group

add_data_group(where, data_dim, title='', settings_as_xml='', metadata=None, group_name=None)

Creates a group node at given location in the tree

Parameters:

• where (group node) – where to create data group

• data_dim (DataDim) – the dimensionality of the data group

• title (str, optional) – a title for this node, will be saved as metadata

• settings_as_xml (str, optional) – XML string created from a Parameter object to be saved as metadata

• metadata (dict, optional) – will be saved as a new metadata attribute with name: key and value: dict value

• group_name (str) – the name of the group to create if None, the name of the DataDim enum is used (default)

Returns:

group

Return type:

group node

See also

add_group()

add_det_group(where, title='', settings_as_xml='', metadata=None)

Add a new group of type detector .. seealso:: add_incremental_group

add_generic_group(where='/RawData', title='', settings_as_xml='', metadata=None, group_type=GroupType.scan)

Add a new group of type given by the input argument group_type

At creation adds the attributes description to be used elsewhere

See also

add_incremental_group

add_incremental_group(group_type, where, title='', settings_as_xml='', metadata=None)

Add a node in the h5 file tree of the group type with an increment in the given name :param group_type: one of the possible values of group_types :type group_type: Union[str, GroupType, Enum] :type where: str or node :param where: parent node where to create the new group :type where: str or node :type title: str :param title: node title :type title: str :type settings_as_xml: str :param settings_as_xml: XML string containing Parameter representation :type settings_as_xml: str :type metadata: dict :param metadata: extra metadata to be saved with this new group node :type metadata: dict

Returns:

node

Return type:

newly created group node

add_log(msg)

add_move_group(where, title='', settings_as_xml='', metadata=None)

Add a new group of type actuator .. seealso:: add_incremental_group

add_scan_group(where='/RawData', title='', settings_as_xml='', metadata=None)

Add a new group of type scan

deprecated, use add_generic_group with a group type as GroupType.scan

add_string_array(where, name, title='', metadata={})

get_groups(where, group_type)

Get all groups hanging from a Group and of a certain type

get_last_group(where, group_type)

get_node_from_attribute_match(where, attr_name, attr_value)

Get a Node starting from a given node (Group) matching a pair of node attribute name and value

get_node_from_title(where, title)

Get a Node starting from a given node (Group) matching the given title

get_set_group(where, name, title='', **kwargs)

Get the group located at where if it exists otherwise creates it

This also set the _current_group property

get_set_logger(where=None)

Retrieve or create (if absent) a logger enlargeable array to store logs Get attributed to the class attribute logger_array :param where: location within the tree where to save or retrieve the array :type where: Node

Returns:

enlargeable array accepting strings as elements

Return type:

VLARRAY

init_file(file_name, raw_group_name='RawData', new_file=False, metadata=None, swmr_mode=False)

Initializes a new h5 file,

Should have an extension with h5 in it.

Parameters:

• file_name (Path) – a complete Path pointing to a h5 file

• raw_group_name (str) – Base node name

• new_file (bool) – If True create a new file, otherwise append to a potential existing one

• metadata (dict) – A dictionary to be saved as attributes

• swmr_mode (bool) – If True, prepare the file for SWMR (h5py backend only)

Returns:

True if new file has been created, False otherwise

Return type:

bool

save_file(filename=None)

set_swmr_flush_interval(interval)

Set how often to flush data for SWMR readers.

Parameters:

interval (int) – 0 = flush only at end, N = every N writes

tick_flush()

Increment the write counter and flush if the interval is reached.

No-op if SWMR is not active or flush interval is 0. To be called by data savers after each logical data write.

fill_value: float

property h5_file

property raw_group