8.6.1.2. pymodaq_data.h5modules.backends module

Created the 15/11/2022

@author: Sebastien Weber

exception pymodaq_data.h5modules.backends.InvalidDataDimension[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.InvalidDataType[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.InvalidExport[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.InvalidGroupDataType[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.InvalidGroupType[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.InvalidSave[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.InvalidScanType[source]: Bases: Exception

exception pymodaq_data.h5modules.backends.NodeError[source]: Bases: Exception

class pymodaq_data.h5modules.backends.Attributes(node, backend='tables')[source]

Bases: object

Attributes:

attrs_name
node

Methods

`get`(key[, default])	Return the attribute value for key, or default if not present.
`to_dict`()	Returns attributes name/value as a dict

get(key, default=None)[source]: Return the attribute value for key, or default if not present.

to_dict()[source]

Returns attributes name/value as a dict

Return type:: dict

property attrs_name

property node

class pymodaq_data.h5modules.backends.CARRAY(node, backend)[source]

Bases: Node

Attributes:

array

Methods

read

read()[source]

property array

class pymodaq_data.h5modules.backends.EARRAY(array, backend)[source]

Bases: CARRAY

Methods

append(data[, expand])

appends a ndarray after the current data in the enlargeable array

append_backend

append(data, expand=True)[source]

appends a ndarray after the current data in the enlargeable array

Considering the shape length of the enlargeable array is n+1

The data to append could be:

a single element (without the enlargeable shape index that is always the first

index, that is of shape length n). In that case the first index of the enlargeable array is increased by one. * an ensemble of elements (a ndarray) of shape length of (n+1).

Parameters:

data (ndarray) – the data array to append to the enlargeable node
expand (bool) – If True the data array will have its shape expanded by one dim

append_backend(data)[source]

class pymodaq_data.h5modules.backends.GROUP(node, backend)[source]

Bases: Node

Methods

`children`()	Get a dict containing all children node hanging from self whith their name as keys
`children_name`()	Gets the sorted list of children name hanging from self

get_child
remove_children

children()[source]

Get a dict containing all children node hanging from self whith their name as keys

Returns:: dict
Return type:: Dict[str, Node]

See also

children_name

children_name()[source]

Gets the sorted list of children name hanging from self

Returns:: list
Return type:: list of name of the children

get_child(name)[source]

Return type:: Node

remove_children()[source]

class pymodaq_data.h5modules.backends.GroupType(*values)[source]

Bases: BaseEnum

actuator = 1

ch = 3

data = 2

data_dim = 6

data_logger = 7

detector = 0

external_h5 = 5

scan = 4

class pymodaq_data.h5modules.backends.H5Backend(backend='tables')[source]

Bases: object

Attributes:

filename
h5file
is_swmr_active: Return True if SWMR mode is currently active on the file.
is_swmr_capable: Return True if the current backend supports SWMR mode.
is_swmr_compatible: Return True if the open file was created with SWMR support.

Methods

`add_group`(group_name, group_type, where[, ...])	Add a node in the h5 file tree of the group type :type group_name: `(str) a custom name for this group` :param group_name: :type group_name: `(str) a custom name for this group` :param group_type: one of the possible values of GroupType, should be enforced by higher level modules not here :type group_type: `Union`[`GroupType`, `str`] :type where: `(str` or `node) parent node where` to `create the new group` :param where: :type where: `(str` or `node) parent node where` to `create the new group` :type metadata: `(dict) extra metadata` to `be saved with this new group node` :param metadata: :type metadata: `(dict) extra metadata` to `be saved with this new group node`
`close_file`()	Flush data and close the h5file
`create_earray`(where, name, dtype[, ...])	create enlargeable arrays from data with a given shape and of a given type.
`create_vlarray`(where, name, dtype[, title])	create variable data length and type and enlargeable 1D arrays
`define_compression`(compression, compression_opts)	Define cmpression library and level of compression :type compression: `(str) either gzip` and `zlib are supported here as they are compatible` :param compression: but zlib is used by pytables while gzip is used by h5py :type compression: `(str) either gzip` and `zlib are supported here as they are compatible` :param compression_opts (int): :type compression_opts (int): `0` to `9 0`: `None`, `9`: `maximum compression`
`enable_swmr`()	Activate SWMR mode on the open h5py file.
`finalize_swmr`([keep_open])	End SWMR by closing the file, reopening in 'a' mode, and reconciling deferred attrs.
`get_children`(where)	Get a dict containing all children node hanging from where with their name as keys and types among Node, CARRAY, EARRAY, VLARRAY or StringARRAY
`get_node_name`(node)	return node name :param node (str or node instance): :param see h5py and pytables documentation on nodes:
`get_node_path`(node)	return node path :param node (str or node instance): :param see h5py and pytables documentation on nodes:
`get_set_group`(where, name[, title])	Retrieve or create (if absent) a node group Get attributed to the class attribute `current_group`
`is_node_in_group`(where, name)	Check if a given node with name is in the group defined by where (comparison on lower case strings) :type where: `(str` or `node)` :param where: path or parent node instance :type where: `(str` or `node)` :type name: `(str)` :param name: group node name :type name: `(str)`
`reconcile_swmr_attrs`()	Walk all EARRAY/VLARRAY nodes and update attrs['shape'] from actual data.
`set_backend`(backend)	Switch the active backend, closing any open file first.

create_carray
flush
get_attr
get_group_by_title
get_node
get_parent_node
has_attr
isopen
open_file
read
root
save_file_as
set_attr
walk_groups
walk_nodes

add_group(group_name, group_type, where, title='', metadata=None)[source]

Add a node in the h5 file tree of the group type :type group_name: (str) a custom name for this group :param group_name: :type group_name: (str) a custom name for this group :param group_type: one of the possible values of GroupType, should be enforced by higher level modules not here :type group_type: Union[GroupType, str] :type where: (str or node) parent node where to create the new group :param where: :type where: (str or node) parent node where to create the new group :type metadata: (dict) extra metadata to be saved with this new group node :param metadata: :type metadata: (dict) extra metadata to be saved with this new group node

Returns:: (node)
Return type:: GROUP

close_file()[source]: Flush data and close the h5file

create_carray(where, name, obj=None, title='')[source]

create_earray(where, name, dtype, data_shape=None, title='')[source]: create enlargeable arrays from data with a given shape and of a given type. The array is enlargeable along the first dimension

create_vlarray(where, name, dtype, title='')[source]

create variable data length and type and enlargeable 1D arrays

Parameters:

where ((str) group location in the file where to create the array node)
name ((str) name of the array)
dtype ((dtype) numpy dtype style, for particular case of strings, use dtype=``’string’``)
title ((str) node title attribute (written in capitals))

Return type:

array

define_compression(compression, compression_opts)[source]: Define cmpression library and level of compression :type compression: (str) either gzip and zlib are supported here as they are compatible :param compression: but zlib is used by pytables while gzip is used by h5py :type compression: (str) either gzip and zlib are supported here as they are compatible :param compression_opts (int): :type compression_opts (int): 0 to 9 0: None, 9: maximum compression

enable_swmr()[source]

Activate SWMR mode on the open h5py file.

Must be called after all groups/datasets have been created. Raises RuntimeError if backend is not h5py or file was not opened with swmr_mode=True. Idempotent: does nothing if already enabled.

finalize_swmr(keep_open=False)[source]

End SWMR by closing the file, reopening in ‘a’ mode, and reconciling deferred attrs.

After SWMR mode, attrs[‘shape’] on EARRAY/VLARRAY nodes may be stale. This method closes the file (ending SWMR), reopens it in append mode, and updates all deferred attributes.

Parameters:: keep_open (bool) – If True, leaves the file open in ‘a’ mode after reconciling. If False (default), closes the file after reconciling.

flush()[source]

get_attr(node, attr_name=None)[source]

get_children(where)[source]

Get a dict containing all children node hanging from where with their name as keys and types among Node, CARRAY, EARRAY, VLARRAY or StringARRAY

Parameters:: instance) (where (str or node) – see h5py and pytables documentation on nodes, and Node objects of this module
Returns:: dict
Return type:: keys are children node names, values are the children nodes

See also

GROUP.children_name()

get_group_by_title(where, title)[source]

get_node(where, name=None)[source]

Return type:: Node

get_node_name(node)[source]

return node name :param node (str or node instance): :param see h5py and pytables documentation on nodes:

Returns:: str
Return type:: name of the node

get_node_path(node)[source]

return node path :param node (str or node instance): :param see h5py and pytables documentation on nodes:

Returns:: str
Return type:: full path of the node

get_parent_node(node)[source]

get_set_group(where, name, title='', **kwargs)[source]

Retrieve or create (if absent) a node group Get attributed to the class attribute current_group

Parameters:

where (str or node) – path or parent node instance
name (str) – group node name
title (str) – node title
Arguments (Keyword) – any other metadata related to this node (for example: origin)

Returns:

group

Return type:

group node

has_attr(node, attr_name)[source]

is_node_in_group(where, name)[source]

Check if a given node with name is in the group defined by where (comparison on lower case strings) :type where: (str or node) :param where: path or parent node instance :type where: (str or node) :type name: (str) :param name: group node name :type name: (str)

Returns:: True if node exists, False otherwise
Return type:: bool

isopen()[source]

open_file(fullpathname, mode='r', title='PyMoDAQ file', swmr_mode=False, **kwargs)[source]

read(array, *args, **kwargs)[source]

reconcile_swmr_attrs()[source]

Walk all EARRAY/VLARRAY nodes and update attrs[‘shape’] from actual data.

Called after SWMR is ended (file closed and reopened in ‘a’ mode) to fix deferred attribute writes that were skipped during SWMR.

root()[source]

save_file_as(filenamepath='h5copy.txt')[source]

set_attr(node, attr_name, attr_value)[source]

set_backend(backend)[source]

Switch the active backend, closing any open file first.

Updates both self.backend (the name string) and self.h5_library (the imported module), which both need to be consistent for file operations.

Parameters:: backend (str) – One of 'tables', 'h5py', or 'h5pyd'.

walk_groups(where)[source]

walk_nodes(where)[source]

property filename

property h5file

property is_swmr_active: Return True if SWMR mode is currently active on the file.

property is_swmr_capable: Return True if the current backend supports SWMR mode.

property is_swmr_compatible: Return True if the open file was created with SWMR support.

class pymodaq_data.h5modules.backends.Node(node, backend)[source]

Bases: object

Attributes:

attrs
h5file
name: return node name
node
parent_node
path: return node path
title

Methods

get_attr
set_attr
to_h5_backend

get_attr(item)[source]

set_attr(key, value)[source]

to_h5_backend()[source]

Return type:: H5Backend

property attrs

property h5file

property name: return node name

property node

property parent_node: GROUP

property path

return node path :param node (str or node instance): :param see h5py and pytables documentation on nodes:

Returns:: str
Return type:: full path of the node

property title

class pymodaq_data.h5modules.backends.SaveType(*values)[source]

Bases: BaseEnum

actuator = 4

custom = 3

detector = 1

logger = 2

optimizer = 5

scan = 0

class pymodaq_data.h5modules.backends.StringARRAY(array, backend)[source]

Bases: VLARRAY

Methods

append(string)

array_to_string
read
string_to_array

append(string)[source]

array_to_string(array)[source]

read()[source]

string_to_array(string)[source]

class pymodaq_data.h5modules.backends.VLARRAY(array, backend)[source]

Bases: EARRAY

Methods

`append`(data)
`append_backend`(data)	Append one variable-length element.

append(data)[source]

append_backend(data)[source]

Append one variable-length element.

VLARRAY in h5py stores variable-length arrays as individual elements (one per row). EARRAY.append_backend incorrectly uses data.shape[0] as the number of new rows, which would split one VL element into many rows.

pymodaq_data.h5modules.backends.check_mandatory_attrs(attr_name, attr)[source]

for cross compatibility between different backends. If these attributes have binary value, then decode them

Parameters:

attr_name
attr

pymodaq_data.h5modules.backends.get_attr(node, attr_name, backend='tables')[source]

pymodaq_data.h5modules.backends.set_attr(node, attr_name, attr_value, backend='tables')[source]