8.4.6. Data Management

DataDim(*values)	Enum for dimensionality representation of data
DataSource(*values)	Enum for source of data
DataDistribution(*values)	Enum for distribution of data
Axis(*args, **kwargs)	Object holding info and data about physical axis of some data
DataBase(*args, **kwargs)	Base object to store homogeneous data and metadata generated by pymodaq's objects.
DataWithAxes(*args, **kwargs)	Data object with Axis objects corresponding to underlying data nd-arrays
DataRaw(name[, dim, distribution, data, ...])	Specialized DataWithAxes set with source as 'raw'.
DataCalculated(name[, dim, distribution, ...])	Specialized DataWithAxes set with source as 'calculated'.
DataFromRoi(name[, dim, distribution, data, ...])	Specialized DataWithAxes set with source as 'calculated'.
DataToExport(*args, **kwargs)	Object to store all raw and calculated DataWithAxes data for later exporting, saving, sending signal...
pymodaq.utils.data.DataFromPlugins(*args, ...)	Specialized DataWithAxes set with source as 'raw'.
pymodaq.utils.data.DataActuator(*args, **kwargs)	Specialized DataWithAxes set with source as 'raw'.

8.4.6.1. Axes

Created the 28/10/2022

@author: Sebastien Weber

class pymodaq_data.data.Axis(*args, **kwargs)

Object holding info and data about physical axis of some data

In case the axis’s data is linear, store the info as a scale and offset else store the data

Parameters:

• label (str) – The label of the axis, for instance ‘time’ for a temporal axis

• units (str) – The units of the data in the object, for instance ‘s’ for seconds

• data (ndarray) – A 1D ndarray holding the data of the axis

• index (int) – an integer representing the index of the Data object this axis is related to

• scaling (float) – The scaling to apply to a linspace version in order to obtain the proper scaling

• offset (float) – The offset to apply to a linspace/scaled version in order to obtain the proper axis

• size (int) – The size of the axis array (to be specified if data is None)

• spread_order (int) – An integer needed in the case where data has a spread DataDistribution. It refers to the index along the data’s spread_index dimension

Examples

>>> axis = Axis('myaxis', units='seconds', data=np.array([1,2,3,4,5]), index=0)

static create_simple_linear_data(nsteps)

Return the array [0, 1, …, nsteps-1]

static deserialize(bytes_str)

Convert bytes into an Axis object

Convert the first bytes into an Axis reading first information about the Axis

Return type:

Tuple[Axis, bytes]

Returns:

• Axis (the decoded Axis)

• bytes (the remaining bytes string if any)

static from_quantity(quantity, label='axis', index=0)

Create an Axis whose data and units are those of a pint Quantity

Return type:

Axis

static serialize(axis)

Convert an Axis object into a bytes message together with the info to convert it back

Parameters:

axis (Axis)

Returns:

bytes

Return type:

the total bytes message to serialize the Axis

Notes

The bytes sequence is constructed as:

• serialize the axis label

• serialize the axis units

• serialize the axis array

• serialize the axis

• serialize the axis spread_order

as_dwa(set_itself_as_axis=False)

Convert this axis to a DataRaw whose data are the axis values

If set_itself_as_axis is False, the axes of the returned object are created with default values.

Return type:

DataWithAxes

copy()

Return a shallow copy of this axis

create_linear_data(nsteps)

replace the axis data with a linear version using scaling and offset

find_index(threshold)

find the index of the threshold value within the axis

Return type:

int

find_indexes(thresholds)

Find the indexes of several values within the axis, see find_index()

Return type:

Iterable[int]

flip()

flip the direction of the axis

force_units(units)

Change immediately the units to whatever else. Use this with care!

get_data()

Convenience method to obtain the axis data (usually None because scaling and offset are used)

Return type:

ndarray

get_data_at(indexes)

Get data at specified indexes

Parameters:

indexes (Union[int, Iterable, slice])

Return type:

Union[ndarray, generic]

get_quantity()

Convenience method to obtain the numerical data as a quantity array

Return type:

Quantity

get_scale_offset_from_data(data=None)

Get the scaling and offset from the axis’s data

If data is not None, extract the scaling and offset

Parameters:

data (Union[ndarray, Quantity, None])

is_axis_linear(data=None)

Check if the axis values are linearly spaced

Uses data if given, else the axis data. Returns False if there is no data.

max()

Maximum value of the axis

mean()

Mean value of the axis (for linear axes, the value at the middle of the axis)

min()

Minimum value of the axis

to_base_units(inplace=False)

Convert the axis to pint base units

If inplace is True the axis is modified and None is returned, else a new Axis is returned.

to_reduced_units(inplace=False)

Convert the axis to reduced units (see dimensionless_aware_reduce_units())

If inplace is True the axis is modified and None is returned, else a new Axis is returned.

units_as(units, inplace=True, context=None, **context_kwargs)

Convert the axis to other units, optionally using a pint context

If inplace is True the axis is modified and None is returned, else a new Axis is returned.

property axis_width: Quantity

get the width of the axis in axis unit

That is the difference between the max value and the min value of the axis

property data: ndarray | None

get/set the data of Axis (None if the axis is linear, see get_data())

Type:

np.ndarray

Setter accepts:

Union[ndarray, Quantity, None]

property index: int

get/set the index this axis corresponds to in a DataWithAxis object

Type:

int

property label: str

get/set the label of this axis

Type:

str

property size: int

get/set the size/length of the 1D ndarray

Type:

int

property units: str

get/set the units for this axis without conversion (equivalent to force_units)

Type:

str

8.4.6.2. DataObjects

Created the 28/10/2022

@author: Sebastien Weber

class pymodaq_data.data.DataBase(*args, **kwargs)

Base object to store homogeneous data and metadata generated by pymodaq’s objects.

To be inherited for real data

Parameters:

• name (str) – the identifier of these data

• source (DataSource or str) – Enum specifying if data are raw or processed (for instance from roi)

• dim (DataDim or str) – The identifier of the data type

• distribution (DataDistribution or str) – The distribution type of the data: uniform if distributed on a regular grid or spread if on specific unordered points

• data (list of ndarray or Quantities) – The data the object is storing. In case of Quantities, the object units attribute will be forced to the unit of this quantity, ignoring the units argument.

• labels (list of str) – The labels of the data nd-arrays

• origin (str) – An identifier of the element where the data originated, for instance the DAQ_Viewer’s name. Used when appending DataToExport in DAQ_Scan to disintricate from which origin data comes from when scanning multiple detectors.

• units (str) – A unit string identifier as specified in the UnitRegistry of the pint module

• kwargs (named parameters) – All other parameters are stored dynamically using the name/value pair. The name of these extra parameters are added into the extra_attributes attribute

Variables:

• name (str) – the identifier of these data

• source (DataSource or str) – Enum specifying if data are raw or processed (for instance from roi)

• dim (DataDim or str) – The identifier of the data type

• distribution (DataDistribution or str) – The distribution type of the data: uniform if distributed on a regular grid or spread if on specific unordered points

• data (list of ndarray) – The data the object is storing

• labels (list of str) – The labels of the data nd-arrays

• origin (str) – An identifier of the element where the data originated, for instance the DAQ_Viewer’s name. Used when appending DataToExport in DAQ_Scan to disintricate from which origin data comes from when scanning multiple detectors.

• shape (Tuple[int, ]) – The shape of the underlying data

• size (int) – The size of the ndarrays stored in the object

• length (int) – The number of ndarrays stored in the object

• extra_attributes (List[str]) – list of string giving identifiers of the attributes added dynamically at the initialization (for instance to save extra metadata using the DataSaverLoader

See also

DataWithAxes, DataFromPlugins, DataRaw, DataSaverLoader

Examples

>>> import numpy as np
>>> from pymodaq.utils.data import DataBase, DataSource, DataDim, DataDistribution
>>> data = DataBase('mydata', source=DataSource['raw'], dim=DataDim['Data1D'],     distribution=DataDistribution.uniform, data=[np.array([1.,2.,3.]), np.array([4.,5.,6.])],    labels=['channel1', 'channel2'], origin='docutils code')
>>> data.dim
<DataDim.Data1D: 1>
>>> data.source
<DataSource.raw: 0>
>>> data.shape
(3,)
>>> data.length
2
>>> data.size
3

static get_origin_name_from_full_name(full_name)

Standardize the obtention of the origin and name from the full name expression

Origin is always the first bit before the first ‘/’ character while the name will be the remaining characters

abs()

Take the absolute value of itself

angle()

Take the phase value of itself

append(data)

Append data content if the underlying arrays have the same shape and compatible units

as_dte(name='mydte')

Convenience method to wrap the DataWithAxes object into a DataToExport

Return type:

DataToExport

average(other, weight)

Compute the weighted average between self and other DataBase

Parameters:

• other_data (DataBase)

• weight (int) – The weight the ‘other’ holds with respect to self

Returns:

DataBase

Return type:

DataBase

check_shape_from_data(data)

Set the shape of the object from the first array of data

deepcopy()

Return a deep copy of this object

equal_to(other, epsilon)

Check if two data object are equal within epsilon

Return type:

bool

fliplr()

Reverse the order of elements along axis 1 (left/right)

flipud()

Reverse the order of elements along axis 0 (up/down)

force_units(units)

Change immediately the units to whatever else. Use this with care!

get_data_index(index=0)

Get the data by its index in the list, same as self[index]

Return type:

ndarray

get_dim_from_data(data)

Get the dimensionality DataDim from data

get_full_name()

Get the data ful name including the origin attribute into the returned value

Returns:

str

Return type:

str

Examples

d0 = DataBase(name=’datafromdet0’, origin=’det0’)

imag()

Take the imaginary part of itself

pop(index)

Returns a copy of self but with data taken at the specified index

Return type:

DataBase

real()

Take the real part of itself

set_dim(dim)

Addhoc modification of dim independantly of the real data shape, should be used with extra care

split_as_dte(name='mydte')

Convenience method to split each ndarray into a DataWithAxes object

Return type:

DataToExport

stack_as_array(axis=0, dtype=None)

Stack all data arrays in a single numpy array

Parameters:

• axis (int) – The new stack axis index, default 0

• dtype (str or np.dtype) – the dtype of the stacked array

Return type:

ndarray

See also

np.stack()

to_dB()

Get a new data object in decibels

new in 4.3.0

Return type:

DataBase

to_dict()

Get the data arrays into dictionary whose keys are the labels

units_as(units, inplace=True, context=None, **context_kwargs)

Set the object units to the new one (if possible)

Parameters:

• units (str) – The new unit to convert the data to

• inplace (bool) – default True. If True replace the data’s arrays by array in the new units If False, return a new data object

• context (Optional[str]) – See pint documentation

Return type:

DataBase

unwrap()

unwrap the underlying array (should be angles otherwise meaningless)

value(units=None)

Returns the underlying float value (of the first elt in the data list) if this data holds only a float otherwise returns a mean of the underlying data

Parameters:

units (Optional[str]) – if unit is compatible with self.units, convert the data to these new units before getting the value

Return type:

float

values(units=None)

Returns the underlying float value (for each data array in the data list) if this data holds only a float otherwise returns a mean of the underlying data

Return type:

List[float]

property averaged: bool

Get/Set a boolean depending if self is the result of an average operation

See also

n_averaged, average

property data: List[ndarray]

get/set (and check) the data the object is storing

Type:

List[np.ndarray]

Setter accepts:

Optional[List[Any]]

property dim

the enum representing the dimensionality of the stored data

Type:

DataDim

property distribution

the enum representing the distribution of the stored data

Type:

DataDistribution

property length

The length of data. This is the length of the list containing the nd-arrays

property n_averaged: int

Get/set the number of averaging that resulted in this data

See also

averaged, average

property quantities: list[Quantity]

Get the arrays as pint quantities (with units)

property shape

The shape of the nd-arrays

property size

The size of the nd-arrays

property source

the enum representing the source of the data

Type:

DataSource

Setter accepts:

Union[str, DataSource]

class pymodaq_data.data.DataCalculated(name, dim=None, distribution=DataDistribution.uniform, data=None, labels=None, origin='', units='', axes=[], nav_indexes=(), errors=None, **kwargs)

Specialized DataWithAxes set with source as ‘calculated’. To be used for processed/calculated data

class pymodaq_data.data.DataFromRoi(name, dim=None, distribution=DataDistribution.uniform, data=None, labels=None, origin='', units='', axes=[], nav_indexes=(), errors=None, **kwargs)

Specialized DataWithAxes set with source as ‘calculated’. To be used for processed data from region of interest

class pymodaq_data.data.DataRaw(name, dim=None, distribution=DataDistribution.uniform, data=None, labels=None, origin='', units='', axes=[], nav_indexes=(), errors=None, **kwargs)

Specialized DataWithAxes set with source as ‘raw’. To be used for raw data

class pymodaq_data.data.DataWithAxes(*args, **kwargs)

Data object with Axis objects corresponding to underlying data nd-arrays

Parameters:

• axes (list of Axis) – the list of Axis object for proper plotting, calibration …

• nav_indexes (tuple of int) – highlight which Axis in axes is Signal or Navigation axis depending on the content: For instance, nav_indexes = (2,), means that the axis with index 2 in a at least 3D ndarray data is the first navigation axis For instance, nav_indexes = (3,2), means that the axis with index 3 in a at least 4D ndarray data is the first navigation axis while the axis with index 2 is the second navigation Axis. Axes with index 0 and 1 are signal axes of 2D ndarray data

• errors (list of ndarray.) – The list should match the length of the data attribute while the ndarrays should match the data ndarray

classmethod deserialize(bytes_str)

Convert bytes into a DataWithAxes object

Convert the first bytes into a DataWithAxes reading first information about the underlying data

Return type:

Tuple[DataWithAxes, bytes]

Returns:

• DataWithAxes (the decoded DataWithAxes)

• bytes (the remaining bytes string if any)

classmethod from_xarray(ds)

Construct a DataWithAxes from an xarray Dataset (or DataArray).

Parameters:

ds (xr.Dataset or xr.DataArray)

Return type:

DataWithAxes

Raises:

ImportError – If xarray is not installed.

static serialize(dwa)

Convert a DataWithAxes into a bytes string

Parameters:

dwa (DataWithAxes)

Returns:

bytes

Return type:

the total bytes message to serialize the DataWithAxes

Notes

The bytes sequence is constructed as:

• serialize the timestamp: float

• serialize the name

• serialize the source enum as a string

• serialize the dim enum as a string

• serialize the distribution enum as a string

• serialize the list of numpy arrays

• serialize the list of labels

• serialize the origin

• serialize the nav_index tuple as a list of int

• serialize the list of axis

• serialize the errors attributes (None or list(np.ndarray))

• serialize the list of names of extra attributes

• serialize the extra attributes

accumulate(other, weight)

Compute the weighted average as average() does, and set the standard error of the mean as the errors of the result

It is meant to be used to average step by step N acquisitions, self being the new acquisition and other the average of the weight previous ones, as returned by this method (the first acquisition is just a copy of itself). The standard error of the mean is updated from the one of other (Welford’s algorithm), so no more than the averaged data is needed to follow the statistics.

Parameters:

• other (DataWithAxes) – The average of the weight previous acquisitions. If weight is larger than 1, it should carry the errors set by the previous call, else the result has no errors. The errors of the first acquisition (weight equal to 1) are not taken into account.

• weight (int) – The number of acquisitions averaged in other

Returns:

DataWithAxes

Return type:

DataWithAxes

See also

average

Notes

The update of the sum of squared deviations from the mean is Welford’s online algorithm [1] [2], with S_N = S_(N-1) + (x_N - mean_(N-1)) * (x_N - mean_N) (complex conjugate on the second factor for complex data). The errors are then sqrt(S_N / (N (N - 1))), i.e. the sample standard deviation (ddof=1) divided by sqrt(N).

self is taken as a single acquisition without errors (weight one). To combine two measurements that each carry an error (inverse-variance weighting), see combine_with_errors().

References

[1]

B. P. Welford, “Note on a method for calculating corrected sums of squares and products”, Technometrics 4 (1962) 419-420.

[2]

4. 5. Knuth, The Art of Computer Programming, vol. 2, section 4.2.2.

axes_limits(axes_indexes=None)

Get the limits of specified axes (all if axes_indexes is None)

Return type:

List[Tuple[float, float]]

check_axes_linear(axes=None)

Check if any axis may be non linear

Should trigger a spread like distribution except id dim is Data1D, in which cas, it doesn’t matter

Return type:

bool

combine_with_errors(other)

Combine two measurements of the same quantity, each carrying its own errors, with an inverse-variance weighted mean

With w_i = 1 / e_i**2, the result is, for each point of each array:

mean = (w_1 * x_1 + w_2 * x_2) / (w_1 + w_2) = (x_1 * e_2**2 + x_2 * e_1**2) / (e_1**2 + e_2**2)
error = 1 / sqrt(w_1 + w_2) = e_1 * e_2 / sqrt(e_1**2 + e_2**2)

Use accumulate() instead to average successive acquisitions into a running mean and its standard error.

Parameters:

other (DataWithAxes) – The other measurement, with the same shape, units and number of arrays as self

Returns:

DataWithAxes

Return type:

DataWithAxes

Raises:

• ValueError – If one of the two objects has no errors or if some of them are not strictly positive

• DataUnitError – If the units of self and other differ

• DataShapeError – If the arrays of self and other have different shapes

Notes

The errors are taken as the standard deviations of independent, normally distributed measurements. The weights are the inverse of their variances, the minimum-variance unbiased estimator [1] [2].

References

[1]

P. R. Bevington and D. K. Robinson, Data Reduction and Error Analysis for the Physical Sciences, McGraw-Hill, section on the weighted average.

[2]

J. R. Taylor, An Introduction to Error Analysis, University Science Books, section on the weighted average.

create_missing_axes()

Check if given the data shape, some axes are missing to properly define the data (especially for plotting)

deepcopy_with_new_data(data=None, remove_axes_index=None, source=DataSource.calculated, keep_dim=False, errors=None)

deepcopy without copying the initial data (saving memory)

The new data, may have some axes stripped as specified in remove_axes_index

Parameters:

• data (List[ndarray]) – The new data

• remove_axes_index (Union[int, integer, List[Union[int, integer]], Tuple[Union[int, integer], ...], None]) – indexes of the axis to be removed

• source (Union[DataSource, str])

• keep_dim (bool) – if False (the default) will calculate the new dim based on the data shape else keep the same (be aware it could lead to issues)

• errors (Optional[List[ndarray]]) – The new errors corresponding to the new data

Return type:

DataWithAxes

errors_as_dwa()

Get a dwa from self replacing the data content with the error attribute (if not None)

New in 4.2.0

find_peaks(height=None, threshold=None, **kwargs)

Apply the scipy find_peaks method to 1D data

Parameters:

• height (number or ndarray or sequence, optional)

• threshold (number or ndarray or sequence, optional)

• kwargs (dict) – extra named parameters applied to the find_peaks scipy method

Return type:

DataToExport

See also

find_peaks()

fit(function, initial_guess, data_index=None, axis_index=0, **kwargs)

Apply 1D curve fitting using the scipy optimization package

Parameters:

• function (Callable) – a callable to be used for the fit

• initial_guess (Iterable) – The initial parameters for the fit

• data_index (Optional[int]) – The index of the data over which to do the fit, if None apply the fit to all

• axis_index (int) – the axis index to use for the fit (if multiple) but there should be only one

• kwargs (dict) – extra named parameters applied to the curve_fit scipy method

Return type:

DataCalculated

See also

curve_fit()

ft(axis=0, axis_label=None, axis_units=None, labels=None)

Process the Fourier Transform of the data on the specified axis and returns the new data

Parameters:

• axis (int) – Apply the FT on this axis index

• axis_label (Optional[str]) – A new label for the FT computed axis

• axis_units (Optional[str]) – New units (without conversion on top of the one from the FT) for the computed axis

• labels (Optional[List[str]]) – list of string for new labels

Return type:

DataWithAxes

See also

ft(), fft()

get_axes_sizes()

Get the width (as a pint Quantity) of each axis, creating the missing axes if needed

Return type:

Iterable[Quantity]

get_axis_from_label(label)

Get the axis referred by a given label

Parameters:

label (str) – The label of the axis

Returns:

Axis or None

Return type:

Optional[Axis]

get_axis_indexes()

Get all present different axis indexes

Return type:

List[int]

get_data_as_dwa(index=0)

Get the underlying data selected from the list at index, returned as a DataWithAxes

Return type:

DataWithAxes

get_data_dimension()

Get the string describing the dimensions of the data, e.g. (nav0, nav1|sig0)

Return type:

str

get_dim_from_data_axes()

Get the dimensionality DataDim from data taking into account nav indexes

Return type:

DataDim

get_error(index)

Get a particular error ndarray at the given index in the list

new in 4.2.0

get_nav_axes_with_data()

Get the data’s navigation axes making sure there is data in the data field

Return type:

List[Axis]

ift(axis=0, axis_label=None, axis_units=None, labels=None)

Process the inverse Fourier Transform of the data on the specified axis and returns the new data

Parameters:

• axis (int)

• axis_label (Optional[str]) – A new label for the FT computed axis

• axis_units (Optional[str]) – New units (without conversion on top of the one from the FT) for the computed axis

• labels (Optional[List[str]]) – list of string for new labels

Return type:

DataWithAxes

See also

ift(), ifft()

interp(new_axis_data, **kwargs)

Performs linear interpolation for 1D data only.

For more complex ones, see scipy.interpolate()

Parameters:

• new_axis_data (Union[Axis, ndarray]) – The coordinates over which to do the interpolation

• kwargs (dict) – extra named parameters to be passed to the interp() method

Return type:

DataWithAxes

See also

interp(), interpolate()

mean(axis=0)

Process the mean of the data on the specified axis and returns the new data

Parameters:

axis (int)

Return type:

DataWithAxes

mean_with_errors(axis=0)

Process the mean of the data on the specified axis, as mean() does, and set the standard error of the mean as the errors of the result

It is meant for a buffer that gives a series of N acquisitions at once (as the first axis of the data, a navigation axis for instance), when accumulate() which averages them one after the other is not needed. The errors are the sample standard deviation (ddof=1) divided by sqrt(N), as accumulate() gives.

Parameters:

axis (int) – The axis of the acquisitions

Returns:

DataWithAxes

Return type:

DataWithAxes

See also

mean, accumulate

moment()

returns the two first moments of the data over the axis

only valid for Data1D data

Return type:

Tuple[DataWithAxes, DataWithAxes]

Returns:

• DataCalculated (containing the moment of order 0 (mean))

• DataCalculated (containing the moment of order 1 (std))

pad(pad_width, **kwargs)

Pad the data arrays using the numpy pad function

The accepted pad_witdh type is the same than the numpy pad function

see numpy.pad method for the signature and possible named arguments

plot(plotter_backend='matplotlib', *args, viewer=None, **kwargs)

Call a plotter factory and its plot method over the actual data

rot90(k=1, axes=(0, 1))

Rotate an array by 90 degrees in the plane specified by axes.

Valid only for 2D data

set_axes_manager(data_shape, axes, nav_indexes, **kwargs)

Create the axes manager (uniform or spread, depending on the distribution) of this object

sort_data(axis_index=0, spread_index=0, inplace=False)

Sort data along a given axis, default is 0

Parameters:

• axis_index (int) – The index along which one should sort the data

• spread_index (int) – for spread data only, specifies which spread axis to use

• inplace (bool) – modify in place or not the data (and its axes)

Return type:

DataWithAxes

sum(axis=0)

Process the sum of the data on the specified axis and returns the new data

Parameters:

axis (int)

Return type:

DataWithAxes

to_xarray()

Convert this DataWithAxes to an xarray.Dataset.

Each array in self.data becomes a data variable (keyed by its label). Each Axis becomes a coordinate on the corresponding dimension. Error arrays (if present) are stored as <label>_error data variables.

Return type:

xr.Dataset

Raises:

ImportError – If xarray is not installed.

transpose()

replace the data by their transposed version

Valid only for 2D data

property axes

convenience property to fetch attribute from axis_manager

Setter accepts:

List[Axis]

property errors: Sequence[ndarray] | None

Get/Set the errors bar values as a list of np.ndarray

new in 4.2.0

Setter accepts:

Optional[Iterable[ndarray]]

property n_axes

Get the number of axes (even if not specified)

property nav_indexes

convenience property to fetch attribute from axis_manager

Setter accepts:

Iterable[Union[int, integer]]

property sig_indexes

convenience property to fetch attribute from axis_manager

class pymodaq.utils.data.DataActuator(*args, **kwargs)

Specialized DataWithAxes set with source as ‘raw’. To be used for raw data generated by actuator plugins

class pymodaq.utils.data.DataFromPlugins(*args, **kwargs)

Specialized DataWithAxes set with source as ‘raw’. To be used for raw data generated by Detector plugins

It introduces by default to extra attributes, do_plot and do_save. Their presence can be checked in the extra_attributes list.

Parameters:

• do_plot (bool) – If True the underlying data will be plotted in the DAQViewer

• do_save (bool) – If True the underlying data will be saved

Variables:

• do_plot (bool) – If True the underlying data will be plotted in the DAQViewer

• do_save (bool) – If True the underlying data will be saved

8.4.6.3. Data Characteristics

Created the 28/10/2022

@author: Sebastien Weber

class pymodaq_data.data.DataDim(*values)

Enum for dimensionality representation of data

static from_data_array(data_array)

Get the DataDim corresponding to the shape of a numpy array

A 1D array of size 1 is Data0D, other 1D arrays are Data1D, 2D arrays are Data2D and any array with more dimensions is DataND.

property dim_index

the integer value of this dimension (0 for Data0D, 1 for Data1D, …)

Type:

int

class pymodaq_data.data.DataDistribution(*values)

Enum for distribution of data

class pymodaq_data.data.DataSource(*values)

Enum for source of data

8.4.6.4. Union of Data

When exporting multiple set of Data objects, one should use a DataToExport

Created the 28/10/2022

@author: Sebastien Weber

class pymodaq_data.data.DataToExport(*args, **kwargs)

Object to store all raw and calculated DataWithAxes data for later exporting, saving, sending signal…

Includes methods to retrieve data from dim, source… Stored data have a unique identifier their name. If some data is appended with an existing name, it will replace the existing data. So if you want to append data that has the same name

Parameters:

• name (str) – The identifier of the exporting object

• data (list of DataWithAxes) – All the raw and calculated data to be exported

Variables:

• name

• timestamp

• data

classmethod deserialize(bytes_str)

Convert bytes into a DataToExport object

Convert the first bytes into a DataToExport reading first information about the underlying data

Return type:

Tuple[DataToExport, bytes]

Returns:

• DataToExport (the decoded DataToExport)

• bytes (the remaining bytes if any)

classmethod from_xarray(dt, name=None)

Construct a DataToExport from an xarray.DataTree or a dict of Datasets.

Parameters:

• dt (xr.DataTree or dict[str, xr.Dataset])

• name (Optional[str]) – Override the name; if None, read from dt.attrs['pymodaq_name'].

Return type:

DataToExport

Raises:

ImportError – If xarray is not installed.

static serialize(dte)

Convert a DataToExport into a bytes string

Parameters:

dte (DataToExport)

Returns:

bytes

Return type:

the total bytes message to serialize the DataToExport

Notes

The bytes sequence is constructed as:

• serialize the string type: ‘DataToExport’

• serialize the timestamp: float

• serialize the name

• serialize the list of DataWithAxes

accumulate(other, weight)

Same as average() but the standard error of the mean is also set as the errors of each averaged DataWithAxes, see DataWithAxes.accumulate()

Parameters:

• other (DataToExport) – The average of the weight previous acquisitions

• weight (int) – The weight the ‘other_data’ holds with respect to self

Return type:

DataToExport

affect_name_to_origin_if_none()

Affect self.name to all DataWithAxes children’s attribute origin if this origin is not defined

average(other, weight)

Compute the weighted average between self and other DataToExport and attributes it to self

Parameters:

• other (DataToExport)

• weight (int) – The weight the ‘other_data’ holds with respect to self

Return type:

DataToExport

deepcopy()

Return a deep copy of this object, named ‘Copy’, with deep copies of its data

get_data_from_Naxes(Naxes, deepcopy=False)

Get the data matching the given number of axes

Parameters:

Naxes (int) – Number of axes in the DataWithAxes objects

Returns:

DataToExport

Return type:

DataToExport

get_data_from_attribute(attribute, attribute_value, deepcopy=False, sort_by_name=False)

Get the data matching a given attribute value

Parameters:

• attribute (str) – The name of the attribute to sort data with

• attribute_value (Any) – The value of the attribute

• deepcopy (bool) – If True, the returned data are deepcopied from the original

• sort_by_name (bool) – If True the returned data are sorted alphabetically using their name, default is False

Returns:

DataToExport

Return type:

DataToExport

get_data_from_dim(dim, deepcopy=False, sort_by_name=False)

Get the data matching the given DataDim

Returns:

DataToExport

Return type:

DataToExport

get_data_from_dims(dims, deepcopy=False, sort_by_name=False)

Get the data matching the given DataDim

Returns:

DataToExport

Return type:

DataToExport

get_data_from_full_name(full_name, deepcopy=False)

Get the DataWithAxes with matching full name

Return type:

DataWithAxes

get_data_from_full_names(full_names, deepcopy=False)

Get the data matching the given full names (see get_data_from_full_name()) as a new DataToExport

Return type:

DataToExport

get_data_from_missing_attribute(attribute, deepcopy=False)

Get the data matching a given attribute value

Parameters:

• attribute (str) – a string of a possible attribute

• deepcopy (bool) – if True the returned DataToExport will contain deepcopies of the DataWithAxes

Returns:

DataToExport

Return type:

DataToExport

get_data_from_name(name)

Get the data matching the given name

Return type:

DataWithAxes

get_data_from_name_origin(name, origin='')

Get the data matching the given name and the given origin

Return type:

Optional[DataWithAxes]

get_data_from_names(names)

Get the data whose name is in names as a new DataToExport

Return type:

DataToExport

get_data_from_origins(origins)

Get the data whose origin is in origins as a new DataToExport

Return type:

DataToExport

get_data_from_sig_axes(Naxes, deepcopy=False)

Get the data matching the given number of signal axes

Parameters:

Naxes (int) – Number of signal axes in the DataWithAxes objects

Returns:

DataToExport

Return type:

DataToExport

get_data_from_source(source, deepcopy=False, sort_by_name=False)

Get the data matching the given DataSource

Returns:

DataToExport

Return type:

DataToExport

get_data_with_naxes_lower_than(n_axes=2, deepcopy=False)

Get the data with n axes lower than the given number

Parameters:

Naxes (int) – Number of axes in the DataWithAxes objects

Returns:

DataToExport

Return type:

DataToExport

get_dim_presents()

Get the names of the dimensions (Data0D, Data1D, …) for which at least one data is present

Return type:

List[str]

get_full_names(dim=None)

Get the ful names including the origin attribute into the returned value, eventually filtered by dim

Parameters:

dim (Union[DataDim, str, None])

Returns:

list of str

Return type:

the names of the (filtered) DataWithAxes data constructed as : origin/name

Examples

d0 = DataWithAxes(name=’datafromdet0’, origin=’det0’)

get_names(dim=None)

Get the names of the stored DataWithAxes, eventually filtered by dim

Parameters:

dim (Union[DataDim, str, None])

Returns:

list of str

Return type:

List[str]

get_origins(dim=None)

Get the origins of the underlying data into the returned value, eventually filtered by dim

Parameters:

dim (Union[DataDim, str, None])

Returns:

list of str

Return type:

the origins of the (filtered) DataWithAxes data

Examples

d0 = DataWithAxes(name=’datafromdet0’, origin=’det0’)

index(data)

Here use a comparison to assert data is equal to one element in the list

But the __eq__ method is not checking the name while it is the main issue for elt finding Hence here I’m doing both checks

index_from_name_origin(name, origin='')

Get the index of a given DataWithAxes within the list of data

Return type:

int

mean_with_errors(axis=0)

Apply DataWithAxes.mean_with_errors() to each DataWithAxes: the mean on axis of a buffer of acquisitions given at once, with the standard error of the mean as errors

The acquisitions have to be stacked on the same axis in all the DataWithAxes.

Parameters:

axis (int) – The axis of the acquisitions

Return type:

DataToExport

merge_as_dwa(dim, name=None)

attempt to merge filtered dwa into one

Only possible if all filtered dwa and underlying data have same shape

Parameters:

• dim (Union[str, DataDim]) – will only try to merge dwa having this dimensionality

• name (Optional[str]) – The new name of the returned dwa

Return type:

Optional[DataWithAxes]

plot(plotter_backend='matplotlib', *args, **kwargs)

Call a plotter factory and its plot method over the actual data

pop(index)

return and remove the DataWithAxes referred by its index

Parameters:

index (int) – index as returned by self.index_from_name_origin

Return type:

DataWithAxes

See also

index_from_name_origin

remove(dwa)

Use the DataWithAxes object comparison __eq__ to retrieve the elt to remove

Parameters:

dwa (DataWithAxes) – The da to remove from the list

to_xarray()

Convert this DataToExport to an xarray.DataTree.

The root node carries pymodaq_name in its attrs. Each DataWithAxes becomes a child node whose dataset is produced by DataWithAxes.to_xarray().

Return type:

xr.DataTree

Raises:

ImportError – If xarray is not installed.

property data: List[DataWithAxes]

get the data contained in the object

Type:

List[DataWithAxes]