(to be included in the data management section)
2.7. Saving datas¶
Datas saved using PyMoDAQ, either the DAQ_Scan or the DAQ_Viewer modules, use a binary format known as hdf5. This format was originally developped to save big volume of datas from large instruments. Its structure is hierarchical (a bit as folder trees) and one can add metadata to all entries in the tree. For instance, the data type, shape but also some more complex info such as all the settings related to a module or a plugin. This gives a unique file containing both datas and metadata. For these data and metadata to be saved and interpreted correctly by the H5Browser, a utility module is used: H5Saver within the DAQ_Viewer module and the DAQ_Scan module. This should also be used when building a custom application.
2.7.1. From DAQ_Viewer¶
126.96.36.199. Single Datas¶
Datas saved directly from a DAQ_Viewer (for instance the one on Fig. 2.18) will be recorded in a h5file whose structure will be represented like Fig. 2.61 using PyMoDAQ’s h5 browser (see H5Browser). The various type of data exported by the viewer are saved in a dedicated entry (Data0D, …) and all channels are recorded within as subentries. Metadata from the detector settings, the measurement module and the activated ROIs are also recorded.
188.8.131.52. Continuous Saving¶
When the continuous saving parameter is set, new parameters are appearing on the DAQ_Viewer panel (see Fig. 2.62).
Base path: indicates where the data will be saved. If it doesn’t exist the module will try to create it
Base name: indicates the base name from which the save file will derive
Current Path: readonly, complete path of the saved file
Do Save: Initialize the file and logging can start. A new file is created if clicked again.
Compression options: data can be compressed before saving, using one of the proposed library and the given value of compression [0-9], see pytables documentation.
The saved file will follow this general structure:
With a base path (
D:\Data in this case) followed by a subfolder year, a subfolder day and a filename
formed from a base name followed by the date of the day and the time at which you started to log data.
Fig. 2.63 displays the tree structure of such a file, with
2.7.2. From DAQ_Scan¶
DAQ_Scan module will save your data in datasets. Each dataset is a unique h5 file and may contain multiple scans. The idea behind this is to have a unique file for a set of related data (the dataset) together with all the meta information: logger data, module parameters (settings, ROI…) even png screenshots of the various panels.
Fig. 2.64 displays the content of a typical dataset file containing various scans and how each data and metadata is used by the H5Browser to display the info to the user.
In order to save correctly your datas in custom applications, a utility module is to be used: pymodaq.utils.h5saver,
it will save scans and datas following the rules displayed on
2.7.3. Exploring datas¶
h5file saved using PyMoDAQ can be explored using the H5Browser utility module. Both datas and metadatas can be explored.