6.2. CustomApp and CustomExt

6.2.1. The big picture

PyMoDAQ is split in four packages, each building on the previous one: pymodaq_utils (helpers, no GUI), pymodaq_data (data objects and h5 saving), pymodaq_gui (widgets, CustomApp, the managers) and pymodaq (the instrument layer: control modules, the Dashboard and the extensions). What you build depends on whether your interface needs instruments, and where it is meant to live:

The control modules (DAQ_Move for actuators, DAQ_Viewer for detectors) are the objects that talk to the instruments, through the instrument plugins, each plugin running in its own thread.
The Dashboard creates and owns the control modules of an experiment, and lists the installed extensions in its menu. The extensions that ship with PyMoDAQ (DAQ Scan, DAQ_Logger, PID, optimizers, data mixer, sequencer, ramping…) and the ones installed from plugin packages (see Extension Plugins) are all built the same way.
An application is a program that you start yourself, independent of the Dashboard. It is the right choice only when no instrument is involved.

CustomApp is the common base of the Dashboard, of the extensions and of the applications: it provides the user-interface skeleton (docks, menus, toolbars, actions, settings tree, status bar, h5 file handling, worker threads). CustomExt is the specialisation for what runs inside a Dashboard, because it needs to use the instruments the Dashboard already controls. So the rule is:

You want to build	Derive from	Instruments come from	Started by
An interface that needs no instrument (data viewer, analysis tool, h5 browser)	`CustomApp`	none	your script
A tool using the instruments of the current experiment (scan, logging, feedback, optimization)	`CustomExt`	the Dashboard, through `self.modules_manager`	the Dashboard Extensions menu, or your script through `create_extension`

Fig. 6.1 Where CustomApp and CustomExt sit among PyMoDAQ’s objects

Text version of the diagram:

CustomApp (in pymodaq_gui) is extended by DashBoard, by CustomExt and by your standalone applications.
CustomExt (in pymodaq) is extended by every Dashboard extension, those shipped with PyMoDAQ as well as those in plugin packages.
The DashBoard creates the control modules and loads the extensions declared as entry points.
An extension uses the control modules of the Dashboard through its ModulesManager; it does not create them.
The control modules load the instrument plugins, which drive the hardware.

Note

A generic base class CustomApp located in pymodaq_gui.utils.custom_app can be used to build very quickly standalone Applications. CustomExt (pymodaq.utils.custom_ext) extends CustomApp with what deals with instruments and the Dashboard, and is the base class of any Dashboard extension (the DAQ_Logger extension has been built using it, as well as some examples in the example folder). See Structure of the classes.

6.2.2. Writing a CustomApp or a CustomExt

Below you’ll find the skeleton of a CustomApp (a CustomExt is written the same way) subclassing the base class and methods you have to override with your App/Extension specifics:

class CustomAppExample(gutils.CustomApp):

    # list of dicts enabling a settings tree on the user interface
    params = [
        {'title': 'Main settings:', 'name': 'main_settings', 'type': 'group', 'children': [
            {'title': 'Save base path:', 'name': 'base_path', 'type': 'browsepath',
             'value': config['data_saving']['h5file']['save_path']},
            {'title': 'File name:', 'name': 'target_filename', 'type': 'str', 'value': "", 'readonly': True},
            {'title': 'Date:', 'name': 'date', 'type': 'date', 'value': QDate.currentDate()},
            {'title': 'Do something, such as showing data:', 'name': 'do_something', 'type': 'bool', 'value': False},
        ]},
    ]

    def __init__(self, dockarea, dashboard=None):
        super().__init__(dockarea)
        # init the App specific attributes
        self.raw_data = []
        self.setup_ui()  # calls the methods below, in the order given in the lifecycle table

    def setup_docks_and_widgets(self):
        '''
        subclass method from CustomApp
        '''
        logger.debug('setting docks')
        self.dock_settings = gutils.Dock('Settings', size=(350, 350))
        self.dockarea.addDock(self.dock_settings, 'left')
        self.dock_settings.addWidget(self.settings_tree, 10)
        logger.debug('docks are set')

    def setup_menus_and_toolbars(self, menubar: QtWidgets.QMenuBar = None):
        '''
        subclass method from CustomApp
        '''
        logger.debug('settings menu')
        file_menu = self.add_menu('file', 'File')
        file_menu.addSeparator()
        logger.debug('menu set')

    def setup_actions(self):
        '''
        subclass method from ActionManager
        '''
        logger.debug('setting actions')
        self.add_action('quit', 'Quit', 'close2', "Quit program",
            toolbar=self.toolbar, menu='file')
        self.add_action('grab', 'Grab', 'camera', "Grab from camera", checkable=True, toolbar=self.toolbar)
        logger.debug('actions set')

    def connect_things(self):
        '''
        subclass method from CustomApp
        '''
        logger.debug('connecting things')
        self.connect_action('quit', self.quit_fun)
        self.connect_action('grab', self.grab)
        logger.debug('connecting done')

    def value_changed(self, param):
        logger.debug(f'calling value_changed with param {param.name()}')
        if param.name() == 'do_something':
            if param.value():
                self.settings.child('main_settings', 'something_done').setValue(True)
            else:
                self.settings.child('main_settings', 'something_done').setValue(False)

        logger.debug(f'Value change applied')

    def grab(self, grab: bool):
        ...  # all other methods required by your Application class

In a few lines of codes, you’ll get an application running. For the available Parameter available for your settings_tree, see Settings.

6.2.3. Structure of the classes

CustomApp is a thin class that assembles three building blocks. CustomExt adds what is needed to live inside a Dashboard. Fig. 6.2 shows who brings what.

Fig. 6.2 Class structure of CustomApp and CustomExt

Text version of the diagram:

CustomApp(QObject, ActionManager, ParameterManager) lives in pymodaq_gui.utils.custom_app.
ActionManager brings the actions, and their placement in the toolbars and menus: add_action, add_menu, add_toolbar, connect_action, set_action_enabled, set_action_checked.
ParameterManager brings the settings tree: the class attribute params, the settings Parameter, the settings_tree widget and the callbacks value_changed, child_added and param_deleted.
CustomApp itself brings the dock area (dockarea, docks), the optional mainwindow, the menu and status bars, the h5_manager and the thread_manager, and the setup_ui sequence.
DashboardMixin lives in pymodaq.utils.dashboard_mixin and holds everything dealing with the Dashboard, see DashboardMixin.
CustomExt(DashboardMixin, CustomApp) lives in pymodaq.utils.custom_ext. It takes the dashboard as second argument and sets it up with the mixin.

The parent given to the constructor can be a DockArea, a QMainWindow or a QWidget: this decides whether dockarea and mainwindow are created by the app or reused from the parent.

6.2.3.1. DashboardMixin

DashboardMixin (pymodaq.utils.dashboard_mixin) gathers what an object needs to work with a DashBoard. It is shared by the extensions (CustomExt) and by the managers that act on the Dashboard (ExperimentManager, StateManager, ROIManager, ExtensionManager, Overshooter, ScanManager). It lives in pymodaq because the lower packages (pymodaq_gui, with CustomApp and ManagerBase) cannot import pymodaq: they know nothing about the Dashboard.

dashboard: the DashBoard, or None if the object runs without it. is_dashboard_available tells which.
modules_manager: a ModulesManager created for the object from the detectors and actuators of the Dashboard (None without Dashboard). Its class can be changed with the module_manager_class argument.
experiment_manager, state_manager: the managers of the Dashboard (None without Dashboard).
do_things_after_experiment_set(experiment_name, show_dashboard=None): called each time an experiment is applied, see below. The base implementation refreshes modules_manager.
create_dashboard_toolbar and show_dashboard, see below.

The mixin has no __init__ on purpose: the Qt side of the object (QObject.__init__…) has to be initialized first, and the class using the mixin then calls init_dashboard(dashboard, module_manager_class) and connect_experiment_manager(). CustomExt does so right after CustomApp.__init__, and the managers through two hooks of ManagerBase (init_extra before the user interface is set up, so dashboard is available in setup_ui, and finish_init once the manager is built). A new object working with the Dashboard is therefore written class MyThing(DashboardMixin, CustomApp) (the mixin first) and does not need anything else.

A manager decides itself if its entries can be executed, by reimplementing can_execute_entry (False in ManagerBase). The managers working on the Dashboard return self.is_dashboard_available.

6.2.4. Lifecycle

setup_ui calls, in this order, the methods you reimplement:

Step	Method	What to put in it
1	`__init__`	Call `super().__init__` (creates the settings tree, `H5Manager` and `WorkerThreadManager`), set your own attributes, then call `self.setup_ui()`.
2	`setup_docks_and_widgets`	Create the docks and widgets (the settings tree is `self.settings_tree`).
3	`setup_menus_and_toolbars`	Create menus and toolbars with `add_menu` and `add_toolbar`.
4	`setup_workflow_actions`	Automatic if `show_workflow_actions` is True: start, stop, pause and log actions (`WorkFlowActions`).
5	`setup_actions`	Create the actions with `add_action`.
6	`connect_things`	Connect actions and widget signals to your methods.
7	`do_things_after_ui_setup`	Optional last touches. The window is then resized to `_size_hint`.
run	`value_changed`, `child_added`, `param_deleted`	React to the user’s changes in the settings tree.
end	`quit_fun` (`CustomExt`: `_quit_fun`)	Clean up. `CustomApp.quit_fun` exits the worker threads, closes the main window and disconnects the tree. In a `CustomExt`, return False from `_quit_fun` to refuse quitting.

6.2.5. Supporting objects: threads, files and signals

Three objects are created by CustomApp or by the worker pattern around it.

H5Manager (pymodaq_gui.managers.h5manager): owns the H5Saver, the file actions (FileAction: new file, load, save, show, open, close, settings) and the file status LED. FileStatus tells what happened when a file was opened (new, reopened, closed, another file reopened, no file). Signals: file_open_signal(bool), file_loaded_signal(Path) and command_sig(ThreadCommand). Reach it with self.h5_manager or self.h5saver; hide some file actions with the h5_actions_not constructor argument.
WorkerThreadManager (pymodaq_gui.managers.runner_thread_manager): self.thread_manager. It creates a QThread for a ThreadWorker, moves the worker into it (create_thread_for_worker), starts it (start_thread) and stops it (exit_worker_thread). Always talk to a worker with signals, a direct method call would run in the calling thread.
ExtensionWorker (pymodaq_gui.utils.app_worker): the object that does the work of an extension while the user interface stays responsive (used by DAQ_Scan, DAQ_Logger, the Sequencer and Ramping). It has its own WorkerThreadManager and creates a SaverWorker (and optionally a ProcessorWorker) in separate threads. The actions start, pause and stop of the app are connected to its start, pause and stop methods, and you implement _start, _pause and _stop.

Fig. 6.3 Threads and signals of an extension using an ExtensionWorker

Text version of the diagram:

The app (GUI thread) owns the H5Manager and, for the extensions that need it, an ExtensionWorker.
The user presses start: the app calls ExtensionWorker.start, which creates the SaverWorker (and the ProcessorWorker if has_data_processor) with create_thread_for_worker and starts their threads.
For each piece of data, the ExtensionWorker emits data_to_save_signal (a DataBundle) to the saver, which writes it in the h5 file and answers with n_jobs_done_signal(name, n_saved). The difference with the number of jobs sent is displayed as worker_tasks in the settings.
When the user presses stop, the emission towards the workers is disconnected, then once every queue is empty (_workers_done) the threads are exited, the file is closed through the H5Manager and workers_terminated is emitted.

6.2.6. The patterns

Two patterns cover almost every need: a standalone CustomApp when no instrument is involved, and a CustomExt run by the Dashboard as soon as instruments are. A third one, embedding control modules in a CustomApp, exists but is not advised (see its section).

6.2.6.1. Standalone app without instruments

Derive from CustomApp and give it a QMainWindow (or a DockArea placed in your own window). Nothing else is needed: no Dashboard, no ModulesManager. Examples are in pymodaq_gui/examples (for instance function_plotter.py and saving_loading.py).

Fig. 6.4 Standalone app: who creates and owns what

Text version: the script creates the window and the app; the app owns its H5Manager and WorkerThreadManager and fills the window. The app never calls the Dashboard. The lifecycle is the table above.

6.2.6.2. Extension of the Dashboard

Derive from CustomExt. The DashBoard instantiates it from the extension entry point (see Extension Plugins) with a DockArea and itself: MyExtension(dockarea, dashboard). The control modules already exist and are listed in self.modules_manager.

Fig. 6.5 Extension: who creates and owns what

Text version: the Dashboard owns the control modules and creates the extension. The extension creates a ModulesManager which only references the Dashboard’s modules; it is refreshed each time an experiment entry is applied (do_things_after_experiment_set). The extension owns its H5Manager and, if it acquires and saves data, an ExtensionWorker. Add create_dashboard_toolbar() in setup_menus_and_toolbars to show or hide the Dashboard from the extension. Implement _quit_fun rather than quit_fun; the Dashboard is shown again when the extension quits.

6.2.6.3. Experiment, State and the Dashboard toolbar in a CustomExt

The experiment (the set of actuators and detectors of the Dashboard, see Experiment manager) and its state (values of the instruments settings, see StateManager) belong to the Dashboard. A CustomExt does not create its own managers: it reaches the Dashboard’s ones, and can display their controls in its own window.

self.experiment_manager and self.state_manager are the ExperimentManager and StateManager of the Dashboard (None if the extension runs without Dashboard). Never create others, nor your own save/load of the instrument configurations.
self.modules_manager is created for the extension, and filled with the actuators and detectors of the Dashboard. It is refreshed each time an experiment is applied, see below.
Applying an experiment from the extension is the same thing as applying it from the Dashboard: the control modules are recreated by the Dashboard, then ExperimentManager.applied_entry (a Signal(str) carrying the experiment name) is emitted. Applying a state sets values in the existing control modules, with no change of the list of modules.

React to a new experiment by reimplementing do_things_after_experiment_set(experiment_name, show_dashboard). CustomExt.__init__ connects it to applied_entry and also calls it once at startup if an experiment is already applied (experiment_manager.entry_applied), so the extension is set up the same way whenever it is opened. The base implementation updates self.modules_manager and shows or hides the Dashboard window: call it with super() when you override it, then rebuild what depends on the instruments (lists of modules in your settings, signal connections to the control modules…), as these were destroyed with the previous experiment.

create_dashboard_toolbar creates a toolbar named dashboard in the main window of the extension (call it in setup_menus_and_toolbars or setup_actions, once the actions can be added). It is the same toolbar as the one described in The DashBoard toolbar in extensions, with three optional groups:

Argument	Adds	Provided by
`add_dashboard=True`	the Show/Hide Dashboard toggle (action `DashBoardToolbarActions.SHOW`, handled by `show_dashboard`)	`CustomExt`
`add_experiment=True`	a button opening the Experiment manager, a list of the experiments and a button applying the selected one	`self.experiment_manager.get_external_toolbar_menu`
`add_state=True`	the same three controls for the states of the current experiment	`self.state_manager.get_external_toolbar_menu`

add_break=True puts the toolbar on a new row. For instance self.create_dashboard_toolbar(add_state=False) keeps the experiment list but not the states, which is relevant for an extension that only needs to choose the instruments. The lists of experiments and states are kept in sync with the ones of the Dashboard, they all show the same current entry. A manager can also be used without its toolbar: get_external_toolbar_menu(toolbar, menu) fills any toolbar or menu you give, so you can place the controls where you want in your interface.

Fig. 6.6 Experiment and State managers shared between the Dashboard and an extension

Text version: the Dashboard owns one ExperimentManager and one StateManager. Each manager can display its controls (open the manager, list of entries, apply) in any toolbar through get_external_toolbar_menu: the Dashboard does it for its own toolbars, and create_dashboard_toolbar does it for the extension. Applying an experiment, from either toolbar, makes the Dashboard create the control modules, then applied_entry is emitted and the extension refreshes its modules_manager in do_things_after_experiment_set.

6.2.6.4. App embedding control modules directly (not advised)

An app can create its own DAQ_Move and DAQ_Viewer objects without any Dashboard, but you should not do it. Use a CustomExt run by the Dashboard instead: it is easier and better. The Dashboard already creates, initializes and saves the settings of the instruments (experiment and state managers), and the extension just gets them through self.modules_manager, with the toolbar and the file handling done for you. An app that embeds control modules has to redo all this by itself and loses it (no experiment or state, no shared ModulesManager).

If you still need it, create_load_daq_move and create_load_daq_viewer in pymodaq.utils.gui_utils.loader_utils show how to build a control module with its docks and toolbar. The app then owns the control modules and must quit them in its quit_fun before calling super().quit_fun().

Fig. 6.7 App embedding control modules (not advised): who creates and owns what

Text version: the app creates and owns the control modules and its H5Manager; each control module loads its plugin in its own thread, as in the Dashboard.