Data and analysis
Data streams, HDF5 acquisition, imaging and fitting.
Data path
Experiment runs publish typed data and image streams over ZeroMQ. Data clients subscribe to the channels they need; the Data Manager accumulates runs, writes HDF5 files and exposes live series to the GUI and web console. The message and property Hubs provide the same data to every client, so a plot, a notebook and a remote browser see one coherent experiment.
HDF5 structure
A run is archived with:
- the experiment and task identifiers;
- parameter values and scan definition;
- DAC/TTL settings and sequence metadata;
- scalar data series and image data;
- save-acknowledgement status and error information.
The file is self-describing: a run can be reconstructed and compared without the original GUI session.
Analysis modules
Quatm includes reusable analyses for common quantum-physics measurements:
- absorption imaging and optical density;
- ion / bright-spot detection;
- 1D and 2D Gaussian fits;
- skewed-Gaussian and multi-peak fits;
- ROI and line-profile extraction;
- averaging and histogram utilities.
Analysis can run from the Data Manager, from Python scripts, or on the analysis machine while acquisition continues. The same modules are used in the desktop GUI and in the web console.
Live monitoring
The Stream Monitor and image viewers follow data as it arrives. The history viewer, timeline and live plots are read-only clients: they cannot alter the experiment state. This makes them safe to open during operation and on remote machines.
Export
Data and fit results can be exported to CSV, images or publication-quality PDF. Export runs off the GUI thread, so large datasets do not freeze the interface.
1 - 2D Gaussian fit
Two-dimensional Gaussian fit within the ROI of each frame of an image stream: outputs the fit parameters and 1/e and 1/e² contour masks.
Overview
quatm/analysis/2D_gauss_fit.py provides 2D Gaussian fit analysis (TwoDGaussFit): it subscribes to an image stream and, on receiving an image, performs a 2D Gaussian fit inside the ROI (astropy Gaussian2D + Const2D, Levenberg-Marquardt iterations), publishes the fit parameters to the data stream and sends 1/e and 1/e² contour masks back to the image stream.
A fast variant, quatm/analysis/2D_gauss_fit_fast.py, is also available: the core fit is implemented in a Cython extension, and the interface and functionality are identical.
Output parameters: center x0 / y0, widths sigma_x / sigma_y, amplitude and background.
Usage
from quatm.analysis.2D_gauss_fit import TwoDGaussFit
fitter = TwoDGaussFit(name="gauss2d") # subscribe to the imagestreams image stream
fitter.run()
The analysis process is started from the Analysis Manager (AnalysisManager), where it is selected with a checkbox.
2 - 1D Gaussian fit
Fits a Gaussian to each frame of 1D data in a data stream and publishes the fit parameters and fitted curve.
Overview
quatm/analysis/gaussfit.py provides 1D Gaussian fit analysis (Gaussfit): as a standalone analysis process it subscribes to data streams, fits a Gaussian to each frame of 1D data and publishes the results to the data stream.
The fit function has the form:
f(x) = A0 / (sqrt(2*pi) * sigma) * exp(-(x - pos)^2 / (2*sigma^2)) + offset
The output parameters are A0 (area), sigma (width), pos (center), offset (background) and height (peak height = A0/(sqrt(2π)·σ)).
Configuration properties
| Property | Description |
|---|
datastreams | List of input data streams (each frame may be a 1D array, or a 2D array with two rows x/y) |
Usage
from quatm.analysis.gaussfit import Gaussfit
gf = Gaussfit(name="gauss") # name serves as the property/data-stream namespace
gf.run()
The analysis process is started from the Analysis Manager (AnalysisManager), where it is selected with a checkbox; input streams and parameters are configured in the property tree.
3 - Data management and analysis module
Data aggregation and storage management: DataSummary collects per-run data and writes HDF5, with a companion h5 management interface.
Overview
The data management and analysis module aggregates the data streams produced by an experiment into a complete record for each run and persists them. DataSummary in quatm/servers/datamgr.py is the aggregation core: it subscribes to a number of data channels plus experiment start/end events, organizes the data of each run into a complete dictionary that can be written to HDF5, read back and cleared. quatm/GUI/h5storage/ provides the data-management interface built on it (subscription, periodic saving and plotting).
Core components
| Component | Module | Description |
|---|
DataSummary | servers/datamgr.py | Aggregates multi-channel data streams into a complete dictionary for each run |
H5StorageDataMgr | GUI/h5storage/data_manager.py | Host for the aggregate/single-stream/image data managers |
H5StorageGui | GUI/h5storage/main_window.py | Data management main window (data/plot/event tabs) |
H5DataPlotter / EventPlotter | GUI/h5storage/plotters.py | Measurement data / event data plotting |
Key methods (DataSummary)
| Method | Description |
|---|
initDataq | (Re)subscribes to the channels configured in properties and to the experiment start/end events |
recvData / processIncomingData | Receives data and merges it into the most recent run record |
savetoFile | Writes the collected run dictionaries to a structured HDF5 dataset |
loadFromFile | Reads a set of stored run dictionaries back from HDF5 |
clear | Clears all collected data |
run_forever | Blocking loop that keeps receiving data |
Usage
from quatm.servers.datamgr import DataSummary
summ = DataSummary(name="sum")
summ.initDataq() # subscribe to the channels configured in the properties
summ.run_forever() # blocking acquisition; savetoFile then writes the run dict to the .h5
Input channels and the save path are configured in the property tree; for the GUI side see Data storage.
4 - Data stream subscription management
Main scrolling data-view window: ScrollPlot subscribes to data streams and scrolls them; PlotDataEditor selects the data source.
Overview
quatm/GUI/entries/scrollwindow.py provides data stream subscription management (ScrollWindow): a multi-channel scrolling data-view main window that lays out ScrollPlot subplots by count (two per row) and uniformly distributes data blocks to the corresponding subplots and refreshes them; PlotDataEditor provides drop-down selection of data/image streams.
Components
| Component | Description |
|---|
ScrollWindow | Main window: creates subplots by count, distributes data and refreshes |
ScrollPlot | Scrolling subplot: subscribes to data streams, ring buffer + periodic refresh |
PlotDataEditor | Data-source selection panel: drop-down candidates written back to the property ydata |
Key methods
| Method | Description |
|---|
initPlots / feedData / updatePlot (ScrollWindow) | Create subplots / feed data / batch refresh |
setnewData (ScrollPlot) | Periodically pulls data and refreshes the curve |
dataSelectDialog / subscribe_window / configureWindow (ScrollPlot) | Select channels / change subscription / configure this plot |
updateDatachannel (PlotDataEditor) | Writes the selected stream into the property ydata when the drop-down changes |
Usage
from quatm.GUI.entries.scrollwindow import ScrollWindow
win = ScrollWindow(name="PMT scroll monitor", parent=parent, n_plots=2)
Data-stream subscription, channel selection and configuration are performed on each subplot through its context menu; for a more complete description see Scrolling waveform window.
5 - Graphical ROI manager
Remotely controllable ROIs whose position, size and color are synchronized through the central property store.
Overview
quatm/GUI/widgets/ROI.py provides ROI: a subclass of ROI that communicates its color, position and size through the central property store under a unique name. It is suited to displaying and managing multiple regions of interest in a GUI and letting remote clients view and adjust them in sync.
Methods
| Method | Description |
|---|
update_from_properties | Refreshes the ROI pen color and geometry from the color/position/size in properties |
Usage
from quatm.GUI.widgets.ROI import ROI
roi = ROI(name="analysis_roi", parent=view) # name must be unique; duplicate names share properties
roi.update_from_properties()
For a more complete description, see Remote ROI.
6 - Live updating plot
Real-time plotting that subscribes to data streams and keeps updating: the UpdatingPlot and LivePlot containers.
Overview
quatm/GUI/entries/updating_plot.py provides the live updating plot: UpdatingPlot is a subclass of PlotItem that subscribes to data streams and periodically pulls new data to refresh the curve; LivePlot, by contrast, is a widget container that embeds an UpdatingPlot and can carry an image-stream name, which makes it convenient to embed as a dock panel in other windows.
Components
| Component | Description |
|---|
UpdatingPlot | Live plot item: subscribes to data streams and keeps updating the curve |
LivePlot | Widget container: embeds one UpdatingPlot and can carry an image stream |
Key methods (UpdatingPlot)
| Method | Description |
|---|
setnewData | Periodically pulls new data; for 1D arrays it automatically adds an x axis and then updates the curve |
updateConfiguration | Cancels the old subscriptions, resubscribes according to the stream list configured in properties and rebuilds the curves |
subscribe_window | Opens the subscription editor to select data streams and reapply the configuration |
configureWindow | Opens the property-edit dialog to adjust display settings |
Usage
from quatm.GUI.entries.updating_plot import UpdatingPlot, LivePlot
plot = UpdatingPlot(name="PMT signal") # used directly as a plot item
host = LivePlot(name="PMT signal", parent=parent) # or embedded as a container in the UI
For a more complete description, see Live plotting.
7 - Subscription editor
Manages the program’s subscriptions to data/image streams: display, add, delete and hot property updates.
Overview
quatm/GUI/editors/subscription_editor.py provides the subscription editor (SubscriptionEditor): it manages the program’s subscriptions to data or image streams. The interface shows the current subscription list, offers a drop-down to select a new stream and add it, and allows subscriptions to be deleted; changes are synchronized immediately to the property store and a signal is broadcast, so that plotting/monitoring components can update on the fly.
Key methods
| Method | Description |
|---|
loadSubscriptions | Reads the saved subscriptions from the property store and loads them into the list |
handle_property_changes | Listens for property changes: when this subscription key changes, refreshes the list and broadcasts |
update_streamlist | Rebuilds the drop-down candidates from the currently available data-stream list |
add | Adds the stream selected in the drop-down to the subscriptions and synchronizes properties |
deletemarked | Removes the subscriptions marked for deletion and synchronizes properties |
Usage
from quatm.GUI.editors.subscription_editor import SubscriptionEditor
editor = SubscriptionEditor(props=props, category="Data", parent=parent)
Constructor parameters: category is Image / Data / List; propprefix, streamkey and fullList (List mode) can be used to adjust the property keys and candidate streams. For a more complete description see Subscription editor.
8 - Table widget
Displays and edits parameter types holding NumPy record arrays in tabular form.
Overview
quatm/GUI/widgets/table_parameter.py provides the table widget (table_parameter): it extends the parameter tree so that parameters holding NumPy record arrays can be displayed and edited in tabular form, and can be registered as a reusable custom parameter type via registerParameterType.
Components
| Component | Description |
|---|
table_parameterItem | Table-widget parameter item for editing record arrays (a WidgetParameterItem subclass) |
table_parameter | Table parameter type (a Parameter subclass), used together with the item above |
Key methods
| Method | Description |
|---|
makeWidget (table_parameterItem) | Creates a QTableWidget editor from the recArray dtype |
value / setValue (table_parameterItem) | Reads back cell by cell from the table / writes the array cell by cell |
valueChanged (table_parameterItem) | Refreshes the table and label when the parameter value changes |
setValue / valueIsDefault (table_parameter) | Sets the parameter value / tests whether it is the default value |
Usage
from quatm.GUI.widgets.table_parameter import table_parameter
param = table_parameter(name="records", value=recarray) # insert into the parameter tree
For a more complete description see Table parameter.
9 - Data, image and camera editor
Display and editing of data-stream, image and camera properties: these capabilities are now provided by parameter boxes, the image monitor and the property editor.
Overview
An earlier unified interface provided a set of widgets, ParameterBox / ImageDataBox / CamPropsBox / DataBox, for displaying a single data-stream value, image data and camera properties as compact boxes, with right-click subscription and data selection. As the GUI was split up and evolved, these capabilities have been taken over by the current components:
| Legacy component | Function | Current implementation |
|---|
ParameterBox | Compact display of a single data-stream value | Parameter box widgets (widgets/arg_boxes etc.) |
ImageDataBox | Compact box for image data display | Image monitor (entries/image_monitor.ImageDisplay) |
CamPropsBox | Display and editing of camera properties | Property tree editor (PropEdit) + subscription |
DataBox | Simple label-value box | GUI base frames (BFrame / BWidget) |
Related pages
10 - Image monitor
Automatically updating image display window: subscribes to image/mask streams, with noise reduction, bright-spot detection and saving.
Overview
quatm/GUI/entries/image_monitor.py provides the image monitor (ImageDisplay): an automatically updating image display window that periodically pulls the latest frame from the image streams subscribed in properties, transforms it according to resolution and offset, refreshes the display and synchronously overlays mask streams. It has built-in noise reduction, bright-spot detection, saving of the current frame and property configuration (display levels, ROI, etc.).
Methods
| Method | Description |
|---|
update_image / update_mask | Periodically pulls the image/mask stream and refreshes the display and overlay |
noise_filter | Bilateral filtering + median filtering noise reduction using property parameters |
spot_detector | Binarizes the image by threshold to detect bright spots |
saveCurrentImage | Saves the currently displayed frame as PNG (named by time) |
subscribe_window / subscribe_mask | Opens the subscription dialog to select image/mask streams and resubscribes |
configureWindow | Opens the property-edit dialog to adjust display levels, ROI, etc. |
Usage
from quatm.GUI.entries.image_monitor import ImageDisplay
view = ImageDisplay(imagestreams=["camera/image"], parent=parent)
view.configureWindow() # adjust display levels, ROI, etc.
For a more complete description, see Image monitor.
11 - Image viewer
Main scientific image-viewing window: a central image display with row/column line-profile and histogram color-scale panels.
Overview
quatm/GUI/entries/image_group.py provides the image viewer window (ImageWindow): with a central ImageDisplay image display as the main widget and docked panels for row/column line profiles and the histogram color scale around it, it is used for live monitoring and analysis in imaging experiments.
Layout structure
| Dock area | Widget | Purpose |
|---|
| Center | ImageDisplay | Subscribes to image streams and refreshes automatically; masks can be overlaid |
| Top | LivePlot (row profile) | Intensity distribution for image rows |
| Right | LivePlot (column profile) | Intensity distribution for image columns |
| Left | HistogramLUTWidget | Histogram color scale: level range and false-color mapping |
Display parameters (in the Viewer/ namespace: min/max levels, gradient, tick_position) are restored from properties on construction and saved on close.
Usage
from quatm.GUI.entries.image_group import ImageWindow
win = ImageWindow(name="absorption_imaging") # standalone top-level window
win.show()
For a more complete description, see Image viewer.
12 - Live data monitor
Real-time plotting of measurement data: data sources / multiple y axes, filtering/binning/fitting and export.
Overview
quatm/GUI/alignment/plotter.py provides the live data monitor (DataPlotterWidget): it plots the data acquired during a measurement. It supports switching the data source and provides an x-axis selector and several y-axis selectors; each y axis is processed through a configurable data chain (scale factor → filter → binning/fitting) before being drawn in the plot area, and the result can be exported to PDF.
Components
| Component | Description |
|---|
DataPlotterWidget | Main plot: data-source drop-down, x-axis selector, several y-axis selectors (right-click toggles the configuration panel) |
YAxisWidgetHelper | y-axis data chain: scale factor, filter, binning and fitting |
Key methods
| Method | Description |
|---|
setDataSource | Sets or changes the data source |
checkForNewData / updateDatasources | Checks for new data / new data sources and refreshes |
plotData | Adds a set of x/y data to the plot area as a new curve |
toggleConfig | Toggles display of the data-source and y-axis configuration panel |
exportpdf | Exports the current plot as an archival-quality PDF |
clear | Clears the data summary and all curves in the plot area |
Y-axis chain (YAxisWidgetHelper): updateYscale sets the scale factor; the context menu offers addBinning for binning and addFitting for fitting (e.g. Gaussian/linear), and reevaluateData re-runs the whole data chain after changes such as filters.
Usage
from quatm.GUI.alignment.plotter import DataPlotterWidget
plotter = DataPlotterWidget(
axis_name="myaxis",
prop_root="...",
datastream_names=[".../data"],
n_yaxis=1,
parent=parent,
)
plotter.toggleConfig() # expand the data-source / x, y axis selection config
For a more complete description, see Auxiliary plotting.
13 - Property editor
A tool for viewing and editing the property tree, based on ParameterTree.
Overview
quatm/GUI/entries/property_editor.py provides the property editor: it presents the key/value tree of the property store as an expandable, editable parameter tree, and modifications are written back immediately. It supports browsing by subtree, searching, saving/loading and deleting the selected nodes.
Components
| Component | Description |
|---|
TreeEdit | Extracts information from Properties and organizes it into a tree-view structure |
PropEdit | Property editing main window (ParameterTree): subtree browsing, search, save/delete |
ScalableGroup | Special parameter group: automatically generates two mutually reciprocal child parameters |
PropSelector | Key-list selector: checks keys from the property tree and maintains a key list |
Usage
from quatm.GUI.entries.property_editor import PropEdit
editor = PropEdit(name="device_status", props=props, subtree="/", parent=parent)
Use subtree to restrict the display to properties under a given path. For a more complete description, see Property editor.
14 - Property update manager
Property timestamp watchdog: checks whether properties keep updating and raises an alarm in Deadman fashion after a timeout.
Overview
quatm/analysis/props_check.py provides the property update manager (PropsCheck): a watchdog that runs as a standalone analysis process and periodically reads the timestamp property specified by the property path _prop_path_abs; if it has not been updated for longer than _deadman_delay_sec, an alarm is raised through send_error (Mattermost can be configured).
Key methods
| Method | Description |
|---|
run | Periodically checks whether the timestamp property is fresh and raises an alarm in Deadman fashion on timeout |
Usage
from quatm.analysis.props_check import PropsCheck
pc = PropsCheck(name="props_watchdog") # property namespace
pc.run()
On the GUI side there is also a property checker (PropertiesChecker) for the same purpose, which verifies that the property center is really distributing updates (it sets a flag and reads it back; if the update has not propagated, it shows a dialog and raises an alarm), see Property checker.
15 - Skewed Gaussian fit
Fits a skewed Gaussian (skew-normal) distribution to 1D data in a data stream, additionally reporting skewness and peak position.
Overview
quatm/analysis/gaussfit_skewed.py provides skewed Gaussian fit analysis (GaussfitSkewed): it fits a skew-normal distribution to each frame of 1D data in a data stream and publishes the results to the data stream. Compared with an ordinary Gaussian fit, the skewed Gaussian describes asymmetric (tailed) peak shapes better, and the results additionally include the skewness parameter and the peak position.
Usage
from quatm.analysis.gaussfit_skewed import GaussfitSkewed
gs = GaussfitSkewed(name="skewed") # name serves as the property/data-stream namespace
gs.run()
The input data streams are configured in the property datastreams; the analysis process is started from the Analysis Manager (AnalysisManager), where it is selected with a checkbox.
16 - Data and image stream monitor
Real-time scrolling plot window: subscribes to data/image streams and scrolls them, with running mean and right-click configuration.
Overview
quatm/GUI/entries/live_plot.py provides data-stream/image-stream scrolling monitoring (ScrollPlot): inside the multi-subplot scrolling window ScrollWindow, each ScrollPlot subscribes to several channels of one data stream, maintains a ring buffer and scrolls scatter points and a moving-average curve; PlotDataEditor provides a data-source selection drop-down.
Components
| Component | Description |
|---|
ScrollWindow | Multi-subplot scrolling main window: restorePlots restores / addPlot appends subplots |
ScrollPlot | Scrolling subplot: ring buffer, scatter and moving average, context-menu operations |
PlotDataEditor | Data-source selection panel: drop-down candidates written back to properties/the parent plot |
| Method | Description |
|---|
feedData / setnewData | Pushes a new data block into the buffer / periodically pulls and refreshes |
updatePlot / clearPlot | Refreshes the curve / clears and rebuilds this plot |
running_mean | Efficiently computes the N-point moving average using cumulative sums and differences |
dataSelectDialog / subscribe_window / configureWindow | Select channels / change subscription / configure |
deletePlot / updateTitle | Removes this plot / synchronizes the title and data source |
Usage
from quatm.GUI.entries.live_plot import ScrollWindow
host = ScrollWindow(name="PMT scroll monitor", parent=parent, n_plots=2)
For a more complete description, see Live scrolling plot.
17 - Absorption imaging
Computes the optical density (column density) from with-atom, without-atom and background frames using the Beer-Lambert law.
Overview
quatm/analysis/imagedivider.py provides absorption imaging analysis (ImageDivider): it collects with-atom (atom), without-atom (no-atom) and background (background) images by imgindex, combines them with physical parameters such as the scattering cross-section, and computes and publishes the optical density (two-dimensional atomic column density) according to the Beer-Lambert law.
The intensity relation is I = I₀·exp(−n·σ), so the column density is n = ln(I₀/I)/σ; physical parameters such as detuning and linewidth are updated through event messages when the experiment starts.
Configuration properties
| Property | Description |
|---|
_atomindex / _noatomindex / _backgroundindex | imgindex of the with-atom / without-atom / background frames |
| Detuning / linewidth, etc. | Imaging laser and transition parameters (updated by experiment events) |
Usage
from quatm.analysis.imagedivider import ImageDivider
div = ImageDivider(name="od") # subscribe to the corresponding image streams
div.run()
The analysis process is started from the Analysis Manager (AnalysisManager), where it is selected with a checkbox.
18 - Ion detection series
Bright-spot/single-ion detection and ion-state discrimination: the BrightSpots, SingleIonPresent and IonState analysis processes.
Module overview
| Module | Class | Detection method | Purpose |
|---|
ion_blob | BrightSpots | Threshold segmentation + connected-component counting | Counts the number of ions in each frame |
bright_spots | BrightSpots | skimage blob_dog | Detects bright spots and outputs a position mask and bloblist |
single_ion_present | SingleIonPresent | Counting + multiple Gaussian-fit-parameter criteria | Decides whether a single ion is present |
ion_state | IonState | Multi-image sequences by imgindex | Discriminates the ion product state |
Discrimination workflow
IonState collects the check / survival / hot / shelved image series by imgindex, detects whether an ion is present in each image, and aggregates the results into the product state:
group frames by imgindex (check / survival / hot / shelved)
↓ detect the ion in each frame by blob analysis
aggregate → publish the product-state result
Usage
from quatm.analysis.ion_state import IonState
det = IonState(name="ionstate") # subscribe to the image streams and group by imgindex
det.run()
Each process is started from the Analysis Manager (AnalysisManager), where it is selected with a checkbox; image streams, thresholds, imgindex, etc. are configured in the property tree.
19 - Image processing tools
Image processing analysis processes for hot-pixel filtering/finding, rotation, slicing, frame selection and background subtraction.
Module list
The following are all standalone analysis processes that subscribe to image streams, located in quatm/analysis/:
| Module | Class | Description |
|---|
hot_pixel_filter | HotFilter | Filters/replaces hot pixels according to the coordinate list in the property hotpixels and sends the result back |
hot_pixel_finder | HotFilter | Finds hot-pixel coordinates statistically and writes them to properties (for use by the filtering script) |
rotate_image | ImageRotate | Rotates the image by an angle and rotates the coordinate offsets accordingly |
imageslice | ImageSlice | Computes row/column projections (sum/clip/integrate) over the ROI and sub-region images |
imageselector | ImageSlice | Selects the frame with the specified imgindex from the image stream and forwards it |
substract_image | ImageSubstract | Subtracts a fixed background image (automatically adapting for transposition) and outputs the difference |
Usage
from quatm.analysis.hot_pixel_filter import HotFilter
f = HotFilter(name="hotfix") # subscribe to the image stream and filter with the hot-pixel list from the properties
f.run()
Each process is started from the Analysis Manager (AnalysisManager), where it is selected with a checkbox; input streams and parameters are configured in the property tree.