BCDA-APS
Created find_ar() Plotting errors
Describe the bug created find_ar search function, now get lots of errors from best effort callback. And get open plots for 13 channels. Somehow need to configure BEC better to plot only the one signal. Help needed...
New stream: 'baseline'
New stream: 'primary'
E Sun-09:21:12.758: An exception raised in the callback <function BestEffortCallback.__call__ at 0x7f1a26ce7880> is being suppressed to not interrupt plan execution. To investigate try setting the BLUESKY_DEBUG_CALLBACKS env to '1'
Traceback (most recent call last):
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/core.py", line 65, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 103, in __call__
super().__call__(name, doc, *args, **kwargs)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/mpl_plotting.py", line 80, in __call__
return CallbackBase.__call__(self, name, doc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/event_model/__init__.py", line 191, in __call__
return self._dispatch(name, doc, validate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/event_model/__init__.py", line 199, in _dispatch
output_doc = getattr(self, name)(doc)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 381, in descriptor
self._set_up_plots(doc, stream_name, columns)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 249, in _set_up_plots
dtype = doc["data_keys"][y_key]["dtype"]
~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'channels.A'
E Sun-09:21:12.937: Error in find_ar: Unequal shapes: x.shape=(1,) y.shape=(0,)
E Sun-09:21:12.938: Run aborted
Traceback (most recent call last):
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 604, in contingency_wrapper
ret = yield from plan
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plans.py", line 1166, in inner_scan_nd
yield from per_step(detectors, step, pos_cache)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1952, in __iter__
return (yield from self._iter)
^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plan_stubs.py", line 1729, in one_nd_step
yield from take_reading(list(detectors) + list(motors)) # type: ignore # Movable issue
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1952, in __iter__
return (yield from self._iter)
^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plan_stubs.py", line 1471, in trigger_and_read
return (yield from rewindable_wrapper(inner_trigger_and_read(), rewindable))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 731, in rewindable_wrapper
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plan_stubs.py", line 1466, in inner_trigger_and_read
ret = yield from contingency_wrapper(read_plan(), except_plan=exception_path, else_plan=standard_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 623, in contingency_wrapper
yield from else_plan()
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plan_stubs.py", line 1460, in standard_path
yield from save()
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1952, in __iter__
return (yield from self._iter)
^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plan_stubs.py", line 128, in save
return (yield Msg("save"))
^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 259, in msg_mutator
_s = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
[Previous line repeated 4 more times]
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/run_engine.py", line 1676, in _run
new_response = await coro(msg)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/run_engine.py", line 2076, in _save
await current_run.save(msg)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/bundlers.py", line 570, in save
await self.emit(DocumentNames.event, event_doc)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/run_engine.py", line 2679, in emit
self.emit_sync(name, doc)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/run_engine.py", line 2676, in emit_sync
self.dispatcher.process(name, doc)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/run_engine.py", line 2699, in process
exceptions = self.cb_registry.process(name, name.name, doc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 422, in process
func(*args, **kwargs)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 512, in __call__
return mtd(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/apstools/callbacks/scan_signal_statistics.py", line 159, in receiver
getattr(self, key)(document)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/apstools/callbacks/scan_signal_statistics.py", line 151, in event
y = doc["data"][yname]
~~~~~~~~~~~^^^^^^^
KeyError: 'channels.A'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/run_engine.py", line 1609, in _run
msg = self._plan_stack[-1].send(resp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/apstools/plans/labels_to_streams.py", line 231, in label_stream_wrapper
return (yield from bpp.plan_mutator(plan, action[when.value]))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 1375, in __call__
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 1228, in baseline_wrapper
return (yield from plan_mutator(plan, insert_baseline))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 845, in monitor_during_wrapper
return (yield from plan2)
^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 907, in fly_during_wrapper
return (yield from plan2)
^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams11/USAXS/bits_usaxs/src/usaxs/plans/axis_tuning.py", line 256, in find_ar
yield from lineup2(
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/apstools/plans/alignment.py", line 570, in lineup2
yield from _inner() # Run the scan.
^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1258, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 413, in subs_wrapper
return (yield from finalize_wrapper(_inner_plan(), _unsubscribe()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 536, in finalize_wrapper
ret = yield from plan
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 411, in _inner_plan
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/apstools/plans/alignment.py", line 567, in _inner
yield from bp.rel_scan(detectors, mover, rel_start, rel_end, points, md=_md)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plans.py", line 1596, in rel_scan
return (yield from inner_rel_scan())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1258, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 1175, in reset_positions_wrapper
return (yield from finalize_wrapper(plan_mutator(plan, insert_reads), reset()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 536, in finalize_wrapper
ret = yield from plan
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1258, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 1131, in relative_set_wrapper
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 271, in msg_mutator
msg = plan.send(_s)
^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 181, in plan_mutator
raise ex
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 134, in plan_mutator
msg = plan_stack[-1].send(ret)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plans.py", line 1594, in inner_rel_scan
return (yield from scan(detectors, *args, num=num, per_step=per_step, md=_md))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plans.py", line 1291, in scan
return (yield from scan_nd(detectors, full_cycler, per_step=per_step, md=_md))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/plans.py", line 1168, in scan_nd
return (yield from inner_scan_nd())
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1258, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 995, in stage_wrapper
return (yield from finalize_wrapper(inner(), unstage_devices()))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 536, in finalize_wrapper
ret = yield from plan
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 993, in inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/utils/__init__.py", line 1258, in dec_inner
return (yield from plan)
^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 358, in run_wrapper
yield from contingency_wrapper(plan, except_plan=except_plan, else_plan=close_run)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 614, in contingency_wrapper
ret = yield from except_plan(e)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 259, in msg_mutator
_s = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 89, in plan_mutator
msg = plan_stack[-1].throw(exception)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 898, in new_gen
yield msg
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
inner_ret = yield msg
^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 89, in plan_mutator
msg = plan_stack[-1].throw(exception)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 836, in new_gen
yield msg
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/preprocessors.py", line 206, in plan_mutator
@prjemian this needs probably simple fix, can test today or tomorrow.
code in question is here: https://vscode.dev/github/BCDA-APS/usaxs-bits/blob/main/src/usaxs/plans/axis_tuning.py#L207
This channel (KeyError: 'channels.A') looks like the calculation signal A. Chances are the fail happened when the signal is not marked for plotting (with its kind="hinted").
Looking at these lines: https://github.com/BCDA-APS/usaxs-bits/blob/7f620e2bdefff7cada13f5d351f273259d9ef73a/src/usaxs/plans/axis_tuning.py#L252-L256
These need to be verified, but summarize the general idea (to replace line 253 shown here):
upd_photocurrent_calc.kind = "hinted"
scaler0.select_channels([]) # no scaler channels to be plotted (sets `kind="normal"` for all channels)
An ophyd Signal has a kind attribute that advises BestEffortCallback (and the RunEngine) how to handle that Signal.
kind value |
actions |
|---|---|
"hinted" |
the "normal" actions plus show in LivePlot, LiveTable, ... if named as detector or motor |
"normal" |
publish in event document (with metadata in descriptor) |
"config" |
publish in descriptor (stream header) document |
"omitted" |
do not publish in any document |
None |
Used in __init__ to set kind="normal" |
Summary: "hinted" means the BestEffortCallback will render this signal by all of its enabled visualizations.
Does not work. The problem is not scaler0, where the right command is:
scaler0.select_channels()
But the problem is, that the
upd_photocurrent_calc
has multiple channels (A-L) and it has no method select. How do I pass to the lineup2 what is the right thing to scan? And setup that thing for plotting in BEC?
In [2]: upd_photocurrent_calc
Out[2]: ModifiedSwaitRecord(prefix='usxLAX:USAXS:upd', name='upd_photocurrent_calc', read_attrs=['channels.A', 'channels.B', 'channels.C', 'channels.D', 'channels.E', 'channels.F', 'channels.G', 'channels.H', 'channels.I', 'channels.J', 'channels.K', 'channels.L'], configuration_attrs=['description', 'scanning_rate', 'disable_value', 'scan_disable_input_link_value', 'scan_disable_value_input_link', 'forward_link', 'device_type', 'alarm_status', 'alarm_severity', 'new_alarm_status', 'new_alarm_severity', 'disable_alarm_severity', 'precision', 'high_operating_range', 'low_operating_range', 'calculation', 'output_link_pv', 'output_location_name', 'output_location_data', 'output_data_option', 'output_execute_option', 'output_execution_delay', 'event_to_issue', 'channels', 'channels.A', 'channels.A.input_value', 'channels.A.input_pv', 'channels.A.input_trigger', 'channels.B', 'channels.B.input_value', 'channels.B.input_pv', 'channels.B.input_trigger', 'channels.C', 'channels.C.input_value', 'channels.C.input_pv', 'channels.C.input_trigger', 'channels.D', 'channels.D.input_value', 'channels.D.input_pv', 'channels.D.input_trigger', 'channels.E', 'channels.E.input_value', 'channels.E.input_pv', 'channels.E.input_trigger', 'channels.F', 'channels.F.input_value', 'channels.F.input_pv', 'channels.F.input_trigger', 'channels.G', 'channels.G.input_value', 'channels.G.input_pv', 'channels.G.input_trigger', 'channels.H', 'channels.H.input_value', 'channels.H.input_pv', 'channels.H.input_trigger', 'channels.I', 'channels.I.input_value', 'channels.I.input_pv', 'channels.I.input_trigger', 'channels.J', 'channels.J.input_value', 'channels.J.input_pv', 'channels.J.input_trigger', 'channels.K', 'channels.K.input_value', 'channels.K.input_pv', 'channels.K.input_trigger', 'channels.L', 'channels.L.input_value', 'channels.L.input_pv', 'channels.L.input_trigger'])
-
scaler0.select_channels()will plot all scaler channels (named in EPICS) -
scaler0.select_channels([])will not plot any scaler channels -
upd_photocurrent_calc.channels.A.kind="hinted" -
upd_photocurrent_calc.channels.kind="hinted"(might be necessary to reach A) -
upd_photocurrent_calc.kind="hinted"(might be necessary to reach channels)
scaler0.select_channels([]) as well as scaler0.select_channels([""]) gave me error.
But I will try again if I can.
Could not try, but previously it gave me graphs for all channels in upd_photocurrent_calc.channels, all A-L. I suspect the problem is how do I set all channels except A to omitted. By now it looks like I have to test this off line.
I suggest that because that was needed with Area Detector Stats to use Max value in a lineup for XPCS. In their case, the signal was not shown until each parent in the hierarchy was also hinted. By setting the scaler0 to normal, the chain is broken at the start.
Restarting with debugging again... Existing code:
# control BEC plotting since we use upd_photocurrent_calc
upd_photocurrent_calc.kind = "hinted" # set kind to inlcude in plotting by BEC
upd_photocurrent_calc.channels.kind = "hinted" # set kind to inlcude in plotting by BEC
upd_photocurrent_calc.channels.A.kind = "hinted" # set kind to inlcude in plotting by BEC
scaler0.select_channels([]) # no scaler channels to be plotted (sets 'kinnd=normal' for all channels)
stats = SignalStatsCallback()
yield from lineup2(
[upd_photocurrent_calc, scaler0],
a_stage.r,
-5*a_stage.r.tune_range.get(),
5*a_stage.r.tune_range.get(),
41,
nscans=5,
signal_stats=stats,
md=md,
)
print(stats.report())
fails with this:
I Mon-19:43:11.236: tuning axis: a_stage_r
Transient Scan ID: 2 Time: 2025-10-13 19:43:13
Persistent Unique Scan ID: '869cba58-95e2-4971-8882-6640d34cd892'
New stream: 'label_start_motor'
New stream: 'baseline'
New stream: 'primary'
E Mon-19:43:17.119: An exception raised in the callback <function BestEffortCallback.__call__ at 0x7fdb95063740> is being suppressed to not interrupt plan execution. To investigate try setting the BLUESKY_DEBUG_CALLBACKS env to '1'
Traceback (most recent call last):
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/core.py", line 65, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 103, in __call__
super().__call__(name, doc, *args, **kwargs)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/mpl_plotting.py", line 80, in __call__
return CallbackBase.__call__(self, name, doc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/event_model/__init__.py", line 191, in __call__
return self._dispatch(name, doc, validate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/event_model/__init__.py", line 199, in _dispatch
output_doc = getattr(self, name)(doc)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 381, in descriptor
self._set_up_plots(doc, stream_name, columns)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 249, in _set_up_plots
dtype = doc["data_keys"][y_key]["dtype"]
~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'channels.A'
No strong peak found. Peak is too wide. Computed peak sigma is zero.
Motor: 'a_stage_r' Detector: 'upd_photocurrent_calc'
========= ==============================================================================
statistic value
========= ==============================================================================
n 41
centroid 9.999999546644244
fwhm 0.008000055554012064
min_x 9.995999518867237
mean_x 9.999999055465475
max_x 10.003999574421249
min_y 0.1
mean_y 0.09999999999999999
max_y 0.1
success False
reasons ['No strong peak found.', 'Peak is too wide.', 'Computed peak sigma is zero.']
========= ==============================================================================
None
Result: False
I Mon-19:43:26.753: find_ar failed for ['No strong peak found.', 'Peak is too wide.', 'Computed peak sigma is zero.']
Out[1]: ('869cba58-95e2-4971-8882-6640d34cd892',)
In [2]:
Getting always all 12 channels for upd_photocurrent_calc in BEC plots. Tried setting following to normal: upd_photocurrent_calc.kind = "normal" # set kind to inlcude in plotting by BEC upd_photocurrent_calc.channels.kind = "nomral" # set kind to include in plotting by BEC to stop them from plotting, still plots.
What helped, somehow is to set :
yield from lineup2(
[upd_photocurrent_calc.channels.A, scaler0],
a_stage.r,
-5*a_stage.r.tune_range.get(),
5*a_stage.r.tune_range.get(),
41,
nscans=5,
signal_stats=stats,
md=md,
)
This produces only one plot of channel A. So far so good... BUT now we have different key error :
New stream: 'primary'
E Mon-20:29:11.972: An exception raised in the callback <function BestEffortCallback.__call__ at 0x7ff5a8f9f740> is being suppressed to not interrupt plan execution. To investigate try setting the BLUESKY_DEBUG_CALLBACKS env to '1'
Traceback (most recent call last):
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/core.py", line 65, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 103, in __call__
super().__call__(name, doc, *args, **kwargs)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/mpl_plotting.py", line 80, in __call__
return CallbackBase.__call__(self, name, doc)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/event_model/__init__.py", line 191, in __call__
return self._dispatch(name, doc, validate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/event_model/__init__.py", line 199, in _dispatch
output_doc = getattr(self, name)(doc)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 381, in descriptor
self._set_up_plots(doc, stream_name, columns)
File "/home/beams/USAXS/.conda/envs/bits_usaxs/lib/python3.11/site-packages/bluesky/callbacks/best_effort.py", line 249, in _set_up_plots
dtype = doc["data_keys"][y_key]["dtype"]
~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'input_value'
Reminder:
n [2]: upd_photocurrent_calc.channels.A
Out[2]: SwaitRecordChannel(prefix='usxLAX:USAXS:upd', name='upd_photocurrent_calc_channels_A', parent='upd_photocurrent_calc_channels', read_attrs=['input_value'], configuration_attrs=['input_value', 'input_pv', 'input_trigger'])
I am confused. Scanning the channels A-L is nonsense. those are input values. Those are not read attributes. There doe snot seem to be proper value in BS configured, proper value to read is actually the usxLAX:USAXS:upd.VAL. All of the others are not counters, those are inputs and should not be treated by BS as anything one can count and plot.
I think our swait record is screwy... @prjemian ?
Out[4]: ModifiedSwaitRecord(prefix='usxLAX:USAXS:upd', name='upd_photocurrent_calc', read_attrs=['channels.A', 'channels.B', 'channels.C', 'channels.D', 'channels.E', 'channels.F', 'channels.G', 'channels.H', 'channels.I', 'channels.J', 'channels.K', 'channels.L'], configuration_attrs=['description', 'scanning_rate', 'disable_value', 'scan_disable_input_link_value', 'scan_disable_value_input_link', 'forward_link', 'device_type', 'alarm_status', 'alarm_severity', 'new_alarm_status', 'new_alarm_severity', 'disable_alarm_severity', 'precision', 'high_operating_range', 'low_operating_range', 'calculation', 'output_link_pv', 'output_location_name', 'output_location_data', 'output_data_option', 'output_execute_option', 'output_execution_delay', 'event_to_issue', 'channels', 'channels.A', 'channels.A.input_value', 'channels.A.input_pv', 'channels.A.input_trigger', 'channels.B', 'channels.B.input_value', 'channels.B.input_pv', 'channels.B.input_trigger', 'channels.C', 'channels.C.input_value', 'channels.C.input_pv', 'channels.C.input_trigger', 'channels.D', 'channels.D.input_value', 'channels.D.input_pv', 'channels.D.input_trigger', 'channels.E', 'channels.E.input_value', 'channels.E.input_pv', 'channels.E.input_trigger', 'channels.F', 'channels.F.input_value', 'channels.F.input_pv', 'channels.F.input_trigger', 'channels.G', 'channels.G.input_value', 'channels.G.input_pv', 'channels.G.input_trigger', 'channels.H', 'channels.H.input_value', 'channels.H.input_pv', 'channels.H.input_trigger', 'channels.I', 'channels.I.input_value', 'channels.I.input_pv', 'channels.I.input_trigger', 'channels.J', 'channels.J.input_value', 'channels.J.input_pv', 'channels.J.input_trigger', 'channels.K', 'channels.K.input_value', 'channels.K.input_pv', 'channels.K.input_trigger', 'channels.L', 'channels.L.input_value', 'channels.L.input_pv', 'channels.L.input_trigger'])
I asked chatGPT on this problem and I got answer I can understand and looks reasonable - basically, make simpler device which will not have confusing channels. However, there is no kind noted in the first simple device, so I am not sure how this will work with the kind="hinted" needed for BEC.The more complicated example later has kinds set. May be that is the way to go? May be I need to check source code for swait record...
Quote:
this is a common integration issue between EPICS userCalc/SWAIT records and Bluesky’s ophyd layer. Bluesky doesn’t automatically know that .VAL of your userCalc or swait record is the actual “detector signal”; it just sees a record with multiple input fields (A–L), and by default ophyd auto-generates signals for all fields it can find.
Here’s how to make it behave like a proper counter/detector:
Expose only the calculated value (.VAL) of your SWAIT record as the readable signal for Bluesky, so scans like lineup2 treat it as a scalar detector.
- Define an Ophyd device for your SWAIT record
In your Bluesky startup (e.g., startup/00-load.py or IPython console):
from ophyd import Device, EpicsSignalRO
class SWAITDetector(Device):
value = EpicsSignalRO("your:swait:record.VAL", name="value")
# Example:
my_calc_det = SWAITDetector("your:swait:record.", name="my_calc_det")
This tells Bluesky: • Only .VAL is the readout value. • It’s read-only (RO), since .VAL is computed by EPICS.
to test:
my_calc_det.read()
{'my_calc_det_value': {'value': 123.4, 'timestamp': 1728840000.0}}
Interestingly., this works for existing upd_photocurrent_calc also...
To use: RE(lineup2(motor, my_calc_det))
Optionally, can extend the class If you want to also access inputs A–L for debugging:
from ophyd import Component as Cpt, EpicsSignal
class SWAITDetector(Device):
value = Cpt(EpicsSignalRO, "VAL", kind="hinted")
A = Cpt(EpicsSignal, "A", kind="normal")
B = Cpt(EpicsSignal, "B", kind="normal")
# etc.
Sure, we are using wrong SwaitRecord. SwaitRecord in Synapps
https://bcda-aps.github.io/apstools/latest/api/synApps/_swait.html
is written for userCalc(N) where we have 8 channels of swait records - independent user calcs.
EPICS synApps swait record: used as $(P):userCalc$(N)
This is different, this upd_photocurrent_calc is single channel of calculation only. Need to find different devices for these or define my own device.
Sure, we are using wrong SwaitRecord. SwaitRecord in Synapps
https://bcda-aps.github.io/apstools/latest/api/synApps/_swait.html
is written for userCalc(N) where we have 8 channels of swait records - independent user calcs.
EPICS synApps swait record: used as $(P):userCalc$(N)This is different, this upd_photocurrent_calc is single channel of calculation only. Need to find different devices for these or define my own device.
apstools.synApps.SwaitRecord is the correct Python class to use.
This table (on the page you showed) lists several different Python classes:
-
UserCalcsDeviceis for the 10 instances ofSwaitRecordwhich comprise the "user calcs" in a synApps IOC. -
SwaitRecordis for any instance of answaitrecord.
We need to dig down into your specific use and make sure the kind attributes are set properly so the correct Signal from upd_photocurrent_calc can be used with lineup2().
asked chatGPT on this problem
This proposition from an AI chatbot is ill-advised. As such, it is bad advice. The AI is ill-advised because it lacks sufficient training sets to inform the LLM of proper use. Lacking that context, the LLM prioritizes delivering any answer (with confidence) over correctness. Be very suspicious of how any LLM advises you on selection of code from libraries such as apstools with limited training sets.
The chatbots are great at debugging Python code but very poor at advising which parts to use fromapstools (and many other Bluesky packages).
Well, I'll leave the figuring the "correct way of doing this" out to my friendly BCDA BITS support...
BUT: I have created new a device class (amplifers.py line 51):
class usaxs_calculated_signal(Device):
"""Device for USAXS calculated signal. """
value = Component(EpicsSignalRO,".VAL",kind="hinted")
and use that in scaler_and_amplifiers.yml for upd_photocurrent_calc (and other calculated signals)
and pass as the upd_photocurrent_calc as counter for lineup2 for find_ar() (axis_tuning.py line 209). I tested the find_ar() and run it (without beam) and it works as far as I can test it today:
- Plots the proper (noise only) upd_photocurrent_calc signal
- Runs multiple scans, homing on some random peak noise for now
- I can understand this, it returns epics calculated value and reads proper epics PV.
In other words, as far as I can say, it works and this programmer can understand it.
Is this wrong, OK, fix it.
OK, tested after tweaking the usaxs_calculated_signal calculation in epics to prevent range change artifacts. Now works great. Need to tweak based on experience more - reduce number of passes to 3-4 and add last pass same as tune_ar with fixed gain.
Request @prjemian - lineup 2 seems to print on screen counter assuming it uses counts and so it prints integers. Unluckily, current is ugly low real numbers and so user sees all values as 0. Peak is find, rest works. But can we change the display to something which would look like we have signal? This is simply human interface design issue, but might be helpful.
lineup 2 seems to print on screen counter [as integers]
I'm suspecting this is less a problem with lineup2() (which does no plotting of its own) and the BestEffortCallback's LivePlot.
This can be explored easily using alternative plotting tools, such as gemviz.
Here is a tune_ar scan from 2025-01, plotted with gemviz. The Y axis (apparently) is counts.
This is a plot of lineup2() with a simulated photocurrent (< nA).
Values in the LiveTable() (shown here) are printed with fixed precision (all zeroes here):
Transient Scan ID: 10126 Time: 2025-11-25 16:07:04
Persistent Unique Scan ID: '0f378f94-5ca4-4132-a6ff-138584ed53dc'
New stream: 'label_start_motor'
New stream: 'baseline'
New stream: 'primary'
+-----------+------------+------------+------------+
| seq_num | time | m1 | noisy |
+-----------+------------+------------+------------+
| 1 | 16:07:05.0 | -1.2382 | 0.00000 |
| 2 | 16:07:05.2 | -1.2237 | 0.00000 |
| 3 | 16:07:05.4 | -1.2091 | 0.00000 |
| 4 | 16:07:05.6 | -1.1946 | 0.00000 |
| 5 | 16:07:05.8 | -1.1801 | 0.00000 |
| 6 | 16:07:06.0 | -1.1655 | 0.00000 |
| 7 | 16:07:06.2 | -1.1510 | 0.00000 |
| 8 | 16:07:06.4 | -1.1364 | 0.00000 |
| 9 | 16:07:06.6 | -1.1219 | 0.00000 |
| 10 | 16:07:06.8 | -1.1074 | 0.00000 |
| 11 | 16:07:07.0 | -1.0928 | 0.00000 |
| 12 | 16:07:07.2 | -1.0783 | 0.00000 |
| 13 | 16:07:07.4 | -1.0637 | 0.00000 |
| 14 | 16:07:07.6 | -1.0492 | 0.00000 |
...
I wonder if this is related to this comment in a bluesky issue?
In the case just shown, the source of data (shown in its descriptor document) says dtype: number.
descriptors:
- configuration:
noisy:
data:
noisy: 1.337759031460422e-11
timestamps:
noisy: 1764108425.0445
data_keys:
noisy:
source: PV:gp:userCalc1
dtype: number
shape: []
units: ''
lower_ctrl_limit: 0
upper_ctrl_limit: 0
precision: 5
The problem is not with apstools but with how BestEffortCallback renders numbers with such small magnitude.