charge event building

raw event generator

class module0_flow.reco.charge.raw_event_generator.RawEventBuilder(**params)

Bases: object

Base class for raw event builder algorithms. Defines the following API for implementing new event-building algorithms:

abstract build_events(packets, unix_ts, mc_assn=None)

Run the event builder on a sub-set of packet-formatted array data The unix timestamp for each packet is provided as additional meta-data

Parameters:

packets – packet-formatted array (shape: (N,))
unix_ts – Unix timestamp for each packet in packets (shape: (N,))
mc_assn – array of mc truth associations for each packet in packets (shape: (N,))

Returns:

a tuple of lists of the packet array grouped into events, along with their corresponding unix timestamps

cross_rank_get_attrs(*attrs)

Get an attribute from another MPI process. In particular:

N-1 sends its stored attribute to 0

then, i receives the attribute from i-1

Parameters:: attrs – list of str specifying attributes to pass between ranks

cross_rank_set_attrs(*attrs)

Update an attribute and send to another MPI process. In particular:

i sends the attribute to i+1

N-1 does nothing

Parameters:: attrs – list of str specifying attributes to pass between ranks

get_config()

Returns:: a dict of the instance configuration parameters

version = 0.0.0

class module0_flow.reco.charge.raw_event_generator.RawEventGenerator(**params)

Bases: h5flow.core.H5FlowGenerator

Low-level event builder - generates packet groups according to the specified algorithm from a larpix packet datalog file.

For simulated data, also creates the following datasets:

mc_truth/events: Dataset containing event-level truth
mc_truth/trajectories: Dataset containing true particle trajectories
mc_truth/tracks: Dataset containing true edep-sim segments

and references:

mc_truth/events -> mc_truth/trajectories
mc_truth/events -> mc_truth/tracks
mc_truth/trajectories -> mc_truth/tracks
charge/raw_events -> mc_truth/events
charge/packets -> mc_truth/tracks

Parameters:

packets_dset_name : str, required, output dataset path for packet groups
buffer_size : int, optional, number of packets to load at a time
nhit_cut : int, optional, minimum number of packets in an event
sync_noise_cut_enabled : bool, optional, remove hits occuring soon after a SYNC event
sync_noise_cut : int, optional, if sync_noise_cut_enabled removes all events that have a timestamp less than this value
timestamp_bit_error_fix_enabled : bool, optional, allow fixing bit content from certain chips based on “nearby” data
timestamp_bit_error_window : int, optional, defines the “nearby” region in the file used to set the timestamp bits
timestamp_bit_error_spec : multi-depth dict specifying the chips and bits to fix : <unique_bitmask>: <io group>: <io channel>: [<chip ids>]
event_builder_class : str, optional, event builder algorithm to use (see raw_event_builder.py)
event_builder_config : dict, optional, modify parameters of the event builder algorithm (see raw_event_builder.py)
mc_events_dset_name : str, optional, output dataset for mc events (if present)
mc_tracks_dset_name : str, optional, output dataset path for mc truth tracks (if present)
mc_trajectories_dset_name : str, optional, output dataset path for mc truth trajectories (if present)
mc_packet_fraction_dset_name : str, optional, output dataset path for packet charge fraction truth (if present)

dset_name points to a lightweight array used to organize low-level event references.

Requires Units, RunData, and LArData resources in workflow.

Example config:

raw_event_generator:
    classname: RawEventGenerator
    dset_name: 'charge/raw_events'
    params:
        packets_dset_name: 'charge/packets'
        buffer_size: 38400
        nhit_cut: 100
        sync_noise_cut: [100000, 10000000]
        sync_noise_cut_enabled: True
        event_builder_class: 'SymmetricWindowRawEventBuilder'
        event_builder_config:
            window: 910
            threshold: 10
            rollover_ticks: 10000000

raw_event datatype:

id          u8, unique event identifier
unix_ts     u8, unix timestamp of event [s since epoch]

class_version = 0.3.0

default_buffer_size = 38400

default_event_builder_class = SymmetricWindowRawEventBuilder

default_event_builder_config

default_mc_events_dset_name = mc_truth/events

default_mc_packet_fraction_dset_name = mc_truth/packet_fraction

default_mc_tracks_dset_name = mc_truth/tracks

default_mc_trajectories_dset_name = mc_truth/trajectories

default_nhit_cut = 100

default_packets_dset_name = charge/packets

default_skip_mc_truth = False

default_sync_noise_cut = [100000, 10000000]

default_sync_noise_cut_enabled = True

default_timestamp_bit_error_fix_enabled = True

default_timestamp_bit_error_window = 256

finish()

init()

mc_event_dtype

next()

Read in a new block of LArPix packet data from the input file and apply the raw event building algorithm. Save packets to a dataset (packets_dset_name) and create references to a raw event (raw_event_dset_name).

Returns:: slice into the dataset given by raw_event_dset_name

pass_last_unix_ts(packets)

raw_event_dtype

class module0_flow.reco.charge.raw_event_generator.SymmetricWindowRawEventBuilder(**params)

Bases: RawEventBuilder

A sliding-window based event builder.

Histograms the packets into bins of window width. Events are formed if a bin content is greater than threshold. The event extent covers the bin of interest and +/- 1 bin. If multiple adjacent bins exceed the threshold, they are merged into a single event.

Configurable parameters:

window      - bin width
threshold   - number of correlated hits to initiate event

build_events(packets, unix_ts, mc_assn=None)

Run the event builder on a sub-set of packet-formatted array data The unix timestamp for each packet is provided as additional meta-data

Parameters:

packets – packet-formatted array (shape: (N,))
unix_ts – Unix timestamp for each packet in packets (shape: (N,))
mc_assn – array of mc truth associations for each packet in packets (shape: (N,))

Returns:

a tuple of lists of the packet array grouped into events, along with their corresponding unix timestamps

default_rollover_ticks = 10000000

default_threshold = 10

default_window

get_config()

Returns:: a dict of the instance configuration parameters

version = 0.0.2

class module0_flow.reco.charge.raw_event_generator.TimeDeltaRawEventBuilder(**params)

Bases: RawEventBuilder

Original “gap-based” event building

Searches for separations in data greater than the event_dt parameter. Events are formed at these boundaries. Any events that are greater than max_event_dt in length are broken up into separate events at the max_event_dt boundaries.

Configurable parameters:

event_dt        - gap size to separate into different events
max_event_dt    - maximum event length

build_events(packets, unix_ts, mc_assn=None)

Run the event builder on a sub-set of packet-formatted array data The unix timestamp for each packet is provided as additional meta-data

Parameters:

packets – packet-formatted array (shape: (N,))
unix_ts – Unix timestamp for each packet in packets (shape: (N,))
mc_assn – array of mc truth associations for each packet in packets (shape: (N,))

Returns:

a tuple of lists of the packet array grouped into events, along with their corresponding unix timestamps

default_event_dt = 1820

default_max_event_dt

get_config()

Returns:: a dict of the instance configuration parameters

static split_at_timestamp(timestamp, event, *args)

Breaks event into two arrays at index where event[‘timestamp’] > timestamp Additional arrays can be specified with kwargs and will be split at the same index

Returns:: tuple of two event halves followed by any additional arrays (in pairs)

version = 0.0.0

raw event builder algorithms

class module0_flow.reco.charge.raw_event_builder.RawEventBuilder(**params)

Bases: object

Base class for raw event builder algorithms. Defines the following API for implementing new event-building algorithms:

abstract build_events(packets, unix_ts, mc_assn=None)

Run the event builder on a sub-set of packet-formatted array data The unix timestamp for each packet is provided as additional meta-data

Parameters:

packets – packet-formatted array (shape: (N,))
unix_ts – Unix timestamp for each packet in packets (shape: (N,))
mc_assn – array of mc truth associations for each packet in packets (shape: (N,))

Returns:

a tuple of lists of the packet array grouped into events, along with their corresponding unix timestamps

cross_rank_get_attrs(*attrs)

Get an attribute from another MPI process. In particular:

N-1 sends its stored attribute to 0

then, i receives the attribute from i-1

Parameters:: attrs – list of str specifying attributes to pass between ranks

cross_rank_set_attrs(*attrs)

Update an attribute and send to another MPI process. In particular:

i sends the attribute to i+1

N-1 does nothing

Parameters:: attrs – list of str specifying attributes to pass between ranks

get_config()

Returns:: a dict of the instance configuration parameters

version = 0.0.0

class module0_flow.reco.charge.raw_event_builder.SymmetricWindowRawEventBuilder(**params)

Bases: RawEventBuilder

A sliding-window based event builder.

Histograms the packets into bins of window width. Events are formed if a bin content is greater than threshold. The event extent covers the bin of interest and +/- 1 bin. If multiple adjacent bins exceed the threshold, they are merged into a single event.

Configurable parameters:

window      - bin width
threshold   - number of correlated hits to initiate event

build_events(packets, unix_ts, mc_assn=None)

Run the event builder on a sub-set of packet-formatted array data The unix timestamp for each packet is provided as additional meta-data

Parameters:

packets – packet-formatted array (shape: (N,))
unix_ts – Unix timestamp for each packet in packets (shape: (N,))
mc_assn – array of mc truth associations for each packet in packets (shape: (N,))

Returns:

a tuple of lists of the packet array grouped into events, along with their corresponding unix timestamps

default_rollover_ticks = 10000000

default_threshold = 10

default_window

get_config()

Returns:: a dict of the instance configuration parameters

version = 0.0.2

class module0_flow.reco.charge.raw_event_builder.TimeDeltaRawEventBuilder(**params)

Bases: RawEventBuilder

Original “gap-based” event building

Searches for separations in data greater than the event_dt parameter. Events are formed at these boundaries. Any events that are greater than max_event_dt in length are broken up into separate events at the max_event_dt boundaries.

Configurable parameters:

event_dt        - gap size to separate into different events
max_event_dt    - maximum event length

build_events(packets, unix_ts, mc_assn=None)

Run the event builder on a sub-set of packet-formatted array data The unix timestamp for each packet is provided as additional meta-data

Parameters:

packets – packet-formatted array (shape: (N,))
unix_ts – Unix timestamp for each packet in packets (shape: (N,))
mc_assn – array of mc truth associations for each packet in packets (shape: (N,))

Returns:

a tuple of lists of the packet array grouped into events, along with their corresponding unix timestamps

default_event_dt = 1820

default_max_event_dt

get_config()

Returns:: a dict of the instance configuration parameters

static split_at_timestamp(timestamp, event, *args)

Breaks event into two arrays at index where event[‘timestamp’] > timestamp Additional arrays can be specified with kwargs and will be split at the same index

Returns:: tuple of two event halves followed by any additional arrays (in pairs)

version = 0.0.0

charge hit reconstruction

timestamp corrector

class module0_flow.reco.charge.timestamp_corrector.TimestampCorrector(**params)

Bases: h5flow.core.H5FlowStage

Corrects larpix clock timestamps due to slightly different PACMAN clock frequencies - creates a new dataset with 1:1 relationship to packets, but filled with a single value representing the true number of 10MHz clock cycles since the SYNC.

The applied correction factor is given by:

ts_corrected = (ts_original - correction_factor[0]) / (1. + correction_factor[1])

Parameters:

ts_dset_name : str, required, output dataset path
packets_dset_name : str, required, input dataset path for packets
correction : dict, optional, iogroup : [<constant offset>, <slope>] pairs

The packets_dset_name is required in the data cache along with its indices under the name of '{packets_dset_name}_index'.

Requires RunData resource in workflow.

Example config:

timestamp_corrector:
    classname: TimestampCorrector
    requires:
        - 'charge/packets'
        - name: 'charge/packets_index'
          path: 'charge/packets'
          index_only: True
    params:
        ts_dset_name: 'charge/packets_corr_ts'
        packets_dset_name: 'charge/packets'
        correction:
            1: [-9.597, 4.0021e-6]
            2: [-9.329, 1.1770e-6]

ts datatype:

id u8, unique identifier ts f8, PPS timestamp after correction for timestamp drift [ticks]

class_version = 1.0.0

correction_dtype

init(source_name)

run(source_name, source_slice, cache)

ts_dtype

hit builder

class module0_flow.reco.charge.hit_builder.HitBuilder(**params)

Bases: h5flow.core.H5FlowStage

Converts larpix data packets into hits - assigns geometric properties, filters by packet type, and performs the conversion from ADC -> mV above pedestal.

The external data files used for pedestal_file and configuration_file are searched for in the current working directory, if the paths are not specified as global paths.

Parameters:

hits_dset_name : str, required, output dataset path
packets_dset_name : str, required, input dataset path for packets
packets_index_name : str, required, input dataset path for packet index (defaults to {packets_dset_name}_index')
ts_dset_name : str, required, input dataset path for clock-corrected packet timestamps
pedestal_file : str, optional, path to a pedestal json file
configuration_file : str, optional, path to a vref/vcm config json file

packets_dset_name, ts_dset_name, and packets_index_name are required in the data cache. packets_index_name must point to the index for packets_dset_name.

Requires RunData resource in workflow.

Example config:

hit_builder:
    classname: HitBuilder
    requires:
        - 'charge/packets'
        - 'charge/packets_corr_ts'
        - name: 'charge/packets_index'
          path: 'charge/packets'
          index_only: True
    params:
        hits_dset_name: 'charge/hits'
        packets_dset_name: 'charge/packets'
        packets_index_name: 'charge/packets_index'
        ts_dset_name: 'charge/packets_corr_ts'
        pedestal_file: 'datalog_2021_04_02_19_00_46_CESTevd_ped.json'
        configuration_file: 'evd_config_21-03-31_12-36-13.json'

hits datatype:

id          u4, unique identifier per hit
px          f8, pixel x location [mm]
py          f8, pixel y location [mm]
ts          f8, PPS timestamp (corrected for clock frequency) [ticks]
ts_raw      u8, PPS timestamp [ticks]
q           f8, hit charge [mV]
iogroup     u1, io group id (PACMAN number)
iochannel   u1, io channel id (PACMAN UART number)
chipid      u1, chip id (ASIC number on PACMAN UART)
channelid   u1, channel id (channel number on ASIC)
geom        u1, unused

static charge_from_dataword(dw, vref, vcm, ped)

class_version = 1.2.0

configuration

hits_dtype

init(source_name)

load_configurations()

load_pedestals()

pedestal

run(source_name, source_slice, cache)

unique_id(iogroup, iochannel, chipid, channelid)

external trigger finder

class module0_flow.reco.charge.external_trigger_finder.ExternalTriggerFinder(**params)

Bases: h5flow.core.H5FlowStage

Extracts external triggers from raw packets

Parameters:

ext_trigs_dset_name : str, required, path to output dataset
packets_dset_name : str, required, path to input dataset containing packets
ts_dset_name : str, required, path to input dataset containing corrected packet timestamps
larpix_trigger_channels : dict of <chip key> : [<channels>] pairs, optional
pacman_trigger_enabled : bool, optional, true to extract pacman-level external triggers
pacman_trigger_word_filter : int, optional, bitmask for pacman trigger word (3 == trigger bits 0 and 1 indicate external trigger)

Both packets_dset_name and ts_dset_name are required in the data cache.

The parameter pacman_trigger_enabled configures the ExternalTriggerFinder to extract packets of packet_type == 7 as external triggers

The parameter larpix_trigger_channels configures the ExternalTriggerFinder to extract triggers on particular larpix channels as external triggers. To specify, this parameter should be a dict of <chip-key>: [<channel id>] pairs. A special chip key of ‘All’ can be used in the event that all triggers on a particular channel of any chip key should be extracted as external triggers.

Example config:

ext_trig_finder:
  classname: ExternalTriggerFinder
  requires:
    - 'charge/packets'
    - name: 'charge/packets_corr_ts'
      path: ['charge/packets', 'charge/packets_corr_ts'] # get corrected timestamps for each packet
  params:
    ext_trigs_dset_name: 'charge/ext_trigs'
    packets_dset_name: 'charge/packets'
    ts_dset_name: 'charge/packets_corr_ts'
    pacman_trigger_enabled: True
    pacman_trigger_word_filter: 2

ext_trigs datatype:

id          u8, unique identifier per event
ts          f8, corrected PPS timestamp [ticks]
ts_raw      u8, PPS timestamp [ticks]
type        i2, trigger type from PACMAN
iogroup     u1, PACMAN id

class_version = 1.0.0

default_larpix_trigger_channels

default_pacman_trigger_enabled = True

default_pacman_trigger_word_filter = 2

ext_trigs_dtype

fit(events, metadata)

Pull external triggers from hit data within each event. metadata is a dict of ts: <array of clock corrected timestamps, same shape as events>

Trigger types are inherited from the pacman trigger type bits (with pacman_trigger_enabled) or are given a value of -1 for larpix external triggers.

Returns:: a list of a list of dicts (one list for each event), each dict describes a single external trigger with the following keys: ts-trigger timestamp, type-trigger type, mask-mask for which packets within the event are included in the trigger

get_parameters(*args)

init(source_name)

larpix_trigger_channels_dtype

run(source_name, source_slice, cache)

set_parameters(**kwargs)

time dependent gain calibration

class module0_flow.reco.charge.time_dependent_gain.TimeDependentGain(**params)

Bases: h5flow.core.H5FlowStage

Applies a global, time-dependent gain correction to each hit using a cubic spline. Assumes that the time-dependent gain variation is over a long enough time scale that the gain is constant over a single event.

Uses an input .npz file containing

f: 1D array of correction (q_calib = q_raw * f)
f_err: 1D array of 1-sigma gain correction error
unix_s: 1D array of unix timestamp in [s] for each gain correction term

Parameters:

hits_dset_name: str, path to input hits dataset (only requirement is a q field)
gain_file: str, path to .npz file grab correction factors from
calib_dset_name: str, path to output dataset

Both hits_dset_name and drift_dset are required in the cache.

Requires RunData resource in workflow.

Example config:

time_dependent_gain:
    classname: TimeDependentGain
    path: module0_flow.reco.charge.time_dependent_gain
    requires:
      - 'charge/hits'
    params:
      hits_dset_name: 'charge/hits'
      calib_dset_name: 'charge/q_calib_tdg'
      gain_file: 'module0_time_dependent_gain.npz'

calib datatype:

id          u4,     unique identifier
f           f4,     calibration factor (q_calib = f * q)
q           f4,     resulting calibrated value [mV]

calib_dtype

class_version = 0.0.0

default_calib_dset_name = combined/q_calib_tdg

default_gain_file = data/module0_flow/module0_time_dependent_gain.npz

default_hits_dset_name = charge/hits

init(source_name)

run(source_name, source_slice, cache)

event builder

class module0_flow.reco.charge.event_builder.EventBuilder(**params)

Bases: h5flow.core.H5FlowStage

High-level event builder - converts raw_events with external trigger and hit associations into a high level event

Parameters:

events_dset_name : str, required, output dataset path
hits_dset_name : str, required, input dataset path for hits
ext_trigs_dset_name : str, required, input dataset path for external triggers

Both the hits_dset_name and ext_trigs_dset_name are required in the data cache.

Example config:

event_builder:
    classname: EventBuilder
    requires:
        - 'charge/hits'
        - 'charge/ext_trigs'
    params:
        events_dset_name: 'charge/events'
        hits_dset_name: 'charge/hits'
        ext_trigs_dset_name: 'charge/ext_trigs'

events datatype:

id              u8, unique identifier per event
nhit            u4, number of hits in event
q               f8, total charge in event [mV]
ts_start        f8, first external trigger or hit corrected PPS timestamp [ticks]
ts_end          f8, last external trigger of hit corrected PPS timestamp [ticks]
n_ext_trigs     u4, number of external triggers in event
unix_ts         u8, unix timestamp of event [s since epoch]

class_version = 1.0.0

events_dtype

init(source_name)

run(source_name, source_slice, cache)

charge-to-light matching

class module0_flow.reco.charge.charge2light.Charge2LightAssociation(*args: Any, **kwargs: Any)[source]

Bases: H5FlowStage

Generate references between charge events and light events. In general, matches a given light event to a given charge event if:

|light_unix_ts_second - charge_unix_ts_second| <= unix_ts_window
AND
|light_ts_10MHz - charge_ts_10MHz| <= ts_window

where *_unix_ts_second is the unix timestamp of the event in seconds and *_ts_10MHz is the timestamp in 10MHz ticks since SYNC / PPS. Creates references from both external triggers to light events as well as references from charge events to light events.

Requires the ext_trigs_dset in the data cache as well as its indices (stored under the name ext_trigs_dset + '_idcs').

Also requires RunData resource in workflow.

Example config:

charge_light_associator:
  classname: Charge2LightAssociation
  requires:
    - 'charge/ext_trigs'
    - name: 'charge/ext_trigs_idcs'
      path: 'charge/ext_trigs'
      index_only: True
  params:
    light_event_dset_name: 'light/events'
    ext_trigs_dset_name: 'charge/ext_trigs'
    unix_ts_window: 3
    ts_window: 10

class_version = '0.0.1'

default_ts_window = 1000

default_unix_ts_window = 1

finish(source_name)[source]

init(source_name)[source]

match_on_timestamp(charge_unix_ts, charge_pps_ts)[source]

run(source_name, source_slice, cache)[source]