-
Notifications
You must be signed in to change notification settings - Fork 266
Description
The stream_name (e.g. imec0.ap) is formed as "{device_kind}{device_index}.{stream_kind}" from three components:
device_kind:"imec","nidq", or"obx". Read frommeta["typeThis"], which is a semantic metadata field present in all phases.device_index: the numeric suffix from the device string (e.g."0","1"), or""for nidq. Parsed frommeta["fileName"]by splitting the original filename (e.g.imec0->"0").stream_kind:"ap"or"lf"for imec streams,""for nidq/obx. Also parsed frommeta["fileName"](the last dot-separated component of the filename stem).
| Stream type | device_kind (from typeThis) |
device_index (from fileName) |
stream_kind (from fileName) |
Resulting stream_name |
|---|---|---|---|---|
| imec AP | imec |
0 |
ap |
imec0.ap |
| imec LF | imec |
0 |
lf |
imec0.lf |
| NIDQ | nidq |
"" |
"" |
nidq |
| OneBox | obx |
0 |
"" |
obx0 |
When two files map to the same (seg_index, stream_name) key, the reader raises KeyError: key (seg_index, stream_name) is already in the signals_info_dict. This error is terse and doesn't tell the user which files collided or why. It has come up in several issues: #1606 (renamed files losing probe index), #1656 (isolated folder-per-probe), and #1511 (mixed tcat/non-tcat files). We should improve the error message to name the two colliding .meta file paths and suggest common causes. I am also experiencing this error with some IBL data, where the pipeline's automated tools create LF .meta files by rewriting AP metadata and leave fileName pointing to the AP binary, this assigns the wrong stream_kind, causing the LF stream to collide with the actual AP stream.
While we can't support all the ways in which users might modify .meta files, I think there is an improvement here. Originally, both device_index and stream_kind were parsed from the on-disk filename. Issue #1606 showed that users reorganize multi-probe recordings into separate folders and strip the probe index from filenames, which broke this parsing. PR #1608 switched to parsing meta["fileName"] instead, since it records the original path SpikeGLX wrote at recording time and is robust to renames. PR #1608 also tried deriving device_index from semantic metadata by sorting probes by their (imDatPrb_slot, imDatPrb_port, imDatPrb_dock) tuple, but that broke for isolated probe folders (issue #1656). PR #1661 reverted to parsing device_index from meta["fileName"]. The SpikeGLX team is not inclined to add a probe_index metadata field, so meta["fileName"] remains the best available source for device_index. For stream_kind, however, there was no such deliberation. It was parsed from fileName alongside device_index as a convenience, not because it was the best source. The stream_kind could instead be derived from snsApLfSy. This field is documented in the SpikeGLX metadata guides for all phases (including 3A) and encodes {AP_count},{LF_count},{SY_count}: an AP file has AP > 0, LF == 0 (e.g. 384,0,1), an LF file has AP == 0, LF > 0 (e.g. 0,384,1). Since it describes the actual saved channel composition rather than a filename convention, it is equally reliable and would make the reader a bit more robust. NIDQ and OneBox streams are unaffected since their stream_kind is already empty. Thoughts?