Skip to content

I2C support and IMU driver#124

Merged
ewowi merged 12 commits intomainfrom
tube-pir-gyro
Feb 15, 2026
Merged

I2C support and IMU driver#124
ewowi merged 12 commits intomainfrom
tube-pir-gyro

Conversation

@ewowi
Copy link
Collaborator

@ewowi ewowi commented Feb 10, 2026

Summary by CodeRabbit

  • New Features

    • System-wide IMU support (gyro/accel/gravity) and particle gravity selector (None/Random/Gyro).
    • I²C bus controls with runtime init, frequency setting and device scanning.
    • New periodic 20ms driver loop for more responsive drivers.
  • Behavior Changes

    • Per-node update synchronization to reduce races.
    • Particle gravity unified under selector; VU meter needle clamping tightened.
    • Preset click value 255 is now a no-op.
  • Bug Fixes

    • Ensure minimum sparks/power-dots in effects.
  • Chores

    • Reduced verbose runtime logging; build flag and dependency updates.

pio.ini: add MPU6050 driver

backend
=======
- NodeManager: send onUpdate to node when on/off changed
- Nodes: add sharedData.gravity, add D_MPU6050.h
- Module Drivers: add D_MPU6050.h
- D_MPU6050.h: New
- Effects: particles: add gyro
@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Caution

Review failed

The pull request is closed.

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds an MPU6050-based IMU driver and shared gravity state, synchronizes node onUpdate with layer mutex, consolidates particle gravity selection, introduces I2C scanning/init, implements delayed FS-write queueing and handler control, wires ModuleLightsControl into ModuleEffects, and schedules 20ms driver loops.

Changes

Cohort / File(s) Summary
Platform / deps
platformio.ini, firmware/esp32-s3.ini
Build flags/date adjusted; added MPU6050 lib to lib_deps; FastLED dependency hash updated; minor comment tweaks.
IMU driver & registration
src/MoonLight/Nodes/Drivers/D_IMU.h, src/MoonLight/Modules/ModuleDrivers.h
New IMUDriver (MPU6050 DMP): setup, 20ms FIFO reads → quaternion/gravity/YPR; registered and optionally allocated in ModuleDrivers.
Shared data / include
src/MoonBase/Nodes.h
Added Coord3D gravity to SharedData and included IMU driver header.
Node synchronization / logging
src/MoonBase/NodeManager.h, src/MoonBase/Nodes.cpp
NodeManager now locks node’s layer mutex around nodeClass->onUpdate(...); several verbose EXT_LOG* calls commented out.
Effects — gravity selection
src/MoonLight/Nodes/Effects/E_MoonLight.h
ParticlesEffect: replaced gyro/randomGravity with gravityType (None/Random/Gyro); UI and runtime branches updated; VUMeter needle bounds clamped to topLeft/size.
I2C init & scanning
src/MoonBase/Modules/ModuleIO.h
Adds Wire.h, SDA/SCL pin mapping per ESP32 variant, i2cFreq/i2cBus controls, Wire init/reinit, I2C scan helper and trigger, and runtime updates for I2C changes.
Delayed FS writes
src/MoonBase/SharedFSPersistence.h
Introduces sharedDelayedWrites queue + mutex, registerModule(..., delayedWriting), enableUpdateHandler/disableUpdateHandler, and writeToFSDelayed draining logic; delayed handler registration until begin().
ModuleEffects / lights wiring
src/MoonLight/Modules/ModuleEffects.h, src/main.cpp, src/MoonLight/Modules/ModuleLightsControl.h
ModuleEffects now accepts/stores ModuleLightsControl*, assigns node->moduleControl, adds UI-origin trigger to reset presets; main constructs ModuleEffects with extra arg.
Driver timing / physical layer
src/MoonLight/Layers/PhysicalLayer.h, src/MoonLight/Layers/PhysicalLayer.cpp, src/main.cpp
Added loop20msDrivers() and invoked it every 20ms from main driver task; reduced non-PSRAM maxChannels.
FastLED & other cleanups
src/MoonLight/Nodes/Drivers/D_FastLED.h, src/MoonLight/Nodes/Drivers/D__Sandbox.h, src/MoonLight/Nodes/Effects/E__Sandbox.h, src/MoonLight/Nodes/Effects/E_WLED.h, src/MoonBase/Modules/ModuleDevices.h, src/MoonBase/SharedWebSocketServer.h, interface/src/lib/components/moonbase/FieldRenderer.svelte
FastLED destructor clears channel listeners before reset; added override to example onUpdate; various effect clamping and UI click tweaks; ModuleDevices sort ties by IP; minor logging silences.

Sequence Diagram

sequenceDiagram
    participant NM as NodeManager
    participant MD as ModuleDrivers
    participant IMU as IMUDriver
    participant I2C as Wire/I2C
    participant SD as SharedData
    participant Eff as ParticlesEffect

    NM->>MD: request node allocation / enable IMU node
    MD->>IMU: instantiate & setup IMUDriver
    IMU->>I2C: init / probe MPU6050 (Wire)
    I2C-->>IMU: ACK / responses
    IMU->>IMU: DMP init & calibration
    loop every 20ms
      IMU->>I2C: read FIFO packet
      I2C-->>IMU: sensor packet
      IMU->>IMU: compute quaternion -> gravity / YPR
      IMU->>SD: update sharedData.gravity
    end
    NM->>IMU: nodeClass->onUpdate(...)  -- locked via node->layerMutex
    Eff->>SD: read sharedData.gravity (if gravityType==Gyro)
    Eff->>Eff: apply gravity to particles
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • 71 preps #104: overlaps NodeManager layer-mutex synchronization and IMU/gravity additions.
  • double buffering #84: addresses PhysicalLayer 20ms driver loop and main driver timing integration.
  • LED tubes #123: touches SharedFSPersistence/startup and handler registration related to delayed-write queueing.

Poem

🐇
I nudged the MPU with whiskered care,
Gravity shared and dancing through the air,
Wires hummed secrets, FIFO taps in time,
Particles tumble to a gyro's rhyme,
Moonlit hops — sensors make it rhyme.

🚥 Pre-merge checks | ✅ 2 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.61% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'Tube gyro pir' is vague and does not clearly convey the scope or main purpose of the substantial changes in this pull request. Use a more descriptive title that captures the primary objective—for example, 'Add IMU driver with I2C centralization and persistence refactoring' or 'Integrate MPU6050 IMU driver and centralize I2C management'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Merge Conflict Detection ✅ Passed ✅ No merge conflicts detected when merging into main

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch tube-pir-gyro

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Fix all issues with AI agents
In `@src/MoonBase/NodeManager.h`:
- Line 267: The call to nodeClass->onUpdate currently passes the full nodeState
without holding layerMutex and thus is inconsistent with the controls-value
path; wrap the on-toggle path's nodeClass->onUpdate call with layerMutex (same
scope used in the controls-value path) and normalize the argument semantics by
passing a consistent control object (e.g., construct a single-control JSON
containing only the on field or an explicit {"on": ...} object) instead of the
full nodeState so onUpdate implementations always receive the same JSON shape.

In `@src/MoonLight/Nodes/Drivers/D_MPU6050.h`:
- Line 46: The onUpdate method declaration lacks the override specifier which
risks silent signature drift; update the declaration of void onUpdate(const
Char<20>& oldValue, const JsonObject& control) in class D_MPU6050 to add the
override keyword so it matches the other overridden methods (e.g., setup,
hasOnLayout, onLayout, loop20ms) and lets the compiler validate the override.
- Around line 37-44: The method initI2S incorrectly initializes I2C (Wire.begin)
so rename the function to initI2C (and update all callers/usages) and update the
log message (EXT_LOGI) to reflect "initI2C" to avoid protocol confusion; while
here also remove or replace the hardcoded pins (Wire.begin(5, 6)) by using the
existing pin allocation mechanism or configurable pin variables (e.g., use
allocatePin or pass sda/scl parameters or read configured pins) so the function
no longer hardcodes 5 and 6 and will work across boards.
- Around line 83-84: The hasOnLayout() override currently returns true while
onLayout() is an empty override, causing unnecessary mapping-system calls;
either remove both overrides (delete the hasOnLayout() and onLayout() methods so
the node uses the base-class default of no layout handling) or implement real
layout initialization/update logic inside onLayout() and keep hasOnLayout()
true; locate the D_MPU6050 class methods hasOnLayout() and onLayout() and apply
one of these two changes (prefer removing both overrides if the driver does not
need to react to layout changes).
🧹 Nitpick comments (3)
src/MoonLight/Nodes/Drivers/D_MPU6050.h (2)

26-26: Dead code: gravityVector (public) and devStatus (private member) are unused.

  • gravityVector (line 26) is declared but never written or read. The private gravity (line 135) is the actual gravity vector used in loop20ms. Either remove gravityVector or use it.
  • The member devStatus (line 127) is shadowed by a local variable in onUpdate (line 56) and never referenced elsewhere.
Remove dead declarations
   Coord3D gyro;  // in degrees (not radians)
   Coord3D accell;
-  VectorFloat gravityVector;
 private:
   MPU6050 mpu;
 
   // MPU control/status vars
-  uint8_t devStatus;       // return status after each device operation (0 = success, !0 = error)
   uint8_t fifoBuffer[64];  // FIFO storage buffer

Also applies to: 127-127


46-81: No handling when node is turned off — MPU6050 DMP stays enabled.

onUpdate initializes and enables the DMP when control["on"] == true, but there's no branch for control["on"] == false. When the user disables this driver node, the DMP keeps running and loop20ms just returns early due to motionTrackingReady staying true. Consider disabling DMP and resetting motionTrackingReady on turn-off to save power/bus bandwidth and zero out sharedData.gravity.

Proposed off-handling
   void onUpdate(const Char<20>& oldValue, const JsonObject& control) override {
     if (!control["on"].isNull()) {
       if (control["on"] == true) {
         if (initI2C()) {
           // ... existing init code ...
         }
+      } else {
+        // Disable DMP and reset state
+        if (motionTrackingReady) {
+          mpu.setDMPEnabled(false);
+          motionTrackingReady = false;
+          sharedData.gravity = Coord3D();
+        }
       }
     }
   }
platformio.ini (1)

144-145: MPU6050 dependency is added to [env] (global) — consider scoping to [moonlight] if only MoonLight builds use it.

Placing ElectronicCats/MPU6050 @ 1.3.0 under [env] lib_deps means it's pulled and linked for every board variant, even those without MPU6050 hardware or the FT_MOONLIGHT feature flag. If this library is only used in MoonLight builds, moving it to the [moonlight] section (as done with FastLED and other specialized dependencies) would avoid unnecessary binary bloat for other firmware variants.

Also note: line 144 uses 2 spaces for indentation while line 143 uses a tab — consider standardizing to match surrounding entries.

if (nodeClass != nullptr) {
nodeClass->on = updatedItem.value.as<bool>(); // set nodeclass on/off
// EXT_LOGD(ML_TAG, " nodeclass 🔘:%d 🚥:%d 💎:%d", nodeClass->on, nodeClass->hasOnLayout(), nodeClass->hasModifier());
nodeClass->onUpdate(updatedItem.oldValue, nodeState); // custom onUpdate for the node
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Inconsistent mutex handling and argument semantics for onUpdate.

Two observations:

  1. No mutex: The controls-value path (line 282–285) holds layerMutex around onUpdate, but this new on-toggle path does not. For MPU6050Driver the hardware init may not need it, but other nodes' onUpdate could mutate shared layer state. Consider wrapping consistently:
+            xSemaphoreTake(*nodeClass->layerMutex, portMAX_DELAY);
             nodeClass->onUpdate(updatedItem.oldValue, nodeState);  // custom onUpdate for the node
+            xSemaphoreGive(*nodeClass->layerMutex);
  1. Argument type mismatch: Here nodeState (full node JSON with name, on, controls) is passed as the control parameter, while at line 284 a single control object is passed. The MPU6050Driver relies on checking control["on"] to distinguish, but this convention is implicit and fragile — a future node's onUpdate could misinterpret the JSON shape.
🤖 Prompt for AI Agents
In `@src/MoonBase/NodeManager.h` at line 267, The call to nodeClass->onUpdate
currently passes the full nodeState without holding layerMutex and thus is
inconsistent with the controls-value path; wrap the on-toggle path's
nodeClass->onUpdate call with layerMutex (same scope used in the controls-value
path) and normalize the argument semantics by passing a consistent control
object (e.g., construct a single-control JSON containing only the on field or an
explicit {"on": ...} object) instead of the full nodeState so onUpdate
implementations always receive the same JSON shape.

Comment on lines 37 to 44
bool initI2S() {
// tbd: set pins in ui!!
// allocatePin(21, "Pins", "I2S SDA");
// allocatePin(22, "Pins", "I2S SCL");
bool success = Wire.begin(5, 6);
EXT_LOGI(ML_TAG, "initI2S Wire begin %s", success ? "success" : "failure");
return success;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Method is named initI2S but initializes I2C (Wire/Wire.begin).

I²S (Inter-IC Sound) and I²C (Inter-Integrated Circuit / Wire) are different protocols. This should be initI2C to avoid confusion.

Also, I2C pins 5 and 6 are hardcoded — acknowledged by the TODO, but worth noting this will fail on many boards.

Proposed rename
-  bool initI2S() {
-    // tbd: set pins in ui!!
-    // allocatePin(21, "Pins", "I2S SDA");
-    // allocatePin(22, "Pins", "I2S SCL");
+  bool initI2C() {
+    // tbd: set pins in ui!!
+    // allocatePin(21, "Pins", "I2C SDA");
+    // allocatePin(22, "Pins", "I2C SCL");
     bool success = Wire.begin(5, 6);
-    EXT_LOGI(ML_TAG, "initI2S Wire begin %s", success ? "success" : "failure");
+    EXT_LOGI(ML_TAG, "initI2C Wire begin %s", success ? "success" : "failure");
     return success;
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
bool initI2S() {
// tbd: set pins in ui!!
// allocatePin(21, "Pins", "I2S SDA");
// allocatePin(22, "Pins", "I2S SCL");
bool success = Wire.begin(5, 6);
EXT_LOGI(ML_TAG, "initI2S Wire begin %s", success ? "success" : "failure");
return success;
}
bool initI2C() {
// tbd: set pins in ui!!
// allocatePin(21, "Pins", "I2C SDA");
// allocatePin(22, "Pins", "I2C SCL");
bool success = Wire.begin(5, 6);
EXT_LOGI(ML_TAG, "initI2C Wire begin %s", success ? "success" : "failure");
return success;
}
🤖 Prompt for AI Agents
In `@src/MoonLight/Nodes/Drivers/D_MPU6050.h` around lines 37 - 44, The method
initI2S incorrectly initializes I2C (Wire.begin) so rename the function to
initI2C (and update all callers/usages) and update the log message (EXT_LOGI) to
reflect "initI2C" to avoid protocol confusion; while here also remove or replace
the hardcoded pins (Wire.begin(5, 6)) by using the existing pin allocation
mechanism or configurable pin variables (e.g., use allocatePin or pass sda/scl
parameters or read configured pins) so the function no longer hardcodes 5 and 6
and will work across boards.

return success;
}

void onUpdate(const Char<20>& oldValue, const JsonObject& control) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Missing override keyword on onUpdate.

All other overridden methods in this class (setup, hasOnLayout, onLayout, loop20ms, destructor) use override, but onUpdate does not. Without override, the compiler won't catch signature drift if the base class changes.

Fix
-  void onUpdate(const Char<20>& oldValue, const JsonObject& control) {
+  void onUpdate(const Char<20>& oldValue, const JsonObject& control) override {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
void onUpdate(const Char<20>& oldValue, const JsonObject& control) {
void onUpdate(const Char<20>& oldValue, const JsonObject& control) override {
🤖 Prompt for AI Agents
In `@src/MoonLight/Nodes/Drivers/D_MPU6050.h` at line 46, The onUpdate method
declaration lacks the override specifier which risks silent signature drift;
update the declaration of void onUpdate(const Char<20>& oldValue, const
JsonObject& control) in class D_MPU6050 to add the override keyword so it
matches the other overridden methods (e.g., setup, hasOnLayout, onLayout,
loop20ms) and lets the compiler validate the override.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 10, 2026

@coderabbitai, the mcu6050 is not working, I wired the scanner/scl/gnd/vcc correctly. got the following logging, what is wrong?

[ 56027][D][NodeManager.h:262] onUpdate(): [💫] httpd (1524) MPU6050 driver ☸️ on: true (#2)
[ 56028][W][Wire.cpp:300] begin(): Bus already started in Master Mode.
[ 56028][I][D_MPU6050.h:42] initI2S(): [💫] httpd (1396) initI2S Wire begin success
m
0m
ALID_STATE

ransmit_receive failed: [259] ESP_ERR_INVALID_STATE
eturned Error 259
[ 56232][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 56233][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
transmit failed: [259] ESP_ERR_INVALID_STATE
[ 56334][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 56335][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
[ 56335][I][D_MPU6050.h:77] onUpdate(): [💫] httpd (848) Testing device connections MPU6050 connection failed

@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

# Read the MPU6050 driver implementation
echo "=== D_MPU6050.h implementation ==="
cat src/MoonLight/Nodes/Drivers/D_MPU6050.h

echo -e "\n\n=== Searching for Wire.begin() calls ==="
rg -n "Wire\.begin" --type cpp --type-add 'cpp:*.{h,cpp,ino}'

echo -e "\n\n=== Searching for other I2C initializations ==="
rg -n "Wire\." --type cpp --type-add 'cpp:*.{h,cpp,ino}' -C 2 | head -100

Length of output: 6271


@ewowi, I found the issue! The error "Bus already started in Master Mode" indicates that the I2C Wire interface is already initialized somewhere else (possibly by the ESP32 Arduino framework with default pins), and then your driver is trying to call Wire.begin(5, 6) again with different pins. This causes the ESP_ERR_INVALID_STATE errors.

The Problem

When Wire.begin() is called on an already-initialized I2C bus with different pin assignments, the ESP32 I2C driver gets into an invalid state and all subsequent I2C operations fail.

The Solution

You need to end the existing I2C bus before reinitializing it with your custom pins. Modify the initI2S() method in D_MPU6050.h:

bool initI2S() {
  // tbd: set pins in ui!!
  // allocatePin(21, "Pins", "I2S SDA");
  // allocatePin(22, "Pins", "I2S SCL");
  
  // End any existing I2C bus before reinitializing
  Wire.end();
  delay(10);  // Small delay to ensure bus is fully released
  
  bool success = Wire.begin(5, 6);
  EXT_LOGI(ML_TAG, "initI2S Wire begin %s", success ? "success" : "failure");
  return success;
}

The Wire.end() call will properly shut down the existing I2C bus, allowing you to reinitialize it with your custom pins (5 and 6).

Additional Checks

Also verify:

  1. Pin 5 = SDA, Pin 6 = SCL - Make sure your physical wiring matches (some boards vary)
  2. Pull-up resistors - Ensure you have 4.7kΩ pull-ups on both SDA and SCL lines (some MPU6050 modules have them built-in, some don't)
  3. Power supply - Verify the MPU6050 is getting clean 3.3V power

Try the Wire.end() fix first - this should resolve the state error!

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 10, 2026

@coderabbitai, regarding

indicates that the I2C Wire interface is already initialized somewhere else (possibly by the ESP32 Arduino framework with default pins)
Could you check the repo where that could have been done?

Wire.end(); delay(10);

Improved it a bit but still errors, please investigate:

[ 69729][D][NodeManager.h:262] onUpdate(): [💫] httpd (1524) MPU6050 driver ☸️ on: true (#2)
[ 69740][I][esp32-hal-i2c-ng.c:112] i2cInit(): Initializing I2C Master: num=0 sda=5 scl=6 freq=100000
[ 69741][I][D_MPU6050.h:44] initI2S(): [💫] httpd (1284) initI2S Wire begin success
dNonStop returned Error 259
[ 69843][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 69844][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
m(): i2cWriteReadNonStop returned Error 259
TATE
m
m
_ERR_INVALID_STATE
259
ed

@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Searching for all Wire.begin() calls in the codebase ==="
rg -n "Wire\.begin" --type-add 'code:*.{h,cpp,ino,c}' --type code

echo -e "\n\n=== Searching for any I2C initialization patterns ==="
rg -n "I2C|i2c" --type-add 'code:*.{h,cpp,ino,c}' --type code -g '!*.json' -g '!*.md' | grep -i "init\|begin\|start" | head -50

echo -e "\n\n=== Looking for other driver files that might use I2C ==="
fd -e h -e cpp "D_.*\.h" src/

echo -e "\n\n=== Checking MPU6050 driver for address and initialization sequence ==="
cat src/MoonLight/Nodes/Drivers/D_MPU6050.h | grep -A 5 -B 5 "initialize\|setDeviceID\|0x68\|0x69"

echo -e "\n\n=== Searching for MPU6050 related configurations ==="
rg -n "MPU6050" --type-add 'code:*.{h,cpp,ino,c}' --type code -C 2

Length of output: 5209


🌐 Web query:

ESP32 I2C ESP_ERR_INVALID_STATE error 259 causes

💡 Result:

259 is ESP_ERR_INVALID_STATE (0x103). In ESP-IDF/Arduino-ESP32 I2C, it generally means “the I2C driver/state machine isn’t in a usable state for this call.” Common causes:

  1. Driver not installed / not initialized for that port
  • Calling i2c_master_*() (or Arduino Wire.requestFrom()/read()) before the port has been set up with i2c_param_config() + i2c_driver_install() (or before Wire.begin(...)). The ESP-IDF I2C docs explicitly require installing the driver before doing master transactions. [1]
  • Internally, ESP-IDF checks that the per-port driver object exists (e.g., p_i2c_obj[i2c_num] != NULL); if it’s NULL, functions return ESP_ERR_INVALID_STATE. [2]
  1. Driver was deleted/deinitialized (or never reinstalled)
  • Calling I2C APIs after i2c_driver_delete() (or Arduino Wire.end() / a library deinit) without reinstalling/reinitializing first can lead to ESP_ERR_INVALID_STATE. [2]
  1. Wrong I2C port / resource conflict
  • Using the wrong i2c_port_t (ESP32 typically has two controllers) or having two components both trying to own/reconfigure the same I2C port can leave one caller operating on an uninstalled or torn-down driver state (manifesting as INVALID_STATE). (The “driver per port must be installed before use” requirement is the key underlying constraint.) [1]
  1. Arduino-ESP32 “i2c-ng” regression/bug (especially when upgrading cores)
  • There are reports on Arduino-ESP32 3.x (new I2C implementation) where transactions start failing and then i2c_master_receive/i2cRead() begin returning [259] ESP_ERR_INVALID_STATE after running for a while. If your setup is correct, this can be a core/driver issue rather than your wiring. [3]

Sources:
[1] ESP-IDF I2C driver docs (install driver before master comms). (docs.espressif.com)
[2] Notes showing ESP_ERR_INVALID_STATE returned when the I2C driver object for the port is NULL (not installed / deleted). (gist.github.com)
[3] Arduino-ESP32 issue reporting i2c_master_receive / i2cRead() failing with [259] ESP_ERR_INVALID_STATE after some runtime (3.x). (github.com)

Citations:


@ewowi, Good news: I found no other Wire/I2C initialization in the entire codebase - only your MPU6050 driver initializes I2C.

The Real Problem

Based on the logs and research, the issue is likely that Wire.end() is causing the problem, not solving it. Here's why:

ESP_ERR_INVALID_STATE occurs when the I2C driver isn't properly installed or was deleted/deinitialized. When you call Wire.end(), it deletes the I2C driver. Then Wire.begin() succeeds in starting to initialize it, but the driver may not be in a fully usable state yet when mpu.testConnection() tries to communicate immediately after.

Solution: Remove Wire.end() and Add Proper Delay

Try this updated initI2S():

bool initI2S() {
  // tbd: set pins in ui!!
  // allocatePin(21, "Pins", "I2S SDA");
  // allocatePin(22, "Pins", "I2S SCL");
  
  // Don't call Wire.end() - it deletes the driver and causes INVALID_STATE
  bool success = Wire.begin(5, 6);
  EXT_LOGI(ML_TAG, "initI2S Wire begin %s", success ? "success" : "failure");
  
  if (success) {
    delay(100);  // Give I2C bus time to stabilize
  }
  
  return success;
}

Verify Hardware with I2C Scanner

Before testing again, let's verify the MPU6050 is actually present and at the correct address. Add this temporary diagnostic function:

void scanI2C() {
  EXT_LOGI(ML_TAG, "Scanning I2C bus...");
  byte count = 0;
  for (byte i = 1; i < 127; i++) {
    Wire.beginTransmission(i);
    if (Wire.endTransmission() == 0) {
      EXT_LOGI(ML_TAG, "Found I2C device at address 0x%02X", i);
      count++;
    }
  }
  EXT_LOGI(ML_TAG, "Found %d device(s)", count);
}

Call it in onUpdate() right after initI2S() succeeds:

if (initI2S()) {
  scanI2C();  // Diagnostic - remove after testing
  mpu.initialize();
  // ... rest of code

Hardware Checklist

The MPU6050 should appear at 0x68 or 0x69 (depending on AD0 pin). Verify:

  1. Wiring:

    • GPIO 5 → SDA
    • GPIO 6 → SCL
    • GND → GND
    • 3.3V → VCC
  2. Pull-up resistors: 4.7kΩ on both SDA and SCL (check if your module has them built-in)

  3. Power: Clean 3.3V supply with stable ground

Let me know what addresses the scanner finds!

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 10, 2026

@coderabbitai, regarding

Let me know what addresses the scanner finds!

[ 18026][I][esp32-hal-i2c-ng.c:112] i2cInit(): Initializing I2C Master: num=0 sda=5 scl=6 freq=100000
[ 18027][I][D_MPU6050.h:44] initI2C(): [💫] httpd (1284) initI2C Wire begin success
[ 18128][I][D_MPU6050.h:54] scanI2C(): [💫] httpd (1284) Scanning I2C bus...
[ 24429][I][D_MPU6050.h:63] scanI2C(): [💫] httpd (1284) Found 0 device(s)
[ 24480][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 24481][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
[ 24531][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 24532][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
[ 24583][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 24584][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
[ 24634][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 24635][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
[ 24685][E][esp32-hal-i2c-ng.c:275] i2cWrite(): i2c_master_transmit failed: [259] ESP_ERR_INVALID_STATE
[ 24736][E][esp32-hal-i2c-ng.c:369] i2cWriteReadNonStop(): i2c_master_transmit_receive failed: [259] ESP_ERR_INVALID_STATE
[ 24737][E][Wire.cpp:520] requestFrom(): i2cWriteReadNonStop returned Error 259
[ 24737][I][D_MPU6050.h:99] onUpdate(): [💫] httpd (916) Testing device connections MPU6050 connection failed

@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

@ewowi, The scanner found 0 devices, which means the MPU6050 isn't responding at all on the I2C bus. The root cause is likely GPIO pin selection. GPIO 5 is a strapping pin on ESP32, which must be high (1) during boot. Using strapping pins for I2C can cause initialization and communication issues.

Critical Issue: GPIO Pin Choice

ESP32 allows any GPIO pin to be configured as I2C SDA and SCL, but the default I2C pins for ESP32 with Arduino IDE are GPIO 22 (SCL) and GPIO 21 (SDA). These are the most reliable pins for I2C communication.

Immediate Solution: Use Standard I2C Pins

Change your initI2C() function to use the standard pins:

bool initI2C() {
  // Use standard ESP32 I2C pins
  // GPIO 21 = SDA, GPIO 22 = SCL
  bool success = Wire.begin(21, 22);
  EXT_LOGI(ML_TAG, "initI2C Wire begin %s", success ? "success" : "failure");
  
  if (success) {
    delay(100);  // Give I2C bus time to stabilize
  }
  
  return success;
}

Rewire your MPU6050:

  • GPIO 21 → SDA
  • GPIO 22 → SCL
  • GND → GND
  • 3.3V → VCC

Additional Troubleshooting

If the scanner still finds 0 devices after switching pins:

  1. Check power with multimeter: Verify the MPU6050 VCC pin has 3.3V
  2. Verify connections: Use a continuity tester to ensure wires are properly connected
  3. Test MPU6050 on another device: Try connecting it to an Arduino/Teensy with a working I2C scanner
  4. Check for shorts: Ensure SDA and SCL aren't shorted together or to GND/VCC
  5. Use pull-up resistors of 3.3kΩ or 4.7kΩ if your MPU6050 module doesn't have them built-in

The fact that the scanner took 6+ seconds and then failed with ESP_ERR_INVALID_STATE suggests the I2C bus couldn't communicate at all — typically a sign of wrong pins, disconnected hardware, or a dead module.

Try the pin change first and let me know what the scanner finds!

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 10, 2026

@coderabbitai,
I use this board: https://www.seeedstudio.com/XIAO-ESP32S3-p-5627.html

Which has pin 5 and 6, please check their website and let me know if 5,6 is indeed right

@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🌐 Web query:

Seeed XIAO ESP32S3 I2C pins GPIO pinout

💡 Result:

For the Seeed Studio XIAO ESP32S3 / XIAO ESP32S3 Sense, the default I2C pins are:

  • SDA → GPIO5
  • SCL → GPIO6 [1][2][3]

Note: the Sense camera module uses a different I2C pair:

  • Camera SDA → GPIO40
  • Camera SCL → GPIO39 [4]

Sources: [1] [2] [3] [4]


@ewowi, Good news! Your pin configuration is correct

The Seeed XIAO ESP32S3 uses GPIO 5 (SDA) and GPIO 6 (SCL) as the default I2C pins, so your code is using the right pins.

Since the pins are correct but the scanner found 0 devices, the issue is likely one of the following:

Hardware Issues to Check

  1. Physical connections:

    • Double-check all 4 wires (SDA, SCL, VCC, GND) are firmly seated
    • Try different jumper wires (bad wires are common)
  2. MPU6050 module power:

    • Verify the module's power LED is ON
    • Measure voltage at the MPU6050's VCC pin (should be 3.3V)
  3. Pull-up resistors:

    • The XIAO ESP32S3 does NOT have built-in I2C pull-ups
    • Your MPU6050 module MUST have pull-up resistors (most do, but verify)
    • If missing, add external 4.7kΩ resistors from SDA→3.3V and SCL→3.3V
  4. Test the MPU6050 module:

    • Try the module on another board/Arduino if possible
    • The module might be defective

Software Issue: Arduino-ESP32 Core Bug

The ESP_ERR_INVALID_STATE error you're seeing is a known issue with Arduino-ESP32 3.x's new I2C driver. There are reports of i2c_master_receive failing with error 259 (ESP_ERR_INVALID_STATE) in Arduino-ESP32 3.x.

Try adding a small delay after Wire.begin():

bool initI2C() {
  bool success = Wire.begin(5, 6);  // Correct pins for XIAO ESP32S3
  EXT_LOGI(ML_TAG, "initI2C Wire begin %s", success ? "success" : "failure");
  
  if (success) {
    delay(200);  // Longer delay for I2C bus stabilization
    Wire.setClock(100000);  // Explicitly set to 100kHz
  }
  
  return success;
}

What Arduino-ESP32 core version are you using? (Check platform.ini or Arduino IDE board manager)

@ewoudwijma
Copy link

ewoudwijma commented Feb 10, 2026

@coderabbitai,

This is my mcu6050 board , do I need to short wire anything?

image

I am on arduino v3.3.5

@coderabbitai
Copy link

coderabbitai bot commented Feb 10, 2026

Tip

For best results, initiate chat on the files or code changes.

@ewoudwijma, Great photo! I can see your GY-521 MPU6050 board clearly.

What I See in the Image

The Comment image shows a blue GY-521 MPU6050 breakout board with:

  • Green LED is ON - This confirms the module is receiving power correctly
  • Pin headers: VCC, GND, SCL, SDA, XDA, XCL, AD0, INT
  • Surface mount resistors visible (pull-ups are built into this module)
  • Wires connected to the header

The AD0 Pin - Important!

Yes, you should connect the AD0 pin! Here's why:

The AD0 pin controls the I2C address:

  • AD0 = GND (or floating) → Address 0x68 (default)
  • AD0 = VCC → Address 0x69

Right now, your AD0 pin appears to be floating (not connected), which can cause unreliable I2C communication.

Solution: Connect the AD0 pin to GND for the standard 0x68 address.

Arduino-ESP32 3.0.5 I2C Issues

You're on Arduino-ESP32 3.0.5, which has known I2C bugs causing ESP_ERR_INVALID_STATE errors. This is likely contributing to your problem.

Updated Code to Try

Update your initI2C() and onUpdate() methods:

bool initI2C() {
  bool success = Wire.begin(5, 6);  // XIAO ESP32S3 I2C pins
  EXT_LOGI(ML_TAG, "initI2C Wire begin %s", success ? "success" : "failure");
  
  if (success) {
    Wire.setClock(50000);  // Lower clock speed to 50kHz for stability
    delay(200);  // Longer stabilization delay
  }
  
  return success;
}

void onUpdate(const Char<20>& oldValue, const JsonObject& control) {
  EXT_LOGI(ML_TAG, "onUpdate %s", oldValue.charArr);
  if (control["n"].as<Char<20>>() == "on" && nodeState[control["n"].as<Char<20>>()].as<bool>()) {
    if (initI2C()) {
      scanI2C();  // Keep the scanner for now
      
      // Explicitly set MPU6050 address to 0x68
      mpu.setAddr(MPU6050_ADDRESS_AD0_LOW);  // or just use default
      mpu.initialize();
      
      delay(100);
      
      if (mpu.testConnection()) {
        EXT_LOGI(ML_TAG, "MPU6050 connection successful at address 0x68");
        // ... rest of DMP setup
      } else {
        EXT_LOGI(ML_TAG, "MPU6050 connection failed - check AD0 wiring");
      }
    }
  }
}

Wiring Checklist

XIAO ESP32S3 → MPU6050:

  • GPIO 5 → SDA
  • GPIO 6 → SCL
  • 3.3V → VCC
  • GND → GND
  • GND → AD0Add this connection!

Next Steps

  1. Connect AD0 to GND
  2. Lower I2C clock to 50kHz (more stable with ESP32 3.x)
  3. Run the scanner again and let me know if it finds the device at 0x68

If it still doesn't work after grounding AD0, we may need to try Arduino-ESP32 v2.x or use a workaround for the 3.x I2C driver bug.

Let me know what the scanner shows after connecting AD0!

Change MPU6050 driver to IMU driver and add board control (MPU6050 and BMI160)
I2S -< I2C, including safety delays - WIP
add scanI2C
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Fix all issues with AI agents
In `@src/MoonBase/Nodes.h`:
- Around line 347-348: The field Coord3D gravity likely uses integer components
and will truncate the MPU6050 float gravity vector; update the type or handling:
replace Coord3D gravity with a float-based struct (e.g., VectorFloat or a new
Coord3DFloat) or store gravity as an array of floats and ensure any code using
gravity performs the proper float reads/conversions (see related D_IMU.h
handling), or if you must keep Coord3D, add clear documentation and explicit
scaling/quantization conversions where gravity is assigned so values in
[−1.0,1.0] are preserved correctly.

In `@src/MoonLight/Modules/ModuleDrivers.h`:
- Line 109: The build fails because the code calls getNameAndTags<IMUDriver>()
in addControlValue but the actual class in D_IMU.h is defined as MPU6050Driver;
update the template type references to the real class name (replace IMUDriver
with MPU6050Driver) wherever getNameAndTags<IMUDriver>() and related uses appear
(e.g., the addControlValue(control, getNameAndTags<...>()) calls around the IMU
driver registrations), or alternatively rename the class in D_IMU.h to
IMUDriver—ensure consistency between the class declaration (MPU6050Driver) and
all template/type usages (getNameAndTags and addControlValue).

In `@src/MoonLight/Nodes/Drivers/D_IMU.h`:
- Around line 41-56: In initI2C(), fix the comment typos (replace "I2S" with
"I2C"), remove the hardcoded Wire.begin(5, 6) and instead read user-configurable
I2C SDA/SCL pins (use the existing UI control pattern such as
addControl/allocatePin or the pin config variables you have) and pass those pin
values into Wire.begin, and correct the clock mismatch by either setting
Wire.setClock(100000) to match the "100kHz" comment or update the comment to
"50kHz" if 50000 is intended; ensure references to Wire.begin and Wire.setClock
and the initI2C() function are updated accordingly.
- Around line 71-112: onUpdate is performing long blocking initialization (delay
calls, mpu.dmpInitialize, CalibrateAccel, CalibrateGyro) on the HTTP/UI thread
and logging errors with EXT_LOGI; move the heavy work out of onUpdate into a
background task/thread or schedule a non-blocking state-machine so onUpdate only
enqueues/requests initialization and returns immediately, call a new InitIMUTask
(or similar) that runs initI2C(), scanI2C(), mpu.initialize(),
mpu.dmpInitialize(), CalibrateAccel(6), CalibrateGyro(6) and sets
motionTrackingReady when done; also change the two incorrect log calls that
currently use EXT_LOGI for failures (the mpu.dmpInitialize error and the
mpu.testConnection failure) to use EXT_LOGW or EXT_LOGE so errors are logged at
the proper severity.
- Around line 1-20: The class is named MPU6050Driver but other code expects
IMUDriver; rename the class MPU6050Driver to IMUDriver (and rename its
destructor ~MPU6050Driver to ~IMUDriver) so the symbol matches ModuleDrivers.h
references (IMUDriver), and update the file header `@file` from D_MPU6050.h to
D_IMU.h to reflect the real filename; alternatively you may add a type alias
(using IMUDriver = MPU6050Driver) but be sure the destructor/name usages match
across the module.
- Around line 126-153: The gravity VectorFloat components (gravity.x, gravity.y,
gravity.z) are being assigned to sharedData.gravity which is a Coord3D with int
fields, causing truncation; update the code so gravity is stored with proper
precision—either change the SharedData.gravity type from Coord3D (int) to a
float-based struct (e.g., VectorFloat/float fields) or scale the float values
before assignment (e.g., multiply gravity.x/y/z by a factor like 1000 and
convert to int) and document the unit change; locate assignments in the block
handling board == 0 (mpu.dmpGetGravity -> sharedData.gravity.x/y/z) and apply
the chosen fix consistently wherever sharedData.gravity is used.
🧹 Nitpick comments (2)
src/MoonLight/Nodes/Drivers/D_IMU.h (2)

86-86: Local devStatus shadows the private member on Line 162.

Line 86 declares uint8_t devStatus as a local variable, which shadows the private devStatus member (Line 162). The private member is never used elsewhere. Either remove the private member or reuse it here by dropping the local uint8_t declaration.

Proposed fix: remove unused private member
  // MPU control/status vars
-  uint8_t devStatus;       // return status after each device operation (0 = success, !0 = error)
   uint8_t fifoBuffer[64];  // FIFO storage buffer

156-156: Destructor doesn't clean up I2C or DMP state.

When the driver node is removed, the destructor doesn't call mpu.setDMPEnabled(false) or Wire.end(). If the node is destroyed and recreated, this could leave the I2C bus in an inconsistent state — particularly relevant given the I2C initialization issues discussed in the PR comments.

Proposed cleanup
-  ~MPU6050Driver() override {};  // e.g. to free allocated memory
+  ~MPU6050Driver() override {
+    if (motionTrackingReady) {
+      mpu.setDMPEnabled(false);
+    }
+  }

Comment on lines +347 to +348

Coord3D gravity;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Coord3D gravity may not be the right type for gravity data.

As noted in the D_IMU.h review, the MPU6050 gravity vector contains float values in the range [–1.0, 1.0]. If Coord3D uses integer fields (e.g., int16_t), this will silently truncate to 0/±1. Consider using a float-based struct (e.g., VectorFloat) or documenting the expected scale/encoding if you intend to pre-scale the values.

🤖 Prompt for AI Agents
In `@src/MoonBase/Nodes.h` around lines 347 - 348, The field Coord3D gravity
likely uses integer components and will truncate the MPU6050 float gravity
vector; update the type or handling: replace Coord3D gravity with a float-based
struct (e.g., VectorFloat or a new Coord3DFloat) or store gravity as an array of
floats and ensure any code using gravity performs the proper float
reads/conversions (see related D_IMU.h handling), or if you must keep Coord3D,
add clear documentation and explicit scaling/quantization conversions where
gravity is assigned so values in [−1.0,1.0] are preserved correctly.

Comment on lines 41 to 56
bool initI2C() {
// tbd: set pins in ui!!
// allocatePin(21, "Pins", "I2S SDA");
// allocatePin(22, "Pins", "I2S SCL");
Wire.end();
delay(10);
bool success = Wire.begin(5, 6);
EXT_LOGI(ML_TAG, "initI2C Wire begin %s", success ? "success" : "failure");

if (success) {
delay(200); // Give I2C bus time to stabilize
Wire.setClock(50000); // Explicitly set to 100kHz
}

return success;
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Hardcoded I2C pins and comment/code mismatch on clock speed.

Three items here:

  1. Lines 43–44: Comments reference "I2S" — should be "I2C".
  2. Line 47: Wire.begin(5, 6) hardcodes GPIO5/GPIO6 (XIAO ESP32S3). This will break on other boards. Consider making the pins configurable via UI controls (the commented-out addControl approach on lines 42–44 is the right direction).
  3. Line 52: Comment says "Explicitly set to 100kHz" but the code sets 50000 (50 kHz).
Fix the comment typos
-    // tbd: set pins in ui!!
-    // allocatePin(21, "Pins", "I2S SDA");
-    // allocatePin(22, "Pins", "I2S SCL");
+    // tbd: set pins in ui!!
+    // allocatePin(21, "Pins", "I2C SDA");
+    // allocatePin(22, "Pins", "I2C SCL");
-      Wire.setClock(50000);  // Explicitly set to 100kHz
+      Wire.setClock(50000);  // Explicitly set to 50kHz (lower for reliability)
🤖 Prompt for AI Agents
In `@src/MoonLight/Nodes/Drivers/D_IMU.h` around lines 41 - 56, In initI2C(), fix
the comment typos (replace "I2S" with "I2C"), remove the hardcoded Wire.begin(5,
6) and instead read user-configurable I2C SDA/SCL pins (use the existing UI
control pattern such as addControl/allocatePin or the pin config variables you
have) and pass those pin values into Wire.begin, and correct the clock mismatch
by either setting Wire.setClock(100000) to match the "100kHz" comment or update
the comment to "50kHz" if 50000 is intended; ensure references to Wire.begin and
Wire.setClock and the initI2C() function are updated accordingly.

ModuleIO
- Include Wire
- Add controls i2cFreq and i2cBus
- default assign sda to pin 21 and scl to pin 22
- readPins: add I2C pin assignment
- add updateDevices: Scanning I2C bus and send frequency and addresses to UI

IMU driver
- remove initI2C and scan I2C
- gravity xyz float to int correction - WIP
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Line 222: The i2cFreq control created via addControl(controls, "i2cFreq", ...)
uses a max of 65534 which is below typical Wire.getClock() values (100000,
400000); update the max bound on the "i2cFreq" control in ModuleIO.h to a larger
value (e.g., 1000000 or UINT32_MAX) so real I2C clock frequencies aren't
clamped, ensuring the chosen max fits the addControl parameter type (int vs.
unsigned) and adjusting the type if necessary.
- Around line 928-929: Remove the sentinel fake I2C device being appended with
address 255: stop creating the JsonObject via
newState["i2cBus"].as<JsonArray>().add<JsonObject>() and assigning
i2cDevice["address"] = 255 so the i2cBus array only contains real scan results;
if a non-empty-array indicator is needed, handle that in the UI layer instead of
adding a sentinel entry (look for usages of i2cDevice/newState["i2cBus"] in
ModuleIO.h and related scan/serialize functions to ensure no other code depends
on the sentinel).
- Around line 819-826: readPins() can call Wire.begin() repeatedly without
cleanup causing "Bus already started in Master Mode" on ESP32; before calling
Wire.begin(pinI2CSDA, pinI2CSCL) ensure you call Wire.end() if Wire is already
initialized (check Wire.isEnabled() or track a local/field flag), then call
Wire.begin(...) and log via EXT_LOGI/EXT_LOGE as before; also re-enable the
stabilization delay and explicit clock set (uncomment the delay(200) and
Wire.setClock(...) lines and set the clock to the recommended value for
Arduino-ESP32 3.x) so initI2C sequence (pins pinI2CSDA/pinI2CSCL, Wire.begin/
Wire.end, delay, Wire.setClock) is safe when readPins() runs repeatedly.
- Around line 905-908: Loop1s() is calling updateDevices() every second which
triggers a full I2C bus scan and causes blocking/traffic; change this so
updateDevices() is not invoked from loop1s(); instead call updateDevices() once
after a successful Wire.begin() (e.g., in readPins() after initializing I2C when
pinI2CSCL and pinI2CSDA are valid) and cache the discovered devices, and provide
an explicit on-demand way to rescan if needed; update references to pinI2CSCL
and pinI2CSDA checks so the scan only happens once after initialization and
remove the per-second call in loop1s().
🧹 Nitpick comments (4)
src/MoonLight/Nodes/Drivers/D_IMU.h (4)

22-27: gravityVector (Line 26) is declared but never used.

The public VectorFloat gravityVector member is never read or written in this file. The private gravity member (Line 139) is what's actually used for DMP calculations. Either remove gravityVector or wire it up to serve its intended purpose.


55-55: Local devStatus on Line 55 shadows the private member on Line 131.

The uint8_t devStatus declared inside onUpdate shadows the class member of the same name, leaving the member uninitialized and unused. Remove the member declaration on Line 131 since the variable is only needed locally.

Proposed fix
  // MPU control/status vars
-  uint8_t devStatus;       // return status after each device operation (0 = success, !0 = error)
   uint8_t fifoBuffer[64];  // FIFO storage buffer

Also applies to: 131-131


100-105: gyro values also suffer float→int truncation (fractional degrees lost).

ypr[i] * 180 / M_PI produces floats (e.g., 45.73°) that truncate to integers when assigned to Coord3D (int fields). This loses sub-degree resolution. The gravity scaling via INT16_MAX (Lines 103–105) addresses the equivalent issue for gravity — consider a similar scaling approach for gyro, or document that integer-degree precision is intentional.


96-121: Redundant re-parsing of quaternion, accel, and gravity (Lines 111–113) with only a comment as justification.

The comment "needed to repeat the following 3 lines (yes if you look at the output: otherwise not 0)" suggests an empirical workaround. The DMP packet is deterministic — re-parsing the same fifoBuffer should yield the same results. This likely masks an issue with FIFO alignment or a stale-buffer read. Consider investigating the root cause rather than doubling the parsing work every 20ms cycle.

Comment on lines 928 to 929
JsonObject i2cDevice = newState["i2cBus"].as<JsonArray>().add<JsonObject>();
i2cDevice["address"] = 255;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Sentinel entry with address 255 pollutes the device list.

A fake device at address 255 is appended after the real scan results. This is not a valid I2C address (7-bit range is 0–127) and will appear as a spurious device in the UI. If this is meant to ensure the i2cBus array is non-empty or signal end-of-list, consider handling that in the UI layer instead.

Proposed fix: remove the sentinel
     EXT_LOGI(ML_TAG, "Found %d device(s)", count);
-    JsonObject i2cDevice = newState["i2cBus"].as<JsonArray>().add<JsonObject>();
-    i2cDevice["address"] = 255;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
JsonObject i2cDevice = newState["i2cBus"].as<JsonArray>().add<JsonObject>();
i2cDevice["address"] = 255;
// Lines 928-929 removed - sentinel entry deleted
🤖 Prompt for AI Agents
In `@src/MoonBase/Modules/ModuleIO.h` around lines 928 - 929, Remove the sentinel
fake I2C device being appended with address 255: stop creating the JsonObject
via newState["i2cBus"].as<JsonArray>().add<JsonObject>() and assigning
i2cDevice["address"] = 255 so the i2cBus array only contains real scan results;
if a non-empty-array indicator is needed, handle that in the UI layer instead of
adding a sentinel entry (look for usages of i2cDevice/newState["i2cBus"] in
ModuleIO.h and related scan/serialize functions to ensure no other code depends
on the sentinel).

Module IO
- setup, onUpdate: i2cFreq is modifyable (in kHz)
- setBoardPresetDefaults: i2c default 21,22 on esp32d0 and 8, 9 on others - wip
- readPins: reinit Wire after i2c pins changed, using freq, and updateDevices(not every s)

IMU module
- remove gravityVector
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Around line 813-836: Reset pinI2CSDA and pinI2CSCL to UINT8_MAX before
scanning the _state.data["pins"] array so stale GPIO values don't persist when a
preset has no I2C pins; in the function/block that iterates over pins (the loop
referencing pinObject["usage"] and setting pinI2CSDA/pinI2CSCL), explicitly set
pinI2CSDA = UINT8_MAX and pinI2CSCL = UINT8_MAX immediately prior to the for
loop, then keep the existing conditional Wire.begin(...) logic that checks for
UINT8_MAX.

In `@src/MoonLight/Nodes/Drivers/D_IMU.h`:
- Around line 40-80: Replace the hardcoded i2cInited = true in onUpdate with a
real I2C readiness check: query the ModuleIO state (e.g. moduleIO->pinI2CSDA
and/or moduleIO->pinI2CSCL != UINT8_MAX or a Wire.begin() success flag) to
determine if I2C is initialized before proceeding with MPU6050 initialization;
update the i2cInited variable accordingly and short-circuit the MPU setup (and
log a warning) when the check fails (references: onUpdate, i2cInited,
moduleIO->pinI2CSDA, Wire.begin).
- Around line 90-117: The code calls mpu.dmpGetLinearAccel(&aaReal, &aa,
&gravity) using aa which is never populated; restore or add a call to
mpu.dmpGetAccel(&aa, fifoBuffer) after obtaining the FIFO packet/quaternion
(e.g., after mpu.dmpGetQuaternion(&q, fifoBuffer)) so aa is filled before
dmpGetLinearAccel, and remove/adjust the commented duplicate lines accordingly
to ensure accell.x/y/z come from a valid aaReal.
🧹 Nitpick comments (1)
src/MoonBase/Modules/ModuleIO.h (1)

839-840: pinI2CSDA / pinI2CSCL are in the public section — consider moving to private.

These members are declared between the public readPins() / updateDevices() methods and the explicit private: label on line 943. If external access from the IMU driver is needed (e.g., to verify pin config), keep them public intentionally; otherwise move them below line 943.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 11, 2026

@coderabbitai,
can you do a thorough review of D_IMU.h and ModuleIO.h based on the changes in this PR. Especially focus on the decision to move the Wire code into ModuleIO.h making it a generic solution for all I2C devices. Is this the right approach. Also include a recommendation for default SDA/SCL pins for all supported ESP32 MCU's. Include follow up recommendations, including:

  • The i2cBus control in ModuleIO.h now only shows an address, can we also add the name of the device? Should the IMU driver node tell that to the IO module? How do we know which address belongs to which module? Can the IMU module query for this in the IMU board
  • Als add a proposal how to add support for the BMI160 IMU. We now use ElectronicCats/MPU6050 (see platformio.ini) for MPU650 support, do we need another library for the BMI160 - and for each future supported IMU? would be nice if there is one generic library for all IMU boards

Also make a recommendation how to do the same with SPI (generic in Module IO, other modules can use it).

@coderabbitai
Copy link

coderabbitai bot commented Feb 11, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Reading D_IMU.h ==="
cat src/MoonLight/Nodes/Drivers/D_IMU.h

echo -e "\n\n=== Reading ModuleIO.h ==="
cat src/MoonBase/Modules/ModuleIO.h

echo -e "\n\n=== Reading platformio.ini to see library dependencies ==="
cat platformio.ini | grep -A 20 "lib_deps"

echo -e "\n\n=== Checking for other I2C device drivers ==="
fd -e h "D_.*\.h" src/MoonLight/Nodes/Drivers/ -x echo "File: {}" \; -x head -30 {}

echo -e "\n\n=== Looking for ESP32 board definitions and pin configurations ==="
rg -n "ESP32|ARDUINO_" platformio.ini

echo -e "\n\n=== Searching for existing SPI usage in the codebase ==="
rg -n "SPI\.|#include.*SPI\.h" --type cpp --type-add 'cpp:*.{h,cpp}' -C 2

Length of output: 61352


🌐 Web query:

ESP32 ESP32-S2 ESP32-S3 ESP32-C3 ESP32-C6 ESP32-P4 default I2C pins GPIO

💡 Result:

On ESP32-family SoCs there aren’t “fixed” I2C pins in hardware—the I2C signals are routed through the GPIO matrix, so you can map SDA/SCL to many usable GPIOs (subject to each pin’s constraints). [7]

What people call “default I2C pins” usually means the defaults chosen by a software platform/board variant (most commonly Arduino-ESP32 Wire.begin() with no pin arguments).

Arduino-ESP32 (Wire) default SDA/SCL by generic SoC variant

SoC Default SDA Default SCL Notes / source
ESP32 GPIO21 GPIO22 Arduino-ESP32 docs (Generic ESP32). [1]
ESP32-S2 GPIO8 GPIO9 Mentioned as coming from pins_arduino.h for ESP32-S2. [2]
ESP32-S3 GPIO8 GPIO9 Shown from variants/esp32s3/pins_arduino.h. [3]
ESP32-C3 GPIO8 GPIO9 Stated as default for “ESP32C3 Dev Module” in Arduino-ESP32. [4]
ESP32-C6 GPIO23 GPIO22 Shown as variants/esp32c6/pins_arduino.h defaults; also notes LP I2C pins SDA1=GPIO6, SCL1=GPIO7. [5]
ESP32-P4 (no single universal default) (no single universal default) Boards commonly pick something like SDA=GPIO7 / SCL=GPIO8 (example ESP32-P4 boards/docs). [6]

Reminder (all chips above)

If you don’t want the defaults, set pins explicitly (Arduino: Wire.begin(sda, scl) / Wire.setPins(sda, scl); ESP-IDF: configure sda_io_num / scl_io_num). ESP-IDF’s I2C setup is explicitly pin-config driven. [8]


🌐 Web query:

BMI160 Arduino ESP32 library IMU accelerometer gyroscope

💡 Result:

Good Arduino/ESP32 libraries for the Bosch BMI160 (accel + gyro)

  1. hanyazou / BMI160-Arduino (API header: BMI160Gen.h)
  • Supports SPI and I2C; example shows BMI160.begin(...SPI_MODE...) and ...I2C_MODE, 0x68) and basic readGyro() usage. [1]
  • I2C note: tie SDO/SA0 to GND for the default address (example uses 0x68). [1]
  1. DFRobot_BMI160 (easy I2C-focused library; includes step counter helpers)
  • Provides I2cInit() and getSensorData() methods; documents 0x68 when SDIO is GND and 0x69 as the alternate address. [2]
  • README lists FireBeetle-ESP32 as “Work Well” (so it’s known to run on ESP32-class boards). [2]
  1. EmotiBit_BMI160 (Arduino Library Manager listing exists)
  • Published as “EmotiBit BMI160”; supports BMI160 (+ BMM150) and is based on the CurieIMU-style API. [3]

Minimal wiring (I2C, typical ESP32)

  • VCC → 3.3V, GND → GND, SDA/SCL → ESP32 SDA/SCL
  • Set address with SDO/SA0: low → 0x68, high → 0x69 (per DFRobot docs). [2]

Minimal code (hanyazou / BMI160Gen.h, I2C)

`#include` <Wire.h>
`#include` <BMI160Gen.h>

const int i2c_addr = 0x68;

void setup() {
  Serial.begin(115200);
  Wire.begin(); // (On ESP32 you can also pass SDA,SCL pins if needed)
  BMI160.begin(BMI160GenClass::I2C_MODE, i2c_addr);  // shown in repo example [1]
}

void loop() {
  int gx, gy, gz;
  BMI160.readGyro(gx, gy, gz); // shown in repo example [1]
  Serial.printf("g: %d %d %d\n", gx, gy, gz);
  delay(200);
}

Sources: [1] [2] [3]


🌐 Web query:

Arduino unified IMU library MPU6050 BMI160 LSM6DS3 multi-sensor support

💡 Result:

  • Closest thing to a “unified” Arduino IMU API: Adafruit’s Unified Sensor abstraction (Adafruit_Sensor) which standardizes reads into sensors_event_t across many Adafruit drivers. (arduinolibraries.info)
  • MPU6050: Adafruit_MPU6050 uses Adafruit’s stack (BusIO + Unified Sensor) and exposes accel/gyro (and temp). (learn.adafruit.com)
  • LSM6DS3 family: Adafruit’s Adafruit_LSM6DS library supports multiple LSM6DS variants (including LSM6DS33 and others) and also uses Unified Sensor. (learn.adafruit.com)
  • BMI160: There isn’t a commonly-used Arduino library for BMI160 that plugs into Adafruit Unified Sensor the same way; typical options are vendor/community drivers like DFRobot_BMI160 or EmotiBit_BMI160, with their own APIs. (arduinolibraries.info)

Practical multi-sensor approach (MPU6050 + LSM6DS3 + BMI160):

  1. Use Adafruit Unified Sensor for MPU6050 + LSM6DS3*, so your code reads both via a similar getEvent() pattern. (learn.adafruit.com)
  2. Wrap your BMI160 driver in a small adapter that outputs the same fields (accel/gyro/temp) in a struct compatible with your unified path (or mimic Adafruit’s sensors_event_t). (arduinolibraries.info)

If you tell me which BMI160 breakout you’re using (DFRobot? Bosch? other) and whether you need I2C or SPI, I can point you to the best-maintained driver + sketch an adapter interface.

Citations:


@ewowi, I've completed a thorough analysis of the I2C centralization approach, researched ESP32 pin defaults, and explored BMI160 and generic IMU library options. Here's my comprehensive review:

Architectural Review: I2C Centralization in ModuleIO.h

Is the centralized approach correct?

Yes, this is an excellent architectural decision! The centralization of I2C in ModuleIO.h provides:

  1. Single Source of Truth: One place manages Wire initialization, pin configuration, and bus state
  2. Prevents conflicts: Avoids multiple drivers calling Wire.begin() with different pins
  3. Runtime reconfiguration: Pin changes trigger automatic Wire reinitialization
  4. Bus discovery: Centralized updateDevices() scans all I2C devices once
  5. Frequency control: Single i2cFreq parameter affects all I2C devices

Current Implementation Quality

ModuleIO.h I2C Management:

  • ✅ Pin reading from UI configuration (pin_I2C_SDA, pin_I2C_SCL)
  • ✅ Wire.end() before reinit to clean up previous state
  • ✅ Frequency configuration with Wire.setClock()
  • ✅ Device scanning with updateDevices()
  • ⚠️ Issue: i2cBus only shows addresses, not device names

D_IMU.h Integration:

  • ✅ Removed I2C initialization logic (no longer calls Wire.begin)
  • ✅ Relies on Module IO for bus setup
  • ⚠️ Issue: bool i2cInited = true; // todo: check in moduleIO if successfull - hardcoded, should actually check ModuleIO state

📌 Recommendation 1: Default I2C Pins for All ESP32 Variants

Based on Arduino-ESP32 platform defaults, update the default pin configuration in ModuleIO.h:

// In setBoardPresetDefaults() for board_none (default case):
`#ifdef` CONFIG_IDF_TARGET_ESP32
  pinAssigner.assignPin(21, pin_I2C_SDA);   // ESP32 classic
  pinAssigner.assignPin(22, pin_I2C_SCL);
`#elif` defined(CONFIG_IDF_TARGET_ESP32S2)
  pinAssigner.assignPin(8, pin_I2C_SDA);    // ESP32-S2
  pinAssigner.assignPin(9, pin_I2C_SCL);
`#elif` defined(CONFIG_IDF_TARGET_ESP32S3)
  pinAssigner.assignPin(8, pin_I2C_SDA);    // ESP32-S3
  pinAssigner.assignPin(9, pin_I2C_SCL);
`#elif` defined(CONFIG_IDF_TARGET_ESP32C3)
  pinAssigner.assignPin(8, pin_I2C_SDA);    // ESP32-C3
  pinAssigner.assignPin(9, pin_I2C_SCL);
`#elif` defined(CONFIG_IDF_TARGET_ESP32C6)
  pinAssigner.assignPin(23, pin_I2C_SDA);   // ESP32-C6
  pinAssigner.assignPin(22, pin_I2C_SCL);
`#elif` defined(CONFIG_IDF_TARGET_ESP32P4)
  pinAssigner.assignPin(7, pin_I2C_SDA);    // ESP32-P4 (common board default)
  pinAssigner.assignPin(8, pin_I2C_SCL);
`#else`
  pinAssigner.assignPin(21, pin_I2C_SDA);   // Fallback
  pinAssigner.assignPin(22, pin_I2C_SCL);
`#endif`

Note: Your current code already handles ESP32 vs others, but could be more explicit for C6 and P4.


📌 Recommendation 2: Device Naming Strategy for i2cBus

Problem: i2cBus control only shows numeric addresses (e.g., 0x68), not device names.

Solution: Implement a device identification registry

Step 1: Add device name field to i2cBus control

In ModuleIO.h:

control = addControl(controls, "i2cBus", "rows");
control["crud"] = "r";
rows = control["n"].to<JsonArray>();
{
  addControl(rows, "address", "number", 0, 255, true);  // ro
  addControl(rows, "name", "text", 0, 32, true);        // ro - NEW
}

Step 2: Create I2C device registry in ModuleIO

// In ModuleIO.h private section:
struct I2CDeviceInfo {
  uint8_t address;
  const char* name;
  Node* owner;  // Which node claimed this device
};
std::vector<I2CDeviceInfo> i2cDeviceRegistry;

public:
// Allow drivers to register their I2C devices
void registerI2CDevice(uint8_t address, const char* name, Node* owner) {
  for (auto& dev : i2cDeviceRegistry) {
    if (dev.address == address) {
      dev.name = name;
      dev.owner = owner;
      return;
    }
  }
  i2cDeviceRegistry.push_back({address, name, owner});
}

Step 3: Update updateDevices() to use registry

void updateDevices() {
  JsonDocument doc;
  doc["i2cBus"].to<JsonArray>();
  JsonObject newState = doc.as<JsonObject>();

  EXT_LOGI(ML_TAG, "Scanning I2C bus...");
  byte count = 0;
  for (byte i = 1; i < 127; i++) {
    Wire.beginTransmission(i);
    if (Wire.endTransmission() == 0) {
      JsonObject i2cDevice = newState["i2cBus"].as<JsonArray>().add<JsonObject>();
      i2cDevice["address"] = i;
      
      // Look up device name in registry
      const char* deviceName = "Unknown";
      for (const auto& dev : i2cDeviceRegistry) {
        if (dev.address == i) {
          deviceName = dev.name;
          break;
        }
      }
      i2cDevice["name"] = deviceName;
      
      EXT_LOGI(ML_TAG, "Found I2C device at address 0x%02X: %s", i, deviceName);
      count++;
    }
  }
  EXT_LOGI(ML_TAG, "Found %d device(s)", count);
  
  doc["i2cFreq"] = Wire.getClock() / 1000;
  update(newState, ModuleState::update, _moduleName);
}

Step 4: IMU driver registers itself

In D_IMU.h onUpdate():

if (control["on"] == true) {
  bool i2cInited = true;  // TODO: get from ModuleIO
  if (i2cInited) {
    if (board == 0) {  // MPU6050
      // Register this device with Module IO
      ModuleIO* io = modules->getModuleOfClass<ModuleIO>();
      if (io) {
        io->registerI2CDevice(0x68, "MPU6050", this);  // or 0x69 if AD0 high
      }
      
      mpu.initialize();
      // ... rest of code
    }
  }
}

Alternative Simpler Approach: Known I2C address database in ModuleIO:

const char* getDeviceNameByAddress(uint8_t addr) {
  switch(addr) {
    case 0x68: return "MPU6050/MPU9250";  // Could be either
    case 0x69: return "MPU6050/MPU9250 (AD0=HIGH)";
    case 0x76: return "BME280/BMP280";
    case 0x77: return "BME280/BMP280 (ALT)";
    case 0x3C: return "OLED SSD1306";
    case 0x3D: return "OLED SSD1306 (ALT)";
    case 0x40: return "PCA9685/INA219";
    // ... add more as needed
    default: return "Unknown";
  }
}

This is simpler but less accurate (address collisions between different chips). I recommend the registry approach for accuracy.


📌 Recommendation 3: BMI160 IMU Support

Library Recommendation

Use hanyazou/BMI160-Arduino (BMI160Gen.h API) - supports both I2C and SPI with simple API.

Why this library:

  • ✅ I2C and SPI support
  • ✅ I2C addresses: 0x68 (SDO=GND) or 0x69 (SDO=VCC)
  • ✅ Similar API structure to MPU6050 libraries
  • ✅ Actively maintained

Add to platformio.ini:

lib_deps = 
  ElectronicCats/MPU6050 @ 1.3.0 ; for MPU6050
  hanyazou/BMI160-Arduino @ ^1.0.0 ; for BMI160

Implementation in D_IMU.h

`#if` FT_MOONLIGHT

`#include` <MPU6050_6Axis_MotionApps20.h>
`#include` <BMI160Gen.h>  // NEW

class IMUDriver : public Node {
  // ... existing code ...
  
  void onUpdate(const Char<20>& oldValue, const JsonObject& control) override {
    if (!control["on"].isNull()) {
      if (control["on"] == true) {
        bool i2cInited = true;  // TODO: check ModuleIO
        if (i2cInited) {
          if (board == 0) {  // MPU6050
            // Register with Module IO
            ModuleIO* io = modules->getModuleOfClass<ModuleIO>();
            if (io) io->registerI2CDevice(0x68, "MPU6050", this);
            
            mpu.initialize();
            // ... existing MPU6050 code ...
            
          } else if (board == 1) {  // BMI160 - NEW
            // Register with Module IO
            ModuleIO* io = modules->getModuleOfClass<ModuleIO>();
            if (io) io->registerI2CDevice(0x68, "BMI160", this);
            
            BMI160.begin(BMI160GenClass::I2C_MODE, 0x68);
            
            if (BMI160.getDeviceID() == 0xD1) {  // BMI160 device ID
              EXT_LOGI(ML_TAG, "BMI160 connection successful");
              motionTrackingReady = true;
            } else {
              EXT_LOGW(ML_TAG, "BMI160 connection failed");
            }
          }
        }
      }
    }
  }
  
  void loop20ms() override {
    if (!motionTrackingReady) return;
    
    if (board == 0) {  // MPU6050
      // ... existing MPU6050 loop code ...
      
    } else if (board == 1) {  // BMI160 - NEW
      int gx, gy, gz, ax, ay, az;
      BMI160.readGyro(gx, gy, gz);
      BMI160.readAccelerometer(ax, ay, az);
      
      // Convert raw values to degrees (BMI160 gyro: 16.4 LSB/°/s at ±2000°/s range)
      gyro.x = gx / 16.4f;
      gyro.y = gy / 16.4f;
      gyro.z = gz / 16.4f;
      
      // Convert raw accel values (BMI160 accel: 16384 LSB/g at ±2g range)
      accell.x = ax;
      accell.y = ay;
      accell.z = az;
      
      // Calculate gravity vector from accelerometer
      float norm = sqrt(ax*ax + ay*ay + az*az);
      if (norm > 0) {
        sharedData.gravity.x = (ax / norm) * INT16_MAX;
        sharedData.gravity.y = (ay / norm) * INT16_MAX;
        sharedData.gravity.z = (az / norm) * INT16_MAX;
      }
    }
  }
  
 private:
  MPU6050 mpu;
  // BMI160 is accessed via static BMI160 object from library
};

`#endif`

Note: BMI160 uses I2C address 0x68 when SDO is grounded, same as MPU6050, so they can't coexist on the same bus without address changes.


📌 Recommendation 4: Generic IMU Library Strategy

Reality Check

Adafruit Unified Sensor is the closest to a unified Arduino IMU API, but BMI160 doesn't have Adafruit Unified Sensor support.

Recommended Approach: Internal Abstraction Layer

Don't try to use a single external library - instead, create your own thin abstraction:

// In D_IMU.h - add this interface
struct IMUSensorData {
  Coord3D gyro;       // degrees/sec
  Coord3D accell;     // raw or m/s²
  Coord3D gravity;    // normalized
  bool valid;
};

class IMUDriver : public Node {
  // ... existing code ...
  
private:
  // Unified read method
  IMUSensorData readSensor() {
    IMUSensorData data = {};
    
    if (board == 0) {  // MPU6050
      if (mpu.dmpGetCurrentFIFOPacket(fifoBuffer)) {
        mpu.dmpGetQuaternion(&q, fifoBuffer);
        mpu.dmpGetGravity(&gravity, &q);
        mpu.dmpGetYawPitchRoll(ypr, &q, &gravity);
        
        data.gyro.x = ypr[1] * 180 / M_PI;
        data.gyro.y = ypr[0] * 180 / M_PI;
        data.gyro.z = ypr[2] * 180 / M_PI;
        
        mpu.dmpGetLinearAccel(&aaReal, &aa, &gravity);
        data.accell.x = aaReal.x;
        data.accell.y = aaReal.y;
        data.accell.z = aaReal.z;
        
        data.gravity.x = gravity.x * INT16_MAX;
        data.gravity.y = gravity.y * INT16_MAX;
        data.gravity.z = gravity.z * INT16_MAX;
        data.valid = true;
      }
      
    } else if (board == 1) {  // BMI160
      int gx, gy, gz, ax, ay, az;
      BMI160.readGyro(gx, gy, gz);
      BMI160.readAccelerometer(ax, ay, az);
      
      data.gyro.x = gx / 16.4f;
      data.gyro.y = gy / 16.4f;
      data.gyro.z = gz / 16.4f;
      
      data.accell.x = ax;
      data.accell.y = ay;
      data.accell.z = az;
      
      float norm = sqrt(ax*ax + ay*ay + az*az);
      if (norm > 0) {
        data.gravity.x = (ax / norm) * INT16_MAX;
        data.gravity.y = (ay / norm) * INT16_MAX;
        data.gravity.z = (az / norm) * INT16_MAX;
      }
      data.valid = true;
    }
    
    return data;
  }
  
public:
  void loop20ms() override {
    if (!motionTrackingReady) return;
    
    IMUSensorData data = readSensor();
    if (data.valid) {
      gyro = data.gyro;
      accell = data.accell;
      sharedData.gravity = data.gravity;
      EXT_LOGD(ML_TAG, "Gyro: %f %f %f", gyro.x, gyro.y, gyro.z);
    }
  }
};

Benefits:

  • ✅ Single loop20ms() code path
  • ✅ Easy to add new IMU chips (just add another else if in readSensor())
  • ✅ No dependency on unified external library
  • ✅ Keep using best-in-class chip-specific libraries

Future IMUs to add:

  • LSM6DS3 / LSM6DSO (common on newer boards)
  • ICM-20948 (9-axis with magnetometer)
  • BNO055/BNO085 (fusion-ready IMUs)

📌 Recommendation 5: SPI Centralization (Similar to I2C)

Current State

  • ✅ SPI pin types defined in ModuleIO (pin_SPI_SCK, pin_SPI_MISO, pin_SPI_MOSI)
  • ⚠️ SPI.begin() called in EthernetSettingsService.cpp (not centralized)

Proposed Changes

1. Add SPI management to ModuleIO.h:

// In setupDefinition():
addControl(controls, "spiFreq", "number", 0, 80000, false, "kHz");

control = addControl(controls, "spiBus", "rows");
control["crud"] = "r";
rows = control["n"].to<JsonArray>();
{
  addControl(rows, "cs", "number", 0, SOC_GPIO_PIN_COUNT-1, true);  // ro
  addControl(rows, "name", "text", 0, 32, true);                    // ro
}

// Private members:
uint8_t pinSPISCK = UINT8_MAX;
uint8_t pinSPIMISO = UINT8_MAX;
uint8_t pinSPIMOSI = UINT8_MAX;
std::vector<SPIDeviceInfo> spiDeviceRegistry;

struct SPIDeviceInfo {
  uint8_t csPin;
  const char* name;
  Node* owner;
};

// Public methods:
void registerSPIDevice(uint8_t csPin, const char* name, Node* owner);
bool isSPIInitialized() const { return pinSPISCK != UINT8_MAX && pinSPIMISO != UINT8_MAX && pinSPIMOSI != UINT8_MAX; }

2. Initialize SPI in readPins():

void readPins() {
  // ... existing I2C code ...
  
  // SPI pin detection
  for (JsonObject pinObject : _state.data["pins"].as<JsonArray>()) {
    uint8_t usage = pinObject["usage"];
    if (usage == pin_SPI_SCK) {
      pinSPISCK = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "SPISCK found %d", pinSPISCK);
    }
    if (usage == pin_SPI_MISO) {
      pinSPIMISO = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "SPIMISO found %d", pinSPIMISO);
    }
    if (usage == pin_SPI_MOSI) {
      pinSPIMOSI = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "SPIMOSI found %d", pinSPIMOSI);
    }
  }
  
  if (pinSPISCK != UINT8_MAX && pinSPIMISO != UINT8_MAX && pinSPIMOSI != UINT8_MAX) {
    SPI.end();  // Clean up any previous SPI initialization
    SPI.begin(pinSPISCK, pinSPIMISO, pinSPIMOSI);
    uint16_t frequency = _state.data["spiFreq"];
    if (frequency > 0) {
      // SPI frequency is set per-transaction, not globally like I2C
      EXT_LOGI(ML_TAG, "SPI bus initialized: sck:%d miso:%d mosi:%d", pinSPISCK, pinSPIMISO, pinSPIMOSI);
    }
  }
}

3. Update EthernetSettingsService to use ModuleIO's SPI:

// In EthernetSettingsService.cpp, replace SPI.begin() with:
ModuleIO* io = /* get ModuleIO reference */;
if (io && io->isSPIInitialized()) {
  // SPI already initialized by ModuleIO, just use it
  ETH.begin(v_ETH_PHY_TYPE, v_ETH_PHY_ADDR, v_ETH_PHY_CS, v_ETH_PHY_IRQ, v_ETH_PHY_RST, SPI);
} else {
  EXT_LOGW("Ethernet", "SPI not initialized by ModuleIO");
}

Benefits:

  • ✅ Consistent architecture (I2C and SPI both in ModuleIO)
  • ✅ Single source of truth for bus configuration
  • ✅ Easy to add SPI-based drivers (displays, SD cards, etc.)
  • ✅ Prevents pin conflicts

📋 Summary of Follow-up Actions

High Priority:

  1. Fix i2cInited check in D_IMU.h - currently hardcoded to true
  2. Add device naming to i2cBus - implement registry approach
  3. Update ESP32-C6 and ESP32-P4 default pins - explicit per-variant defaults
  4. Add BMI160 support - use hanyazou/BMI160-Arduino library

Medium Priority:

  1. Create internal IMU abstraction - unified readSensor() method
  2. Centralize SPI in ModuleIO - mirror I2C architecture
  3. Document I2C/SPI device registration - for future driver developers

Low Priority:

  1. Add more IMU chips (LSM6DS3, ICM-20948, BNO055)
  2. Add SPI device scanning (enumerate CS pins with known devices)
  3. Add I2C pull-up resistor strength configuration

Would you like me to generate code for any specific recommendation, or shall I create a GitHub issue with this analysis as a follow-up task?


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 11, 2026

@coderabbitai, I also want to add my-sr602 pir and digital microphones like the ics43432/4 or inmp441

How to add these, generic solution like for i2c and spi needed ?

And do i2s microphones interfere with i2s FastLED drivers ?

@coderabbitai
Copy link

coderabbitai bot commented Feb 11, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Check current I2S usage in codebase ==="
rg -n "I2S|i2s" --type-add 'code:*.{h,cpp,ino}' --type code -g '!*.json' -g '!*.md' -C 2

echo -e "\n\n=== Check for PIR sensor mentions ==="
rg -n "PIR|pir|SR602|motion" --type-add 'code:*.{h,cpp,ino}' --type code -C 2

echo -e "\n\n=== Check FastLED RMT configuration ==="
rg -n "FASTLED_RMT|RMT|FastLED" --type-add 'code:*.{h,cpp,ino}' --type code -C 2

echo -e "\n\n=== Check platformio.ini for build flags ==="
cat platformio.ini | grep -A 30 "build_flags"

Length of output: 114151


🌐 Web query:

ESP32 I2S microphone Arduino library INMP441 ICS43432

💡 Result:

For INMP441 and ICS-43432/ICS43432 on ESP32 + Arduino, you typically don’t need a mic-specific library—just the Arduino-ESP32 I2S API (or the underlying ESP-IDF I2S driver). (docs.espressif.com)

Recommended (current) option: Arduino-ESP32 I2S API

Arduino-ESP32 provides an I2S class with setPins() + begin() for standard I2S (STD) mode. (docs.espressif.com)

Minimal pattern:

`#include` <I2S.h>

void setup() {
  // bclk, ws(lrclk), dout, din, mclk
  I2S.setPins(26, 25, -1, 32, -1);
  I2S.begin(I2S_MODE_STD, 16000, I2S_DATA_BIT_WIDTH_16BIT, I2S_SLOT_MODE_MONO);
}

void loop() {
  int32_t sample = I2S.read();
}

(Adjust pins/sample rate/bit-width for your board and mic.)

Lower-level option: ESP-IDF “new” I2S driver (i2s_std.h)

If you want maximum control/performance, use the ESP-IDF driver API (driver/i2s_std.h, etc.). (docs.espressif.com)

Avoid / be aware of: legacy driver/i2s.h

Many older INMP441 tutorials use #include <driver/i2s.h> (legacy). It can still be found in older examples, but Arduino-ESP32 has moved toward the newer APIs, and mixing “legacy” and “new” I2S drivers can cause conflicts. (dronebotworkshop.com)

Wiring notes (applies to both INMP441 and ICS-43432)

You wire BCLK/SCK, WS/LRCLK, and DOUT (mic) → DIN (ESP32); some mics have an L/R select pin to choose left vs right slot (often tie to GND for left). (docs.espressif.com)

Citations:


🌐 Web query:

ESP32 I2S RMT peripheral conflict FastLED DMA interference

💡 Result:

On ESP32, FastLED’s RMT and I2S “clockless” (LCD/parallel) drivers are both DMA-based and rely on timely ISR/buffer refills. “Conflicts” typically show up as sparkles/junk pixels, stalls, or timing corruption when something else delays those refills (other peripherals, flash operations, Wi‑Fi/BT, etc.). (reddit.com)

What’s actually conflicting

  • Flash/FS operations can pause the system: when ESP-IDF performs SPI flash operations, it disables caches and suspends other tasks; non‑IRAM-safe interrupts won’t run until flash ops finish. If your LED driver’s critical code/ISRs can’t run during that window, DMA underflows → corrupted LED waveform. (docs.espressif.com)
  • ESP-IDF 5 / Arduino-ESP32 core 3.x breaks FastLED’s classic ESP32 I2S LCD-mode approach: Espressif docs state LCD/Camera mode isn’t implemented by the I2S driver, and FastLED issue reports the legacy low-level approach is incompatible with the ESP-IDF 5 I2S redesign. (docs.espressif.com)
  • DMA-capable memory constraints: DMA buffers must be in internal DMA-capable RAM; if buffers land in PSRAM or non-DMA memory, the framework may copy/bounce-buffer (adding latency/jitter) or fail in edge cases. (github.com)

Practical mitigations (what usually works)

  1. Don’t run FastLED I2S and RMT at the same time (pick one output method for LEDs; avoid parallel “LED engine” + another timing-critical DMA-peripheral competing for ISR time).
  2. If you’re on Arduino-ESP32 core 3.x (ESP-IDF 5.x): prefer RMT (or FastLED’s SPI clockless WS2812 path) over the legacy I2S LCD driver, because I2S LCD-mode is effectively a mismatch with the new driver model. (github.com)
  3. Avoid flash activity during show(): don’t read SPIFFS/flash, don’t log heavily, don’t allocate, don’t do Wi‑Fi-heavy work right around FastLED.show(); flash ops can block non‑IRAM interrupts. (docs.espressif.com)
  4. Make DMA buffers explicitly DMA-capable (internal RAM / MALLOC_CAP_DMA style allocations) in any DMA-based LED/audio/display pipeline. (github.com)

If you tell me chip (ESP32 vs S3), Arduino core version (2.x vs 3.x), which FastLED output you’re using (RMT/I2S/SPI), and what the other peripheral is doing (I2S audio? RMT input? SPIFFS? Wi‑Fi?), I can suggest the most likely “known-good” combination and the least invasive change.

Citations:


@ewowi, Great questions! Let me address each sensor type and the I2S/FastLED interference concern:


📌 1. MY-SR602 PIR Sensor (Passive Infrared Motion Detector)

No bus protocol needed – it's just a digital GPIO input!

The MY-SR602 is a simple digital motion sensor that outputs:

  • HIGH (3.3V) when motion is detected
  • LOW (0V) when no motion

Implementation in ModuleIO.h

Add a new pin usage type:

// In ModuleIO.h pin usage enum:
enum PinUsage {
  // ... existing types ...
  pin_PIR,        // NEW: Passive Infrared Motion sensor
  // ... rest of types ...
};

Add to UI dropdown:

// In setupDefinition():
addControlValue(control, "PIR");

No centralized management needed – each PIR driver node will just read its assigned GPIO pin directly using digitalRead().

Create a PIR Driver Node

Create src/MoonLight/Nodes/Drivers/D_PIR.h:

`#if` FT_MOONLIGHT

class PIRDriver : public Node {
 public:
  static const char* name() { return "PIR Sensor"; }
  static uint8_t dim() { return _NoD; }
  static const char* tags() { return "👁️"; }

  bool motionDetected = false;
  bool lastState = false;
  uint8_t pinPIR = UINT8_MAX;
  uint32_t lastTriggerTime = 0;
  uint32_t cooldownMs = 2000;  // Configurable cooldown period

  void setup() override {
    addControl(motionDetected, "motion", "checkbox", 0, 1, true);  // read-only
    addControl(cooldownMs, "cooldown", "number", 100, 60000, false, "ms");
  }

  void onUpdate(const Char<20>& oldValue, const JsonObject& control) override {
    if (!control["on"].isNull()) {
      if (control["on"] == true) {
        // Find PIR pin from ModuleIO
        pinPIR = UINT8_MAX;
        for (JsonObject pinObject : modules->getModuleOfClass<ModuleIO>()->_state.data["pins"].as<JsonArray>()) {
          if (pinObject["usage"] == pin_PIR) {
            pinPIR = pinObject["GPIO"];
            break;
          }
        }
        
        if (pinPIR != UINT8_MAX) {
          pinMode(pinPIR, INPUT);
          EXT_LOGI(ML_TAG, "PIR sensor initialized on GPIO %d", pinPIR);
        } else {
          EXT_LOGW(ML_TAG, "No PIR pin assigned in IO configuration");
        }
      }
    }
  }

  void loop() override {
    if (pinPIR == UINT8_MAX) return;
    
    bool currentState = digitalRead(pinPIR);
    
    // Detect rising edge (motion started)
    if (currentState && !lastState) {
      motionDetected = true;
      lastTriggerTime = millis();
      
      // Update shared data or trigger effects
      // sharedData.pirTriggered = true;  // If you add this field to SharedData
      
      // Update UI
      JsonDocument doc;
      doc["motion"] = true;
      update(doc.as<JsonObject>(), ModuleState::update, _nodeName);
      
      EXT_LOGI(ML_TAG, "Motion detected!");
    }
    
    // Auto-reset after cooldown
    if (motionDetected && (millis() - lastTriggerTime > cooldownMs)) {
      motionDetected = false;
      
      JsonDocument doc;
      doc["motion"] = false;
      update(doc.as<JsonObject>(), ModuleState::update, _nodeName);
    }
    
    lastState = currentState;
  }
};

`#endif`

Register in ModuleDrivers.h:

addControlValue(control, getNameAndTags<PIRDriver>());
// ...
if (!node) node = checkAndAlloc<PIRDriver>(name);

📌 2. I2S Microphones (INMP441, ICS-43432/4)

Yes, centralize I2S bus management in ModuleIO.h (similar to I2C/SPI)

I2S microphones use the I2S bus protocol with these signals:

  • SD (Serial Data / DOUT from mic → DIN to ESP32)
  • WS (Word Select / LRCLK - left/right channel clock)
  • SCK (Bit Clock / BCLK)
  • MCLK (Master Clock - optional, some mics don't need it)

ModuleIO.h already has I2S pin types defined!

Looking at the code, ModuleIO.h already includes:

pin_I2S_SD,
pin_I2S_WS,
pin_I2S_SCK,
pin_I2S_MCLK,

And some board presets already assign them (e.g., board_SR for SR boards with mic support).

Add I2S Bus Initialization to ModuleIO.h

Similar to the I2C centralization, add I2S management:

// In ModuleIO.h, add includes at top:
`#include` <I2S.h>

// In setupDefinition():
addControl(controls, "i2sFreq", "number", 0, 96000, false, "Hz");  // Sample rate

control = addControl(controls, "i2sBus", "rows");
control["crud"] = "r";
rows = control["n"].to<JsonArray>();
{
  addControl(rows, "device", "text", 0, 32, true);  // ro - device type
  addControl(rows, "active", "checkbox", 0, 1, true);  // ro - is initialized
}

// Private members:
uint8_t pinI2SSD = UINT8_MAX;
uint8_t pinI2SWS = UINT8_MAX;
uint8_t pinI2SSCK = UINT8_MAX;
uint8_t pinI2SMCLK = UINT8_MAX;
bool i2sInitialized = false;

// Public methods:
bool isI2SInitialized() const { return i2sInitialized; }

Extend readPins() to initialize I2S:

void readPins() {
  // ... existing I2C code ...
  
  // I2S pin detection
  for (JsonObject pinObject : _state.data["pins"].as<JsonArray>()) {
    uint8_t usage = pinObject["usage"];
    if (usage == pin_I2S_SD) {
      pinI2SSD = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "I2S SD found %d", pinI2SSD);
    }
    if (usage == pin_I2S_WS) {
      pinI2SWS = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "I2S WS found %d", pinI2SWS);
    }
    if (usage == pin_I2S_SCK) {
      pinI2SSCK = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "I2S SCK found %d", pinI2SSCK);
    }
    if (usage == pin_I2S_MCLK) {
      pinI2SMCLK = pinObject["GPIO"];
      EXT_LOGD(ML_TAG, "I2S MCLK found %d", pinI2SMCLK);
    }
  }
  
  // Initialize I2S if all required pins are assigned
  if (pinI2SSD != UINT8_MAX && pinI2SWS != UINT8_MAX && pinI2SSCK != UINT8_MAX) {
    I2S.end();  // Clean up any previous I2S initialization
    
    // setPins(bclk, ws, dout, din, mclk)
    // For microphone: dout=-1 (we're not outputting), din=SD pin
    int8_t mclkPin = (pinI2SMCLK != UINT8_MAX) ? pinI2SMCLK : -1;
    I2S.setPins(pinI2SSCK, pinI2SWS, -1, pinI2SSD, mclkPin);
    
    uint32_t sampleRate = _state.data["i2sFreq"] | 16000;  // Default 16kHz
    
    // Begin in standard mode, 16-bit samples, mono (microphone)
    bool success = I2S.begin(I2S_MODE_STD, sampleRate, I2S_DATA_BIT_WIDTH_16BIT, I2S_SLOT_MODE_MONO);
    
    if (success) {
      i2sInitialized = true;
      EXT_LOGI(ML_TAG, "I2S bus initialized: sd:%d ws:%d sck:%d mclk:%d @ %d Hz", 
               pinI2SSD, pinI2SWS, pinI2SSCK, pinI2SMCLK, sampleRate);
      
      updateI2SDevices();
    } else {
      EXT_LOGW(ML_TAG, "I2S initialization failed");
    }
  }
}

void updateI2SDevices() {
  JsonDocument doc;
  doc["i2sBus"].to<JsonArray>();
  JsonObject newState = doc.as<JsonObject>();
  
  if (i2sInitialized) {
    JsonObject device = newState["i2sBus"].as<JsonArray>().add<JsonObject>();
    device["device"] = "I2S Microphone";
    device["active"] = true;
  }
  
  doc["i2sFreq"] = _state.data["i2sFreq"] | 16000;
  update(newState, ModuleState::update, _moduleName);
}

Create a Microphone Driver Node

Create src/MoonLight/Nodes/Drivers/D_Microphone.h:

`#if` FT_MOONLIGHT

`#include` <I2S.h>

class MicrophoneDriver : public Node {
 public:
  static const char* name() { return "Microphone"; }
  static uint8_t dim() { return _NoD; }
  static const char* tags() { return "🎤"; }

  uint16_t volumeLevel = 0;      // Current volume (0-1023)
  uint16_t volumePeak = 0;       // Peak volume
  uint16_t volumeAverage = 0;    // Running average
  bool soundDetected = false;
  uint16_t threshold = 100;      // Configurable threshold

  void setup() override {
    addControl(volumeLevel, "volume", "number", 0, 1023, true);      // read-only
    addControl(volumePeak, "peak", "number", 0, 1023, true);         // read-only
    addControl(volumeAverage, "average", "number", 0, 1023, true);   // read-only
    addControl(soundDetected, "sound", "checkbox", 0, 1, true);      // read-only
    addControl(threshold, "threshold", "number", 0, 1023, false);
  }

  void onUpdate(const Char<20>& oldValue, const JsonObject& control) override {
    if (!control["on"].isNull()) {
      if (control["on"] == true) {
        ModuleIO* io = modules->getModuleOfClass<ModuleIO>();
        if (io && io->isI2SInitialized()) {
          EXT_LOGI(ML_TAG, "Microphone driver activated");
        } else {
          EXT_LOGW(ML_TAG, "I2S not initialized by ModuleIO");
        }
      }
    }
  }

  void loop20ms() override {
    ModuleIO* io = modules->getModuleOfClass<ModuleIO>();
    if (!io || !io->isI2SInitialized()) return;
    
    // Read I2S samples
    int32_t sample = I2S.read();
    
    // Convert to absolute value and scale
    uint16_t magnitude = abs(sample) >> 16;  // Adjust scaling as needed
    
    volumeLevel = magnitude;
    
    // Update peak
    if (magnitude > volumePeak) {
      volumePeak = magnitude;
    } else {
      volumePeak = (volumePeak * 15 + magnitude) / 16;  // Slow decay
    }
    
    // Update running average
    volumeAverage = (volumeAverage * 15 + magnitude) / 16;
    
    // Sound detection
    soundDetected = (magnitude > threshold);
    
    // Store in shared data for effects to use
    // sharedData.audioLevel = volumeLevel;  // If you add this to SharedData
    
    // Update UI periodically (not every 20ms to avoid spam)
    static uint8_t updateCounter = 0;
    if (++updateCounter >= 25) {  // Update UI every 500ms
      updateCounter = 0;
      JsonDocument doc;
      doc["volume"] = volumeLevel;
      doc["peak"] = volumePeak;
      doc["average"] = volumeAverage;
      doc["sound"] = soundDetected;
      update(doc.as<JsonObject>(), ModuleState::update, _nodeName);
    }
  }
};

`#endif`

📌 3. Do I2S Microphones Interfere with I2S FastLED Drivers?

✅ Good news: NO direct conflict!

Why:

  1. FastLED uses RMT, not I2S (on modern ESP-IDF 5.x / Arduino-ESP32 3.x)

    • From your learning: "FastLED defaults to RMT5 driver (wraps Espressif's led_strip component)"
    • Your platformio.ini can force RMT4: -D FASTLED_RMT5=0
  2. RMT and I2S are different peripherals

    • RMT = Remote Control Transceiver (generates precise timing pulses for WS2812/SK6812 LEDs)
    • I2S = Inter-IC Sound (serial audio interface for microphones/speakers)
    • They use different hardware blocks on the ESP32
  3. The "I2S" in your D_FastLED.h is OLD/DEPRECATED

    • The FASTLED_USES_ESP32S3_I2S flag refers to FastLED's legacy I2S LCD parallel mode
    • This mode is not compatible with ESP-IDF 5.x and should not be used
    • Your code correctly shows bool usesI2S = false for modern builds

⚠️ Potential indirect issues (DMA bandwidth competition)

While RMT and I2S won't directly conflict, both are DMA-based, so:

Possible symptoms if both are heavily used:

  • LED timing glitches (sparkles/junk pixels)
  • Audio dropouts
  • Increased CPU load from DMA buffer management

Mitigations:

  1. Allocate DMA buffers in internal RAM (not PSRAM):

    buffer = (uint8_t*)heap_caps_malloc(size, MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL);
  2. Avoid flash operations during critical timing (especially during FastLED.show()):

    • Don't read SPIFFS/LittleFS
    • Minimize logging
    • Avoid heavy WiFi activity
  3. Pin FastLED and audio processing to different cores:

    // FastLED on Core 1 (APP_CPU)
    xTaskCreatePinnedToCore(ledTask, "LED", 4096, NULL, 1, NULL, 1);
    
    // Audio on Core 0 (PRO_CPU)
    xTaskCreatePinnedToCore(audioTask, "Audio", 4096, NULL, 1, NULL, 0);
  4. Reduce I2S sample rate if not needed:

    • Use 16kHz instead of 44.1kHz if acceptable for your use case
    • Lower rates = less DMA traffic

Summary Table:

Concern FastLED RMT + I2S Mic FastLED Old I2S LCD + I2S Mic
Direct peripheral conflict No conflict Conflict! Don't use!
DMA bandwidth competition ⚠️ Possible under heavy load ⚠️ Yes, both use I2S DMA
Recommended? Safe combination Avoid - use RMT instead

📋 Action Items Summary

For PIR Sensors:

  1. ✅ Add pin_PIR usage type to ModuleIO.h
  2. ✅ Create D_PIR.h driver (simple GPIO input reading)
  3. ✅ Register in ModuleDrivers.h

For I2S Microphones:

  1. ✅ Centralize I2S bus management in ModuleIO.h (similar to I2C/SPI)
  2. ✅ Initialize I2S bus when pins are assigned
  3. ✅ Create D_Microphone.h driver
  4. ✅ Add audio level to SharedData for effects to use
  5. ✅ Consider adding FFT analysis if you want frequency-reactive effects

Interference Mitigation:

  1. ✅ Verify you're using FastLED RMT mode (not I2S LCD mode)
  2. ✅ Allocate DMA buffers in internal RAM
  3. ✅ Minimize flash/WiFi activity during LED updates
  4. ✅ Consider dual-core task pinning for heavy loads

Would you like me to generate the complete implementation code for either the PIR driver or the I2S microphone driver?


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:45.988Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

ModuleIO: add I2C pins for default boards per MCU type, reset pins before update

IMU Driver: fix accell calculation, BMI160 code (commented ATM)
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Around line 578-601: Remove the redundant initial I2C pin assignment block
that calls pinAssigner.assignPin(21, pin_I2C_SDA)/assignPin(22, pin_I2C_SCL)
(the block guarded by `#ifdef` CONFIG_IDF_TARGET_ESP32 at the top) because
setBoardPresetDefaults() already performs target-specific assignments; keep the
later granular conditional block (the `#ifdef` / `#elif` chain that includes
CONFIG_IDF_TARGET_ESP32, ESP32S2/S3/C3, ESP32C6, ESP32P4, and the fallback) and
delete the earlier duplicate assignments so pinAssigner.assignPin is only
invoked once per board preset.
- Around line 848-859: The i2cFreq control currently defaults to 0 and leads to
calling Wire.begin(pinI2CSDA, pinI2CSCL, 0) and showing "0 kHz" in the UI;
update the setupDefinition to provide a sensible default (e.g., 100) for i2cFreq
and add a guard in the code that reads _state.data["i2cFreq"] (the
readPins/initI2C area) to treat 0 as the default value (replace with 100 or
another chosen default) before multiplying by 1000 and passing to
Wire.begin(pinI2CSDA, pinI2CSCL, frequency * 1000); ensure any logged frequency
uses the adjusted value so the UI shows a nonzero kHz and maintain behavior that
falls back to a safe clock when frequency is absent or zero.

In `@src/MoonLight/Nodes/Drivers/D_IMU.h`:
- Around line 29-35: The board selection control (added in setup via
addControl(board,...)) can flip at runtime without clearing state, so update
handling must reset motion-tracking state; add a guard in onUpdate (or a change
callback for the "board" control) that detects when board changes and sets
motionTrackingReady = false and disables/re-initializes motion tracking (e.g.,
set isEnabled = false or call the MPU6050 teardown/init sequence) so loop20ms's
BMI160 branch cannot run while MPU6050 state remains valid; reference
functions/vars: setup, addControl(board,...), board, onUpdate, loop20ms,
motionTrackingReady, isEnabled (or the MPU init/teardown helpers) to locate
where to implement this reset.
- Around line 38-88: Add a cleanup path for when the driver is turned off by
modifying onUpdate in D_IMU: when control["on"] == false (or control["on"] is
present and false) call the MPU shutdown steps—disable the DMP via
mpu.setDMPEnabled(false) (or equivalent), clear any interrupt/status flags, stop
any periodic reads, and set motionTrackingReady = false; also ensure any
board==1 BMI160 cleanup is mirrored. Then implement the same teardown in the
D_IMU destructor (~D_IMU) to disable DMP/interrupts and reset
motionTrackingReady so loop20ms stops reading after node removal. Ensure you
reference the existing symbols onUpdate, motionTrackingReady, mpu.setDMPEnabled
(and ~D_IMU) when making changes.
🧹 Nitpick comments (2)
src/MoonLight/Nodes/Drivers/D_IMU.h (1)

109-118: Stale comments around the accelerometer read — clean up for clarity.

  • Line 109: "display real acceleration, adjusted to remove gravity" is an orphaned comment (no display code follows it).
  • Line 113: "needed to repeat the following 3 lines" is misleading since only dmpGetAccel (line 115) is active; the other two remain commented out.

Consider removing these leftover comments to avoid confusing future readers about why certain calls are or aren't needed.

src/MoonBase/Modules/ModuleIO.h (1)

862-863: I2C pin members are public — verify this is intentional.

pinI2CSDA and pinI2CSCL are declared in the public section, presumably so D_IMU.h and other drivers can query them. If external read access is the intent, this is fine; consider adding a brief comment to document the intended usage.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 13, 2026

@coderabbitai, something went wrong in saving files (delayedWrite?) between commit 03509aa and 45c03a5 in which I switched from FsPersistence to SharedFsPersistence.

In the logging I see for both commits when I save:

[ 32284][D][SharedFSPersistence.h:165] writeToFSDelayed(): [🐼] calling 6 writeFuncs from delayedWrites
[ 32295][D][SharedFSPersistence.h:120] operator()(): [🐼] delayedWrites: W /.config/devices.json
[ 32380][D][SharedFSPersistence.h:120] operator()(): [🐼] delayedWrites: W /.config/lightscontrol.json
[ 32411][D][SharedFSPersistence.h:120] operator()(): [🐼] delayedWrites: W /.config/channels.json
[ 32435][D][SharedFSPersistence.h:120] operator()(): [🐼] delayedWrites: W /.config/effects.json
[ 32507][D][SharedFSPersistence.h:120] operator()(): [🐼] delayedWrites: W /.config/drivers.json
[ 32585][D][SharedFSPersistence.h:120] operator()(): [🐼] delayedWrites: W /.config/tasks.json
[ 33161][D][SharedFSPersistence.h:114] writeToFS(): [🐼] delayedWrites: Add /.config/tasks.json

[ 32338][D][FSPersistence.h:121] writeToFSDelayed(): [🐼] calling 6 writeFuncs from delayedWrites
[ 32348][D][FSPersistence.h:99] operator()(): [🐼] delayedWrites: W /.config/lightscontrol.json
[ 32379][D][FSPersistence.h:99] operator()(): [🐼] delayedWrites: W /.config/devices.json
[ 32475][D][FSPersistence.h:99] operator()(): [🐼] delayedWrites: W /.config/channels.json
[ 32499][D][FSPersistence.h:99] operator()(): [🐼] delayedWrites: W /.config/effects.json
[ 32663][D][FSPersistence.h:99] operator()(): [🐼] delayedWrites: W /.config/drivers.json
[ 32727][D][FSPersistence.h:99] operator()(): [🐼] delayedWrites: W /.config/tasks.json
[ 32804][D][FSPersistence.h:97] writeToFS(): [🐼] delayedWrites: Add /.config/tasks.json

If I update the effect from Game of Life to VU meter and save and check the files in the file system I see in both cases :

{"name":"VU Meter ⏹️ ♫💫📺","on":true,"controls":[{"name":"speed","value":255,"type":"slider","default":255,"p":1207978129,"valid":true,"size":8},

But when I restart the device in case of the newer commit in which I implemented SharedFsPersistence (45c03a5) I see after restart the old Game of life is restored. So it looks like even the new effect is saved, somewhere along the way another save to the filesystem is done restoring old? (As during reboot, the file system contents is used to build up the state. What is the problem?

@coderabbitai
Copy link

coderabbitai bot commented Feb 13, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Reading FSPersistence.h (old implementation) ==="
cat src/MoonBase/FSPersistence.h

echo -e "\n\n=== Reading SharedFSPersistence.h (new implementation) ==="
cat src/MoonBase/SharedFSPersistence.h

echo -e "\n\n=== Reading StatefulService.h to understand integration ==="
cat src/MoonBase/StatefulService.h

echo -e "\n\n=== Check the commit diff between the two commits ==="
git diff 03509aaaa01b46543b89f0f6ea88dac2f339e2dc 45c03a5de34e0f3e5ee34f7826b9f2598da1c617 -- src/MoonBase/FSPersistence.h src/MoonBase/SharedFSPersistence.h

echo -e "\n\n=== Search for writeToFS calls that might be triggered during init/shutdown ==="
rg -n "writeToFS" --type-add 'code:*.{h,cpp}' --type code -C 3

echo -e "\n\n=== Search for places that instantiate or use StatefulService ==="
rg -n "StatefulService|FsPersistence|SharedFsPersistence" --type-add 'code:*.{h,cpp}' --type code | head -100

echo -e "\n\n=== Look for setup/begin/init methods that might trigger writes ==="
rg -n "void setup\(\)|void begin\(\)|void init\(\)" --type-add 'code:*.{h,cpp}' --type code -A 5 | grep -A 5 "writeToFS\|persistence"

Length of output: 20087


🏁 Script executed:

#!/bin/bash

echo "=== Reading NodeManager.h to find readFromFS call ==="
cat src/MoonBase/NodeManager.h

echo -e "\n\n=== Reading Module.h to understand initialization ==="
cat src/MoonBase/Module.h

echo -e "\n\n=== Look at main.cpp initialization sequence ==="
cat src/main.cpp | grep -A 5 -B 5 "sharedFsPersistence"

echo -e "\n\n=== Search for places that might trigger module updates during init ==="
rg -n "setup\(\)|begin\(\)|onUpdate" --type-add 'code:*.{h,cpp}' --type code src/MoonBase/NodeManager.h src/MoonBase/Module.h -C 3

echo -e "\n\n=== Check if any modules call update() in their constructors or setup ==="
rg -n "class.*Module.*{" --type-add 'code:*.{h,cpp}' --type code -A 30 | grep -E "(Module|setup|begin|update\()" | head -80

Length of output: 28912


🏁 Script executed:

#!/bin/bash

echo "=== Find where Module::begin() is called for all modules ==="
rg -n "->begin\(\)|\.begin\(\)" --type-add 'code:*.{cpp,h}' --type code src/main.cpp -C 3

echo -e "\n\n=== Look at Module::begin() implementation ==="
cat src/MoonBase/Module.cpp

echo -e "\n\n=== Check setupData implementation ==="
rg -n "setupData" --type-add 'code:*.{cpp,h}' --type code -C 5

echo -e "\n\n=== Check if Module constructor or begin calls update ==="
rg -n "Module::Module|Module::begin" --type-add 'code:*.{cpp,h}' --type code -A 20

echo -e "\n\n=== Look for updateWithoutPropagation calls ==="
rg -n "updateWithoutPropagation" --type-add 'code:*.{cpp,h}' --type code -C 3

Length of output: 35050


@ewowi, I found the bug! 🎯 It's a timing issue with when update handlers are registered vs. when files are read from the filesystem.

Root Cause

The SharedFSPersistence change altered the order of operations during initialization, causing update handlers to fire during the initial state setup, which writes stale/default data to the delayed writes queue before the correct data is loaded from the filesystem.

What Happens in the Broken Version (45c03a5):

1. main.cpp:323  → sharedFsPersistence->registerModule(module, true)
                  → Update handler IMMEDIATELY registered and active ⚠️

2. Module::begin() is called (somewhere before sharedFsPersistence->begin())
                  → setupData() populates state with defaults
                  → compareRecursive() processes each default value
                  → onUpdate() fires for each default
                  → Update handler fires → writeToFS() → queues DEFAULTS to delayedWrites ❌

3. main.cpp:337  → sharedFsPersistence->begin()
                  → readFromFS() loads CORRECT data from .json files
                  → Updates state with file data ✅
                  → But delayedWrites still contains old/default data from step 2

4. User clicks Save
                  → writeToFSDelayed('W') processes delayedWrites
                  → Writes recent changes (VU Meter) ✅
                  
5. Later (maybe during another update cycle or at next save)
                  → Another update triggers
                  → Old queued data from step 2 gets written ❌
                  → Overwrites the correct VU Meter data with Game of Life

What Happened in the Working Version (03509aa):

1. Module constructor → FSPersistence created but handler not yet registered

2. Module::begin()  → setupData() with defaults
                    → readFromFS() loads from file immediately
                    → THEN update handler registered ✅
                    
(Handler only active AFTER file is read, so defaults never get written)

The Fix

You have two options:

Option 1: Disable Handlers During Initial Read (Recommended)

Modify SharedFSPersistence::begin() to disable handlers, read files, then re-enable:

void begin() {
  // Disable all update handlers before reading
  for (auto& pair : _modules) {
    disableUpdateHandler(pair.first);
  }
  
  // Read initial state from filesystem
  for (auto& pair : _modules) {
    readFromFS(pair.first);
  }
  
  // Re-enable update handlers after all reads complete
  for (auto& pair : _modules) {
    enableUpdateHandler(pair.first);
  }
}

Option 2: Register Handlers After All Reads (Alternative)

Change the initialization sequence in main.cpp:

// Register modules WITHOUT handlers first
for (Module* module : modules) {
  sharedHttpEndpoint->registerModule(module);
  sharedWebSocketServer->registerModule(module);
  sharedEventEndpoint->registerModule(module);
  sharedFsPersistence->registerModule(module, false);  // ← false: no delayed writing yet
}

// ... module begin() calls ...

// Read from filesystem
sharedFsPersistence->begin();

// NOW enable delayed writing and handlers
for (Module* module : modules) {
  auto it = sharedFsPersistence->_modules.find(module->_moduleName);
  if (it != sharedFsPersistence->_modules.end()) {
    it->second.delayedWriting = true;
    sharedFsPersistence->enableUpdateHandler(module->_moduleName);
  }
}

But this requires making _modules public, so Option 1 is cleaner.

Why This is Subtle

The bug is hard to spot because:

  1. ✅ The file on disk shows correct data (VU Meter) after save
  2. ✅ The delayed write queue logs show VU Meter being written
  3. ❌ But there's stale data from initialization still in the queue that gets written later
  4. ❌ On reboot, either:
    • The stale write happens during shutdown, or
    • Multiple delayed writes are processed and the last one (with stale data) wins

Recommendation: Implement Option 1 by modifying SharedFSPersistence::begin() to temporarily disable update handlers during the initial file read phase. This ensures handlers only become active after all modules have loaded their persisted state.

Would you like me to generate the complete fixed begin() method for you?


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 13, 2026

@coderabbitai, regarding

Implement Option 1 by modifying SharedFSPersistence::begin() to temporarily disable update handlers during the initial file read phase. This ensures handlers only become active after all modules have loaded their persisted state.

I want a solution which is as close as possible to (not shared) FSPersistence as code and functionality should not diverge.

Is this the case with option 1, in other words , is this realigning with how FSPersistence works ?

@coderabbitai
Copy link

coderabbitai bot commented Feb 13, 2026

Tip

For best results, initiate chat on the files or code changes.

@ewowi, Great question! Let me compare the two implementations carefully.

How FSPersistence Works (Original)

Looking at the code flow:

// In Module::begin()
void begin() {
  _fsPersistence = std::make_unique<FSPersistence<ModuleState>>(this, _moduleName);
  // ↑ FSPersistence constructor runs here:
  //   1. Register update handler
  //   2. readFromFS() immediately  ← File data loaded FIRST
  //   3. Uses updateWithoutPropagation (no handler trigger)
  
  setupData();  // ← Runs AFTER file is already loaded
  // Default values are compared against file data, so only actual changes trigger handlers
}

Timeline: Register handler → Read file → Setup defaults

How SharedFSPersistence Currently Works (Broken)

// In main.cpp
sharedFsPersistence->registerModule(module, true);
  // ↑ If delayedWriting=true, immediately calls enableUpdateHandler()

// Later...
module->begin();
  setupData();  // ← Triggers handlers with DEFAULTS before file is read! ❌

// Even later...
sharedFsPersistence->begin();
  readFromFS();  // ← Reads file data, but too late - defaults already queued

Timeline: Register handler → Setup defaults (triggers handler) → Read file

The Right Fix (Matching FSPersistence)

Option 1 doesn't perfectly match FSPersistence because it disables/re-enables handlers, which FSPersistence never does.

The correct fix is to delay handler enabling until after files are read, matching the FSPersistence order:

Modified SharedFSPersistence::registerModule()

void registerModule(Module* module, bool delayedWriting) {
  const char* moduleName = module->_moduleName;
  
  if (_modules.find(moduleName) != _modules.end()) {
    return;
  }
  
  ModuleInfo moduleInfo;
  moduleInfo.service = module;
  moduleInfo.delayedWriting = delayedWriting;
  // Don't enable handler here - remember the setting for later
  
  _modules[moduleName] = moduleInfo;
  
  EXT_LOGD(ML_TAG, "Module %s registered (delayed writing: %s)", 
           moduleName, delayedWriting ? "enabled" : "disabled");
}

Modified SharedFSPersistence::begin()

void begin() {
  EXT_LOGI(ML_TAG, "SharedFSPersistence begin - reading state from filesystem");
  
  // FIRST: Read all files before any handlers are active
  for (auto& pair : _modules) {
    readFromFS(pair.first);
  }
  
  // SECOND: Now enable handlers for modules that requested delayed writing
  for (auto& pair : _modules) {
    if (pair.second.delayedWriting) {
      enableUpdateHandler(pair.first);
      EXT_LOGD(ML_TAG, "Enabled update handler for %s after file read", pair.first);
    }
  }
  
  EXT_LOGI(ML_TAG, "SharedFSPersistence initialization complete");
}

Why This Matches FSPersistence

FSPersistence SharedFSPersistence (Fixed)
1. Handler registered 1. Handler registered (but not enabled)
2. readFromFS() immediately 2. readFromFS() for all modules in begin()
3. setupData() runs after 3. setupData() already ran, but handlers weren't enabled yet
4. Handler active after read 4. Handlers enabled after all reads complete

Result: Both ensure file data is loaded before handlers can queue writes.

Implementation

The fix is actually simpler than Option 1 - just move the enableUpdateHandler() call from registerModule() to begin() after all files are read. This perfectly mirrors FSPersistence's behavior where the handler exists but files are read before any state changes can trigger it.

Would you like me to generate the complete updated SharedFSPersistence.h file with this fix?

🐰✨


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

@ewowi, You're absolutely right! The diagnostic logs confirm that:

✅ Analysis Confirms Platform Bug

Your observation is correct:

  1. ✅ All writes complete successfully between 162845-163268ms (4 files)
  2. ✅ Heap is stable (~325-329KB) - no memory issues
  3. 8 second gap between last write completion (163268ms) and crash (171346ms)
  4. ✅ Crash happens in bootloader ROM (PC: 0x4ff2ab04) during your manual restart
  5. ✅ Your code is not involved in the crash at all

The checkReOrderSwap activity (163502-171036ms) is normal:

  • These are device list reordering operations triggered by the UI
  • They queue additional writes to devices.json (which would be written on the next save)
  • This is expected behavior and not related to the crash

🎯 Conclusion

This is definitively a platform regression bug in 55.03.37 that doesn't exist in 55.03.35. The crash is in the bootloader's WiFi cleanup code during ESP.restart(), triggered by LittleFS activity that occurred seconds earlier.

✅ Next Steps

  1. File the issue with the platform maintainer using the text I provided earlier (it's accurate)
  2. Stay on 55.03.35 until the platform bug is fixed
  3. No code changes needed in MoonLight - this is 100% a platform issue

The issue template I provided earlier is ready to copy-paste to:
https://github.com/pioarduino/platform-espressif32/issues

Your application code is working correctly - the ESP32-P4 support in 55.03.37 has a critical bootloader bug that causes crashes during reboot after filesystem operations.


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 14, 2026

@coderabbitai, see also in the logging the repeatingly change in reorder swap in the devices module. Can you find the root cause and a solution?:

[242884][W][Module.cpp:250] update(): [💫] ESP32 SvelteKit (2616) checkReOrderSwap changed, compareRecursive also changed? devices
[243996][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (255 @ 0) 14 -> 15
[244007][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (15 @ 14) 15 -> 14
[244019][W][Module.cpp:250] update(): [💫] ESP32 SvelteKit (2616) checkReOrderSwap changed, compareRecursive also changed? devices
[245643][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (255 @ 0) 14 -> 15
[245653][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (15 @ 14) 15 -> 14
[245665][W][Module.cpp:250] update(): [💫] ESP32 SvelteKit (2616) checkReOrderSwap changed, compareRecursive also changed? devices
[248182][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (255 @ 0) 14 -> 15
[248192][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (15 @ 14) 15 -> 14
[248204][W][Module.cpp:250] update(): [💫] ESP32 SvelteKit (2616) checkReOrderSwap changed, compareRecursive also changed? devices
[248782][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (255 @ 0) 14 -> 15
[248792][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (15 @ 14) 15 -> 14
[248804][W][Module.cpp:250] update(): [💫] ESP32 SvelteKit (2616) checkReOrderSwap changed, compareRecursive also changed? devices
[250023][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (255 @ 0) 14 -> 15
[250033][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (15 @ 14) 15 -> 14
[250045][W][Module.cpp:250] update(): [💫] ESP32 SvelteKit (2616) checkReOrderSwap changed, compareRecursive also changed? devices
[250403][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (255 @ 0) 14 -> 15
[250412][D][Module.cpp:89] checkReOrderSwap(): [💫] ESP32 SvelteKit (2616) (15 @ 14) 15 -> 14

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Read Module.cpp checkReOrderSwap implementation ==="
cat src/MoonBase/Module.cpp | grep -A 50 "checkReOrderSwap"

echo -e "\n\n=== Read ModuleDevices.h for sorting/ordering logic ==="
cat src/MoonBase/Modules/ModuleDevices.h | grep -A 30 "sort\|order\|swap" -i

echo -e "\n\n=== Check if there's a comparison function for devices ==="
rg -n "compare.*device|device.*compare|sort.*device" --type-add 'code:*.{h,cpp}' --type code src/MoonBase/Modules/ModuleDevices.h -C 5

echo -e "\n\n=== Look at ModuleDevices updateDevices implementation ==="
cat src/MoonBase/Modules/ModuleDevices.h | grep -B 5 -A 50 "void updateDevices"

echo -e "\n\n=== Check what triggers checkReOrderSwap ==="
cat src/MoonBase/Module.cpp | grep -B 10 "checkReOrderSwap" | head -30

Length of output: 11778


@ewowi, I found the root cause! 🎯

🐛 Root Cause: Unstable Sort Comparison

The infinite flip-flop between positions 14 and 15 is caused by an unstable sort in ModuleDevices.h line 190:

std::sort(devicesVector.begin(), devicesVector.end(), 
    [](JsonObject a, JsonObject b) { return a["name"] < b["name"]; });

The Problem

  1. Two devices at positions 14 and 15 have identical or very similar names
  2. When a["name"] < b["name"] returns false for both comparisons (A < B and B < A), the elements are considered equal
  3. std::sort does NOT guarantee stable ordering for equal elements - the order can flip between sorts
  4. Each time a UDP message arrives and triggers updateDevices():
    • Sort produces order A→B (device A at 14, B at 15)
    • checkReOrderSwap detects this and swaps state in-place
    • Next sort produces order B→A (unstable!)
    • checkReOrderSwap detects reverse swap
    • Infinite loop 🔄

✅ Solution: Stable Comparison with Tie-Breaker

Replace the unstable comparison with a stable one that uses IP address as a tie-breaker:

// In ModuleDevices.h line 190, replace:
std::sort(devicesVector.begin(), devicesVector.end(), 
    [](JsonObject a, JsonObject b) { return a["name"] < b["name"]; });

// With:
std::sort(devicesVector.begin(), devicesVector.end(), 
    [](JsonObject a, JsonObject b) {
        // Primary sort: by name
        int nameCompare = strcmp(
            a["name"].as<const char*>(), 
            b["name"].as<const char*>()
        );
        if (nameCompare != 0) return nameCompare < 0;
        
        // Tie-breaker: by IP address (ensures stable sort)
        return strcmp(
            a["ip"].as<const char*>(), 
            b["ip"].as<const char*>()
        ) < 0;
    }
);

Alternative Solution: Use std::stable_sort

If you want to preserve insertion order for devices with identical names:

std::stable_sort(devicesVector.begin(), devicesVector.end(), 
    [](JsonObject a, JsonObject b) { return a["name"] < b["name"]; });

But the first solution is better because it guarantees a consistent order even for devices with the same name.


Why This Fixes It

  • ✅ Devices with identical names will be consistently ordered by IP
  • ✅ No more flip-flopping between positions
  • checkReOrderSwap will only detect actual user-initiated reordering
  • ✅ No more unnecessary writes to devices.json

Would you like me to generate a complete patch for ModuleDevices.h with this fix?


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

pio.ini: latest FastLED

backend
=======

Devices:
- sort on IP as tie breaker

Module IO:
- i2cFreq default 100
- add state.data["I2CReady"]

- SharedFSPersistence: writeToFSDelayed: cache sharedDelayedWrites

IMU driver:
- add initBoard and stopBoard and onUpdate
- use ModuleIO.state.data.I2CReady
- stopBoard at destructor

- Particles effect: gravityType (none, random, gyro)
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Around line 641-642: The multiplication for setting I2C clock overflows
because updatedItem.value.as<uint16_t>() * 1000 is done in 16-bit arithmetic;
modify the Wire.setClock call (the i2cFreq branch that checks updatedItem.name
== "i2cFreq") to cast the value to a wider type (e.g., uint32_t) before
multiplying so the multiplication is performed in 32-bit arithmetic and passes
the correct clock value to Wire.setClock.
- Around line 842-855: The else after Wire.begin(...) is missing braces so
EXT_LOGE(ML_TAG, "initI2C Wire failed") executes unconditionally and
_state.data["I2CReady"] is not properly set to false on failure; add braces
around the else block to include both setting _state.data["I2CReady"] = false
and the EXT_LOGE call. Also prevent uint16_t overflow when computing frequency *
1000 by promoting frequency to a wider type (e.g., uint32_t or cast before
multiplication) when passing to Wire.begin(pinI2CSDA, pinI2CSCL, frequency *
1000) so values ≥66 won't wrap.

In `@src/MoonLight/Nodes/Drivers/D_IMU.h`:
- Around line 37-90: The calibration calls
mpu.CalibrateAccel(6)/mpu.CalibrateGyro(6) inside the moduleIO->read() lambda
which can hold the module IO mutex for seconds; change initBoard() so the lambda
only reads state.data["I2CReady"] and, when ready and board==0, sets a
short-lived flag (e.g., needMPUInit) or posts an async task, then returns
immediately; perform the heavy work (mpu.initialize(), mpu.testConnection(),
mpu.dmpInitialize(), mpu.CalibrateAccel, mpu.CalibrateGyro, mpu.setDMPEnabled
and setting motionTrackingReady) outside the moduleIO->read() callback (either
directly after the read returns if synchronous or in a background thread/task)
so moduleIO->_accessMutex is not held during long I2C operations.
- Around line 102-114: The handler in onUpdate runs stopBoard() and initBoard()
unconditionally when control["name"] == "board"; guard that branch so it only
restarts hardware if the node is currently enabled—check the node's on state
(e.g., using the on member or this->on) and wrap the stopBoard()/initBoard()
calls in if (on) { stopBoard(); initBoard(); } so you don't initialize the MPU
when the node is off.
🧹 Nitpick comments (6)
src/MoonBase/Modules/ModuleDevices.h (1)

190-199: Good fix for the sort instability — minor null-safety note on the IP tie-breaker.

The IP tie-breaker resolves the flip-flop issue nicely. One small concern: as<const char*>() returns nullptr when the key is missing or not a string, and strcmp(nullptr, …) is undefined behavior. This is unlikely given line 174 always sets "ip", but a defensive guard is cheap:

Proposed defensive fix
-        return strcmp(a["ip"].as<const char*>(), b["ip"].as<const char*>()) < 0;
+        const char* ipA = a["ip"].as<const char*>();
+        const char* ipB = b["ip"].as<const char*>();
+        return strcmp(ipA ? ipA : "", ipB ? ipB : "") < 0;

Also, consider removing the commented-out strcmp/nameCompare lines (193-194) to keep the code clean.

src/MoonBase/SharedFSPersistence.h (3)

21-21: Global mutable sharedDelayedWrites has no synchronization.

This inline global vector is mutated from writeToFS (push_back) and consumed from writeToFSDelayed (move + iterate). If these can execute on different tasks/cores or re-entrantly (e.g., the cancel path in writeToFSDelayed calls readFromFSmodule->update → handler → writeToFS → push_back onto sharedDelayedWrites), you risk a data race.

The std::move on line 175 mitigates re-entrancy during the iteration loop, but push_back from another task while writeToFSDelayed is between lines 175–179 is still unprotected.

Worth confirming that all callers run on the same task, or consider guarding with a mutex.


35-51: Map keyed on raw const char* — ensure pointers remain valid for the map's lifetime.

_modules stores module->_moduleName as the key by pointer value. The CStrComparator correctly compares by string content, but the pointer itself must remain valid as long as the entry exists. If _moduleName is a string literal or a stable member (common for module names), this is fine. If it could ever point to freed/moved memory, lookups will dereference a dangling pointer.

Consider using std::string as the key type if module names aren't guaranteed to be string literals, or add a comment documenting the lifetime requirement.


118-140: Delayed write dedup flag hasDelayedWrite is never reset if the queue is cleared externally.

hasDelayedWrite is set to true on line 139 and only reset to false inside the lambda on line 136. If sharedDelayedWrites were ever cleared without invoking the lambdas (e.g., a future code path or error recovery), the flag stays true and that module can never queue another delayed write.

This is fine today given the current call patterns, but it's fragile. Consider resetting hasDelayedWrite for all modules in writeToFSDelayed after processing, or documenting the invariant that every queued lambda must be invoked.

src/MoonBase/Modules/ModuleIO.h (1)

858-859: pinI2CSDA / pinI2CSCL are declared in the public section but are internal state.

These tracking variables are implementation details of pin management and should be in the private section alongside current_board_id.

src/MoonLight/Nodes/Effects/E_MoonLight.h (1)

1254-1262: Axis swaps for gyro gravity look intentional but warrant a brief comment.

The mapping gravity[0] = -sharedData.gravity.x, gravity[1] = sharedData.gravity.z, gravity[2] = -sharedData.gravity.y transforms from IMU orientation to display orientation. A short inline comment explaining the coordinate convention (e.g., "IMU Z-up → display Y-up") would help future maintainers understand the swap rationale.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 14, 2026

@coderabbitai, regarding

Global mutable sharedDelayedWrites has no synchronization.
Can you propose code to guard this with a mutex (mutexes take 100+ more bytes - we are tight on heap on ESP32-D0), please provide the leanest solution

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Reading SharedFSPersistence.h to see sharedDelayedWrites usage ==="
cat src/MoonBase/SharedFSPersistence.h

echo -e "\n\n=== Search for all sharedDelayedWrites access points ==="
rg -n "sharedDelayedWrites" --type-add 'code:*.{h,cpp}' --type code -C 3

Length of output: 8687


@ewowi, Here's the leanest solution using FreeRTOS spinlocks – only 4 bytes of memory overhead:

✅ Spinlock Solution (4 bytes overhead)

In SharedFSPersistence.h, modify lines 20-21:

// ADDED: Global delayed writes queue (matches templated version)
inline portMUX_TYPE sharedDelayedWritesMux = portMUX_INITIALIZER_UNLOCKED;
inline std::vector<std::function<void(char)>> sharedDelayedWrites;

Protect push_back() in writeToFS() (around line 122):

if (info.delayedWriting) {
  if (!info.hasDelayedWrite) {
    ESP_LOGD(SVK_TAG, "delayedWrites: Add %s", info.filePath.c_str());

    portENTER_CRITICAL(&sharedDelayedWritesMux);
    sharedDelayedWrites.push_back([this, module = info.module](char writeOrCancel) {
      auto it = _modules.find(module->_moduleName);
      if (it == _modules.end()) return;

      ESP_LOGD(SVK_TAG, "delayedWrites: %c %s", writeOrCancel, it->second.filePath.c_str());

      if (writeOrCancel == 'W') {
        this->writeToFSNow(module->_moduleName);
      } else {
        // Cancel: read old state back from FS
        this->readFromFS(module->_moduleName);
        // Update UI with restored state
        it->second.module->update([](ModuleState& state) { return StateUpdateResult::CHANGED; }, SVK_TAG);
      }
      it->second.hasDelayedWrite = false;
    });
    info.hasDelayedWrite = true;
    portEXIT_CRITICAL(&sharedDelayedWritesMux);
  }
}

Protect move/clear in writeToFSDelayed() (lines 171-179):

static void writeToFSDelayed(char writeOrCancel) {
  // Move pending writes under lock
  std::vector<std::function<void(char)>> pending;
  portENTER_CRITICAL(&sharedDelayedWritesMux);
  pending = std::move(sharedDelayedWrites);
  sharedDelayedWrites.clear();  // leave in valid-but-empty state
  portEXIT_CRITICAL(&sharedDelayedWritesMux);

  ESP_LOGD(SVK_TAG, "calling %u writeFuncs from delayedWrites", pending.size());

  // Process outside of lock
  for (auto& writeFunc : pending) {
    writeFunc(writeOrCancel);
  }
}

📊 Comparison

Solution Overhead Pros Cons
portMUX_TYPE spinlock 4 bytes ✅ Minimal memory
✅ Works across cores
✅ Lock-free when uncontended
⚠️ Busy-wait under high contention
FreeRTOS mutex ~100+ bytes ✅ Task-blocking (no busy-wait) ❌ Too heavy for ESP32-D0
Critical sections 0 bytes ✅ No memory cost ❌ Disables interrupts (affects timing)

Verdict: portMUX_TYPE is the perfect balance for ESP32-D0 – only 4 bytes, thread-safe, and ideal for brief operations.


🎯 Why This Works

  1. Spinlock is extremely lightweight – just an integer flag
  2. Critical sections are brief – only protecting push_back(), move(), and clear()
  3. No contention expected – writes are infrequent (user actions), reads are even rarer (explicit save)
  4. Works across cores – safe even if httpd and SvelteKit run on different cores

This is the leanest thread-safe solution possible on ESP32! 🐰✨


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 104
File: src/MoonLight/Layers/PhysicalLayer.cpp:145-147
Timestamp: 2026-01-13T15:18:48.740Z
Learning: In MoonLight's PhysicalLayer and Node architecture (src/MoonLight/Layers/PhysicalLayer.cpp, src/MoonBase/Nodes.h): effectsMutex, driversMutex, and layerMutex are mandatory for system operation. These mutexes are validated at creation time (constructor checks log errors if xSemaphoreCreateMutex returns nullptr). Runtime null checks on these mutex pointers are intentionally omitted—the design treats successful mutex allocation as an invariant after initialization, placing responsibility on the programmer to ensure proper allocation. This is a performance-oriented "fail-fast at boot" approach rather than defensive runtime checking.
<!-- </add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T15:13:55.631Z
Learning: In MoonLight effectTask (src/main.cpp): The memcpy(channelsE, channelsD) at line 137 happens AFTER releasing swapMutex, allowing driverTask to swap the channelsD/channelsE pointers mid-copy, causing buffer corruption. This specifically breaks effects like GameOfLife that use getRGB() to read previous-frame data for blending decisions. Other effects work because they only read current-frame data they just wrote. Fix: hold swapMutex during the entire memcpy operation to prevent pointer swap during copy.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 14, 2026

@coderabbitai, regarding

Stable Comparison with Tie-Breaker

There exists double entries in the devices module, same name, same ip, can you check the devices module and avoid double entries to be added

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Read ModuleDevices.h to find device addition logic ==="
cat src/MoonBase/Modules/ModuleDevices.h

echo -e "\n\n=== Search for places where devices array is modified ==="
rg -n "devices.*push_back|devices.*add|devices\[.*\]\s*=" --type-add 'code:*.{h,cpp}' --type code src/MoonBase/Modules/ModuleDevices.h -C 5

echo -e "\n\n=== Look for receiveUDP implementation ==="
rg -n "void receiveUDP|receiveUDP\(" --type-add 'code:*.{h,cpp}' --type code src/MoonBase/Modules/ModuleDevices.h -A 50

Length of output: 20178


@ewowi, I found the root cause! 🎯

🐛 The Problem

In ModuleDevices.h line 160, device lookup checks ONLY by name:

for (JsonObject dev : devices) {
  if (dev["name"] == message.name.c_str()) {  // ← Only checks name!
    device = dev;
    newDevice = false;
    break;
  }
}

Then in the sort phase (lines 183-207), there's NO deduplication:

if (newDevice) {  // sort devices in vector and add to a new document
  // ...
  for (JsonObject device : devicesVector) {
    doc2["devices"].add(device);  // ← Adds ALL devices, including duplicates!
  }
}

How duplicates accumulate:

  1. Device sends UDP message with name "esp32" and IP "192.168.1.10"
  2. Added successfully
  3. Later, same device sends message again with same name and IP
  4. Because the name matches, it updates the existing entry (good!)
  5. But if for some reason the sort/cleanup phase runs and there are already duplicates in the state (from previous bugs or race conditions), they all get copied back

✅ The Fix: Deduplicate During Sort

Replace lines 183-207 with this:

if (newDevice) {  // sort devices in vector and add to a new document and update
  JsonDocument doc2;

  // Build deduplication map: key = "name|ip", value = device
  std::map<String, JsonObject> uniqueDevices;
  
  for (JsonObject dev : devices) {
    if (time(nullptr) - dev["lastSync"].as<time_t>() < 86400) {  // max 1 day
      String key = String(dev["name"].as<const char*>()) + "|" + String(dev["ip"].as<const char*>());
      
      // Only keep the most recent entry for each name+ip combination
      if (uniqueDevices.find(key) == uniqueDevices.end() || 
          dev["lastSync"].as<time_t>() > uniqueDevices[key]["lastSync"].as<time_t>()) {
        uniqueDevices[key] = dev;
      }
    }
  }

  // Extract unique devices into vector for sorting
  std::vector<JsonObject> devicesVector;
  for (auto& pair : uniqueDevices) {
    devicesVector.push_back(pair.second);
  }

  std::sort(devicesVector.begin(), devicesVector.end(), [](JsonObject a, JsonObject b) {
    // Primary sort: by name
    if (a["name"] != b["name"]) return a["name"] < b["name"];
    
    // Tie-breaker: by IP address (ensures stable sort)
    return strcmp(a["ip"].as<const char*>(), b["ip"].as<const char*>()) < 0;
  });

  doc2["devices"].to<JsonArray>();
  for (JsonObject device : devicesVector) {
    doc2["devices"].add(device);
  }
  JsonObject newState = doc2.as<JsonObject>();
  update(newState, ModuleState::update, _moduleName);
}

🎯 What This Fixes

  1. Deduplicates by name+IP combination - only one entry per unique name+IP pair
  2. Keeps the most recent entry - if duplicates exist, keeps the one with the latest lastSync
  3. Stable sort - devices are consistently ordered by name, then IP
  4. No more flip-flopping - combined with the stable sort, devices stay in place

📋 Additional Recommendation

Also improve the device lookup to check both name AND IP:

// Replace lines 156-166 with:
JsonObject device = JsonObject();
bool newDevice = true;
for (JsonObject dev : devices) {
  if (dev["name"] == message.name.c_str() && dev["ip"] == ip.toString()) {  // Check name AND IP
    device = dev;
    newDevice = false;
    break;
    // EXT_LOGD(ML_TAG, "updated ...%d %s", ip[3], name);
  }
}

This prevents duplicates from being added in the first place if a device changes its name but keeps the same IP (or vice versa).

Would you like me to generate the complete patched ModuleDevices.h file?


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-03T21:11:12.471Z
Learning: MoonLight ModuleDevices: To ensure group convergence despite UDP broadcast loss, updateDevices() should also apply control fields to the local device (origin "group") when a message from a group member is processed, skipping self. Also, partOfGroup() must return base == device when the base has no dots (no group prefix) instead of matching all devices.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:31:17.304Z
Learning: In MoonModules/MoonLight compareRecursive (src/MoonBase/Module.cpp): Treat an object’s "name" change as an identity change. When oldData["name"] != newData["name"], emit only the "name" UpdatedItem for that parent and short‑circuit (do not recurse into the subtree like controls). This prevents spurious controls.* updates after renames while keeping the update flow generic and not tied to specific child keys.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 14, 2026

@coderabbitai, regarding

The Fix: Deduplicate During Sort

Can this be done simpler? we now have a map and a vector as local copies, can only one of the constructs be used?

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

@ewowi, Yes! Much simpler – we can eliminate the vector entirely. 🎯

✅ Simplified Version: Map Only

The key insight: std::map is automatically sorted by its key. Since we're using "name|ip" as the key, the lexicographic string comparison gives us exactly the sort order we want (name first, then IP).

Replace lines 183-207 with this simplified version:

if (newDevice) {  // sort devices and deduplicate
  JsonDocument doc2;

  // Build deduplication map: key = "name|ip", value = device
  // std::map automatically keeps entries sorted by key (name|ip)
  std::map<String, JsonObject> uniqueDevices;
  
  for (JsonObject dev : devices) {
    if (time(nullptr) - dev["lastSync"].as<time_t>() < 86400) {  // max 1 day
      String key = String(dev["name"].as<const char*>()) + "|" + String(dev["ip"].as<const char*>());
      
      // Only keep the most recent entry for each name+ip combination
      if (uniqueDevices.find(key) == uniqueDevices.end() || 
          dev["lastSync"].as<time_t>() > uniqueDevices[key]["lastSync"].as<time_t>()) {
        uniqueDevices[key] = dev;
      }
    }
  }

  // Map is already sorted by key (name|ip), just iterate and add
  doc2["devices"].to<JsonArray>();
  for (auto& pair : uniqueDevices) {
    doc2["devices"].add(pair.second);
  }
  
  JsonObject newState = doc2.as<JsonObject>();
  update(newState, ModuleState::update, _moduleName);
}

🎯 Why This Works

Lexicographic string comparison automatically gives correct ordering:

"device1|192.168.1.10" < "device1|192.168.1.20" < "device2|192.168.1.10"
 ↑ name matches          ↑ name matches          ↑ name differs
   IP compared              IP compared             name compared first

Benefits:

  • One data structure instead of two (50% less code)
  • No manual sorting required (map handles it)
  • Clearer intent – map naturally expresses "unique by key"
  • Same memory usage – no extra vector allocation

Perfect for ESP32-D0's tight memory constraints! 🐰✨


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-03T21:11:12.471Z
Learning: MoonLight ModuleDevices: To ensure group convergence despite UDP broadcast loss, updateDevices() should also apply control fields to the local device (origin "group") when a message from a group member is processed, skipping self. Also, partOfGroup() must return base == device when the base has no dots (no group prefix) instead of matching all devices.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:31:17.304Z
Learning: In MoonModules/MoonLight compareRecursive (src/MoonBase/Module.cpp): Treat an object’s "name" change as an identity change. When oldData["name"] != newData["name"], emit only the "name" UpdatedItem for that parent and short‑circuit (do not recurse into the subtree like controls). This prevents spurious controls.* updates after renames while keeping the update flow generic and not tied to specific child keys.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:08:02.954Z
Learning: MoonLight NodeManager (src/MoonBase/NodeManager.h): After switching to direct onUpdate in httpd, compareRecursive continues with its captured newData and can emit controls updates immediately after a node rename. To avoid spurious control updates, NodeManager::onUpdate must (1) guard controls array/index, and (2) skip when the current control property already equals updatedItem.value. Only "value" and "default" should drive nodeClass->updateControl/onUpdate; other control fields are UI metadata and can be ignored or just trigger requestUIUpdate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:18:44.193Z
Learning: MoonLight NodeManager controls updates: After rename, compareRecursive continues with captured newData and will emit controls.* changes. Ensure NodeManager::onUpdate:
- Applies only controls.value/default to the node (updateControl + onUpdate), guarded by equality against current control state to avoid re-applying after rename.
- Propagates all other controls metadata (name/min/max/type/valid/size/…) to state/UI and triggers requestUIUpdate, but does not call nodeClass->updateControl for these metadata fields.
This prevents spurious back-application while keeping the UI in sync with the new control schema.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 114
File: src/MoonBase/Modules/ModuleDevices.h:76-106
Timestamp: 2026-01-26T12:59:33.564Z
Learning: In MoonLight/MoonBase: Do not add defensive guards for internal programming contracts like `updatedItem.parent` or `updatedItem.index` being empty in methods like `onUpdate()`. These should be correctly set up by the caller/framework. Adding guards masks programming errors; prefer fail-fast behavior to expose bugs immediately rather than silently handling them.

pio.ini:
- pioarduino back from 55.03.37 to 55.03.35 (p4 nano reboot crash)
- FastLED: back from 02-14 to 02-12 (c3 compile error)
- issues logged

backend
=======
- (Devices: use map, deduplicate list)
- Module IO: Wire.setClock uint32_t, private variables
- Shared FS Persistence: add sharedDelayedWrites mutex
- IMU driver: add requestInitBoard to get it out of onUpdate
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/MoonBase/SharedFSPersistence.h (2)

123-142: ⚠️ Potential issue | 🟠 Major

Heap allocation inside portENTER_CRITICAL spinlock is unsafe.

portENTER_CRITICAL disables interrupts on the current core. sharedDelayedWrites.push_back(...) may trigger a std::vector reallocation (heap allocation), and the heap allocator itself uses internal locks. Allocating memory with interrupts disabled can deadlock or cause a panic on ESP-IDF.

Use a FreeRTOS mutex (SemaphoreHandle_t with xSemaphoreTake/xSemaphoreGive) instead, which allows preemption and safe heap operations:

Proposed fix: replace spinlock with FreeRTOS mutex

Replace the global spinlock (line 22) with a FreeRTOS mutex:

-inline portMUX_TYPE sharedDelayedWritesMux = portMUX_INITIALIZER_UNLOCKED;
+inline SemaphoreHandle_t sharedDelayedWritesMux = xSemaphoreCreateMutex();

Then in writeToFS and writeToFSDelayed, replace critical sections:

-        portENTER_CRITICAL(&sharedDelayedWritesMux);
+        xSemaphoreTake(sharedDelayedWritesMux, portMAX_DELAY);
         sharedDelayedWrites.push_back([this, module = info.module](char writeOrCancel) {
           ...
         });
         info.hasDelayedWrite = true;
-        portEXIT_CRITICAL(&sharedDelayedWritesMux);
+        xSemaphoreGive(sharedDelayedWritesMux);

Note: xSemaphoreCreateMutex() at global inline scope won't work before FreeRTOS scheduler starts. Initialize it in a function (e.g., a static helper with std::call_once or in SharedFSPersistence constructor).


36-52: ⚠️ Potential issue | 🟡 Minor

Map keyed on const char* — document assumptions about _moduleName lifetime.

_modules uses raw const char* as keys via CStrComparator. This is safe only if module->_moduleName always points to long-lived storage (string literals, static strings, or memory never freed). While the current code appears to work (likely because modules are rarely deleted and _moduleName points to stable strings), this design is fragile. Consider documenting the lifetime guarantee or refactoring to use Module* as the key instead.

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Around line 932-955: updateDevices() currently always writes the runtime I2C
clock from Wire.getClock() into doc["i2cFreq"], which can overwrite the
user-configured frequency; change the behavior to only update doc["i2cFreq"]
when the measured Wire.getClock()/1000 differs from the configured value by a
meaningful tolerance (e.g., a few kHz) or omit writing it entirely and leave the
user-configured value as source of truth — locate the doc["i2cFreq"] assignment
and the update(newState, ModuleState::update, _moduleName) call and modify the
logic to compare Wire.getClock()/1000 against the existing configured value
before setting it (or skip setting), ensuring any update only happens when the
difference exceeds the chosen tolerance.
- Around line 844-845: The Wire.end() call is causing intermittent
ESP_ERR_INVALID_STATE when immediately followed by Wire.begin(); remove the
Wire.end() and the subsequent delay(100) and simply call Wire.begin(sda, scl)
where the I2C is initialized (replace the Wire.end(); delay(100); sequence). Use
Wire.begin(sda, scl) directly — it will reinitialize the bus when pins change
and is a no-op if the bus is already running with the same pins.
- Around line 843-857: The updateDevices() call from readPins() indirectly
triggers update() which fires callUpdateHandlers and re-enters readPins(),
causing a redundant Wire.end()/Wire.begin() cycle; fix this by adding a re-entry
guard (e.g., a boolean member like _inUpdateDevices) around the body of
updateDevices() and/or around readPins() so that if the guard is set you skip
running the I2C teardown/scan/initialization (the
Wire.end()/delay()/Wire.begin() path and the subsequent updateDevices() call),
set the guard at start and clear it at exit to prevent recursive calls, or
alternatively have updateDevices() write _state.data["I2CReady"] and other scan
results directly instead of calling update() so callUpdateHandlers won't be
invoked.
🧹 Nitpick comments (3)
src/MoonBase/Modules/ModuleDevices.h (1)

210-232: Remove or enable the commented-out deduplication block.

This 22-line block is dead code. If the map-based dedup approach isn't ready, consider removing it and tracking it in an issue instead. Leaving large commented blocks hampers readability and risks going permanently stale.

Note: without deduplication, a device that changes IP (e.g., DHCP lease renewal) will appear twice in the list until the 86400s staleness check at Line 188 evicts the old entry—the name-only lookup at Line 160 will match the first occurrence and stop updating the second.

src/MoonLight/Nodes/Drivers/D_IMU.h (1)

102-102: requestInitBoard is in the public section but is internal state.

Move it to the private section (after line 184) alongside the other internal members.

src/MoonBase/SharedFSPersistence.h (1)

178-179: Nit: sharedDelayedWrites.clear() after std::move is redundant.

After std::move, the source vector is in a valid-but-unspecified state — but per the C++ standard, std::vector's move constructor leaves the source empty. The .clear() is harmless but unnecessary.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 14, 2026

@coderabbitai, I have again issues with writing to fs. similar to above: I change an effect, press the save button, check files and see the effects.json file is updated with the new effect. Then I press restart and it restarts with the old effect. How on earth is this possible? If it is on fs it is pretty persistent I would say. Is the old effect preserved somewhere and written back to fs at the moment I press restart?
Please analyse again the code of SharedFSPersistence and compare with FSPersistence. Very strange it seems to be solved along the way (removing addUpdateHandler from the constructor and calling enableUpdateHandler in begin() )

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -e

echo "=== git status (branch/PR context) ==="
git status -sb || true

echo -e "\n=== SharedFSPersistence.h (full) ==="
sed -n '1,300p' src/MoonBase/SharedFSPersistence.h || true

echo -e "\n=== FSPersistence.h (full, if present) ==="
if [ -f src/MoonBase/FSPersistence.h ]; then sed -n '1,260p' src/MoonBase/FSPersistence.h; else echo "FSPersistence.h not present"; fi

echo -e "\n=== Where is writeToFSDelayed called? (who triggers Save/Cancel) ==="
rg -n --no-heading "writeToFSDelayed\\s*\\(" -C 3 || true

echo -e "\n=== Any direct calls to writeToFSNow or writeToFS (outside SharedFSPersistence) ==="
rg -n --no-heading "writeToFSNow\\s*\\(|[^D]writeToFS\\s*\\(" -g '!src/MoonBase/SharedFSPersistence.h' -C 3 || true

echo -e "\n=== Restart paths (ESP.restart/esp_restart/RestartService) ==="
rg -n --no-heading "ESP\\.restart|esp_restart|RestartService|handleRestart\\s*\\(" -C 4 || true

echo -e "\n=== Boot/init sequence (registerModule/begin) ==="
rg -n --no-heading "registerModule\\s*\\(|enableUpdateHandler\\s*\\(|disableUpdateHandler\\s*\\(|readFromFS\\s*\\(|begin\\s*\\(" src -C 3 || true

echo -e "\n=== Which modules map to effects.json / lightscontrol.json ==="
rg -n --no-heading "\\.config\\/effects\\.json|\\.config\\/lightscontrol\\.json" -C 3 || true

echo -e "\n=== Search for modules named 'effects' or LightsControl ==="
rg -n --no-heading "class\\s+Module\\w+|_moduleName|\"effects\"|\"lightscontrol\"" src -C 2 || true

echo -e "\n=== ModuleLightsControl.h (first 500 lines) ==="
sed -n '1,500p' src/MoonLight/Modules/ModuleLightsControl.h || true

echo -e "\n=== Any other persistence class still used? (FSPersistence) ==="
rg -n --no-heading "FSPersistence<|new\\s+FSPersistence" -C 2 || true

echo -e "\n=== Any code touching effects on Restart or just before? ==="
rg -n --no-heading "effects|VU Meter|Game of Life|Effect" src -g '!**/*.md' -C 2 | head -200 || true

echo -e "\n=== main.cpp (init order around persistence and modules) ==="
sed -n '1,260p' src/main.cpp || true

Length of output: 113456


@ewowi Thanks for the detailed report and logs — I dug through the current tree and compared SharedFSPersistence against the templated FSPersistence. The init/order is now correct: SharedFSPersistence reads all module files first in begin(), then enables per‑module update handlers. That realigns behavior with FSPersistence and prevents the “defaults overwrite” bug we fixed earlier.

What’s happening now is different: there’s still an out‑of‑band writer to /.config/effects.json that bypasses SharedFSPersistence.

  • In ModuleLightsControl::onUpdate(), when preset.action == "click", the code does a direct copyFile(presetX.json → "/.config/effects.json") and then raises a file-change event so NodeManager will reread the file.
  • In ModuleLightsControl::loop(), if presetLoop > 0, it periodically fabricates a “click” (next preset) and again raises the "/.config/effects.json" change event.

This means:

  • Save writes the effects state to effects.json via SharedFSPersistence (correct).
  • Any subsequent preset “click” (manual or via presetLoop) directly overwrites effects.json from a preset file outside of SharedFSPersistence.
  • If that happens after you checked the file but before the restart actually occurs, the last writer wins — the file now contains the old preset again, and on boot SharedFSPersistence faithfully loads it.

So the reversion isn’t caused by SharedFSPersistence; it’s caused by ModuleLightsControl writing effects.json directly after Save.

Recommended fixes (pick one):

Option A — Keep Save semantics (best UX)

  • Don’t write effects.json on preset click. Instead, load the preset JSON and apply it to the ModuleEffects state in-memory; mark saveNeeded so the UI shows the Save button. Let SharedFSPersistence persist only when the user presses Save.

Patch sketch (inside ModuleLightsControl::onUpdate preset click branch):

// Replace: copyFile(presetFile, "/.config/effects.json"); _fileManager->update(...effects.json...);

File f = ESPFS.open(presetFile.c_str(), "r");
if (f) {
  JsonDocument presetDoc;
  if (deserializeJson(presetDoc, f) == DeserializationError::Ok) {
    f.close();
    // Apply preset to effects module state without propagation (no handler write)
    if (auto* effects = modules::find<ModuleEffects>()) {
      effects->updateWithoutPropagation(presetDoc.as<JsonObject>(), ModuleState::update, "effects");
      // expose unsaved change in UI (whatever your save flag/UI mechanism is)
      _sveltekit->getFileManager()->setSaveNeeded(true);
    }
  } else {
    f.close();
  }
}

Option B — Persist immediately on click (maintain current “click = commit” behavior, but route through persistence)

  • Apply preset to ModuleEffects in-memory, then persist via SharedFSPersistence instead of raw file copy.

Patch sketch:

// 1) Apply to state (as above)
effects->updateWithoutPropagation(presetDoc.as<JsonObject>(), ModuleState::update, "effects");

// 2) Persist through SharedFSPersistence (no delayed queue for this single write)
sharedFsPersistence->disableUpdateHandler("effects");     // avoid feedback
sharedFsPersistence->writeToFSNow("effects");             // single, atomic write
sharedFsPersistence->enableUpdateHandler("effects");

Why this helps

  • All effects.json writes go through one path (SharedFSPersistence). You won’t get “Save writes new → preset later rewrites old” surprises anymore.
  • Option A preserves the clear “edit → Save → Restart” flow. Option B keeps “click = commit now” but still centralizes the write.

Nice-to-have hardening

  • Before reboot, flush any pending delayed writes:
    • In RestartService::restartNow(), call SharedFSPersistence::writeToFSDelayed('W') just before ESP.restart().
  • Add a tiny breadcrumb to both paths that can write effects.json:
    • Log “effects.json write by Save” vs “write by preset click” with a short content hash to confirm who was the last writer.

Key takeaways

  • SharedFSPersistence is behaving like FSPersistence now.
  • The only remaining actor that can overwrite effects.json after Save is the preset “click”/presetLoop path in ModuleLightsControl via copyFile().
  • Centralizing all effects.json writes through SharedFSPersistence (and stopping raw copies) will resolve the reappearing “old effect” after restart.

If you want, I can prep a small PR plan to switch preset handling to Option A (save semantics) or Option B (immediate persist via SharedFSPersistence).


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-03T21:11:12.471Z
Learning: MoonLight ModuleDevices: To ensure group convergence despite UDP broadcast loss, updateDevices() should also apply control fields to the local device (origin "group") when a message from a group member is processed, skipping self. Also, partOfGroup() must return base == device when the base has no dots (no group prefix) instead of matching all devices.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:31:17.304Z
Learning: In MoonModules/MoonLight compareRecursive (src/MoonBase/Module.cpp): Treat an object’s "name" change as an identity change. When oldData["name"] != newData["name"], emit only the "name" UpdatedItem for that parent and short‑circuit (do not recurse into the subtree like controls). This prevents spurious controls.* updates after renames while keeping the update flow generic and not tied to specific child keys.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:08:02.954Z
Learning: MoonLight NodeManager (src/MoonBase/NodeManager.h): After switching to direct onUpdate in httpd, compareRecursive continues with its captured newData and can emit controls updates immediately after a node rename. To avoid spurious control updates, NodeManager::onUpdate must (1) guard controls array/index, and (2) skip when the current control property already equals updatedItem.value. Only "value" and "default" should drive nodeClass->updateControl/onUpdate; other control fields are UI metadata and can be ignored or just trigger requestUIUpdate.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 14, 2026

@coderabbitai, the issue with not saving new state in filesystem had to do with presets overruling, new commit to solve that will follow. Also found out that httpd stack size must be increased to 8192 (from 6K) to deal with heavy updates. The stack size can decrease very fast in some situations. Could you do an analysis in the onUpdate functions where large amounts of stack size is used (and not freed, e.g. due to lots of onUpdates Called in a hierarchy). Where can I save on stack? I guess Strings are candidates. I use Char a lot , which saves all on stacked, should we change some of them to String (which also uses heap)? It's a difficult choice as heap on non-psram devices is sparse . So any ideas for smart solutions welcome

@coderabbitai
Copy link

coderabbitai bot commented Feb 14, 2026

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

@ewowi Thanks for the detailed context. I’ll start by inventorying stack-heavy patterns across onUpdate paths and JSON handling, then follow up with targeted fixes. Kicking off a focused code scan now.

While that runs, here are immediate, low-risk wins you can apply (even before we pinpoint exact lines):

Quick, surgical mitigations

  • Add a stack probe (compile-out in release) to identify worst offenders in minutes:
    struct StackProbe {
      const char* tag;
      UBaseType_t before;
      StackProbe(const char* t) : tag(t), before(uxTaskGetStackHighWaterMark(nullptr)) {}
      ~StackProbe() {
        UBaseType_t after = uxTaskGetStackHighWaterMark(nullptr);
        if (after < before) EXT_LOGW("STACK", "[%s] used %u bytes (high-water now %u words)", tag, (before - after) * sizeof(StackType_t), after);
      }
    };
    // Use at top of hot paths:
    // StackProbe _sp("ModuleDevices::onUpdate");
    // StackProbe _sp("NodeManager::onUpdate");
  • Move big JSON buffers off httpd stack:
    • Replace large StaticJsonDocument locals inside onUpdate/httpd code with either:
      • file-scope static StaticJsonDocument guarded by a mutex (one per task), or
      • DynamicJsonDocument with small, known capacity and reuse the instance (static) to avoid re-allocations.
    • Prefer const JsonObjectConst& (views) over by-value JsonObject in function params to avoid deep copies.
  • Break heavy work out of onUpdate (httpd) into SvelteKit loop:
    • You already did this for rename; extend the pattern to preset application and large controls rebuilds. onUpdate should enqueue a small job and return.
  • Strings vs Char (guideline):
    • Keep Char for truly small, fixed-width keys (e.g., names up to 20–32). Audit any N > 64 in deep call chains; convert to String (with reserve) or pass as const char* view.
    • For String, always pre-reserve to avoid heap churn: s.reserve(expected).
    • Avoid String concatenation in tight paths; prefer snprintf into a small stack buffer (<=64B) or streaming to ArduinoJson.
  • Logging:
    • Wrap heavy log format strings with F("...") to keep them in flash; avoid building temporary Strings for logs on hot paths.
  • JSON building:
    • Reuse documents: prefer doc.clear() over redeclaration each call.
    • For UI rows, write directly into the target JsonArray via createNestedObject() instead of staging intermediates.

Once the scan returns, I’ll point to the biggest stack consumers (files/lines) and propose precise patches (swap StaticJsonDocument to pooled, shrink Char, or defer work). Also happy to suggest where increasing httpd to 8192 is still necessary (e.g., deep effect graph updates) and where we can claw back 1–2 KB with the changes above.


🏁 Script executed:

#!/bin/bash
set -e

echo "=== onUpdate definitions (file:line) ==="
rg -nP --type-add 'cpp:*.{h,cpp,ino}' --type cpp '\bonUpdate\s*\(' -C2 || true

echo -e "\n=== StaticJsonDocument<> on stack (size, context) ==="
rg -nP --type cpp -n -C2 'StaticJsonDocument\s*<\s*(\d+)[uUlL]?\s*>' || true

echo -e "\n=== DynamicJsonDocument/JsonDocument locals (heap JSON) ==="
rg -nP --type cpp -n -C2 '\b(DynamicJsonDocument|JsonDocument)\b' || true

echo -e "\n=== Large local arrays on stack (char[N>=256]) ==="
rg -nP --type cpp -n -C2 '\bchar\s+\w+\s*\[\s*(\d{3,})\s*\]' || true

echo -e "\n=== Char<N> usage (largest N first) ==="
rg -nP --type cpp -n 'Char<\s*(\d+)\s*>' -o | sed -E 's/^.*Char<\s*([0-9]+)\s*>.*$/\1/' | sort -nr | uniq | head -30
echo -e "\n--- Char<N> call sites ---"
rg -nP --type cpp -n -C2 'Char<\s*(\d+)\s*>' || true

echo -e "\n=== Recursive/combinatorial update flows ==="
rg -nP --type cpp -n -C3 'compareRecursive\s*\(|update\s*\(' | head -200 || true

echo -e "\n=== addUpdateHandler lambdas (check captures) ==="
rg -nP --type cpp -n -C3 'addUpdateHandler\s*\(' || true

echo -e "\n=== ArduinoJson copy points (serialize/deserialize/as<>) ==="
rg -nP --type cpp -n -C2 'serializeJson|deserializeJson|as<JsonObject>|to<JsonArray>' | head -200 || true

Length of output: 99006


@ewowi Thanks for the dump — super helpful. Here’s a focused analysis with concrete, low-risk wins that reduce httpd stack without pushing much onto heap.

What’s actually using stack

  • main.cpp Line 207: custom_vprintf has char buffer[256]. Every log consumes ~256 B of httpd stack (plus formatting frames). Under cascaded onUpdate → compareRecursive → nodeClass->onUpdate with lots of EXT_LOGx, this adds up quickly.
  • Deep call chains in httpd:
    • SharedWebSocketServer.h Lines 66, 91, 105: handle/serialize state with JsonDocument and Strings, then transmit. The JsonDocument object itself is small (heap-backed), but each call adds frames and formatting/logging overhead on httpd.
    • NodeManager.h Lines 129, 268, 286: NodeManager::onUpdate logs updatedItem with value.as() — this allocates a temporary String (heap) and extends the call chain stack depth.
  • Char temporaries appear often in loops (many Char<32> for short labels). Each instance is 32 B; not big alone, but in nested loops/onUpdate bursts they contribute.

Top hotspots to address first

  1. Logging buffer on stack (biggest single chunk)
  • File: src/main.cpp Line 207
  • Fix: move the 256B buffer off the stack and guard with a spinlock. Net stack saving per log call: ~256 B.

Example patch

// main.cpp
static portMUX_TYPE logMux = portMUX_INITIALIZER_UNLOCKED;

static int custom_vprintf(const char* fmt, va_list args) {
  static char s_buffer[256];  // moved off stack
  portENTER_CRITICAL(&logMux);
  int len = vsnprintf(s_buffer, sizeof(s_buffer), fmt, args);
  // ... existing sink (UART / ring / whatever) using s_buffer ...
  portEXIT_CRITICAL(&logMux);
  return len;
}
  • Optionally also reduce to 128 if your longest formatted line fits. This alone noticeably increases httpd headroom when logs spike during bulk updates.
  1. Avoid temporary String creation in hot logs
  • File: src/MoonBase/NodeManager.h Line 129
    Replace value.as() with a small, non-allocating print. ArduinoJson lets you serialize directly into a tiny stack buffer.

Example

// before (allocates a String)
updatedItem.value.as<String>().c_str()

// after (no heap; bounded stack)
char vbuf[48];
size_t n = serializeJson(updatedItem.value, vbuf, sizeof(vbuf));
vbuf[MIN(n, sizeof(vbuf)-1)] = 0; // ensure NUL
EXT_LOGD(ML_TAG, "%s[%d]%s[%d].%s = %s -> %s",
  updatedItem.parent[0].c_str(), updatedItem.index[0],
  updatedItem.parent[1].c_str(), updatedItem.index[1],
  updatedItem.name.c_str(), updatedItem.oldValue.c_str(), vbuf);
  • Also wrap heavy logs with if (ESP_LOG_LEVEL >= LOG_DEBUG) or equivalent, and F("...") to place format strings in flash.
  1. Shrink Char where content is short (keeps stack, avoids heap)
  • Safe downsizes spotted by grep output:
    • ModuleChannels.h Lines 34–36: layerName “Layer %d” → Char<12> (was Char<32>).
    • ModuleEffects.h Lines 66–68: same “Layer %d” → Char<12>.
    • D_ParallelLEDDriver.h Lines 91–104: statusString can start with Char<24> (or Char<16> if you constrain concatenations), tmp already Char<12>.
    • D_FastLED.h Lines 25–27: version/status/engine Char<32> — consider Char<24> if values are bounded.
    • NodeManager.h Line 25 defaultNodeName: if UI permits ≤24 chars, reduce to Char<24>.
  • UpdatedItem in Module.h uses Char<20> fields already; that’s a good compromise (don’t shrink unless you prove UI names <16).
  1. Defer heavy work out of httpd onUpdate (you’ve done this for rename — extend the pattern)
  • NodeManager.h Lines 268/286: nodeClass->onUpdate happens under httpd. For expensive nodes (effects rebuilding buffers, particle init, etc.), enqueue a small job to the SvelteKit loop and return; the job runs on its task with larger stack.
  • ModuleLightsControl.h Line 317+: same trick for preset application and any list rebuilds.

Tiny job queue pattern

// A small lock-free ring or a guarded vector of std::function<void()>
struct PendingJob { void (*fn)(void*), *arg; };
volatile uint8_t head, tail;
PendingJob jobs[8];

// in onUpdate (httpd ctx): enqueue light job
enqueueJob([](void* a){ static_cast<NodeClass*>(a)->applyPreset(); }, nodePtr);

// in SvelteKit loop task:
while (dequeueJob(&job)) job.fn(job.arg);
  1. Reuse a task-local JsonDocument in httpd/WS paths
  • SharedWebSocketServer.h Lines 66/91/105 and lib/framework/WebSocketServer.h Lines 89/119/139 repeatedly create JsonDocument locals. The object is small, but creation adds frames and allocator churn.
  • Make a static Task-Local document per task, clear() each time.

Example (header-private helper)

static JsonDocument& taskDoc() {
  static JsonDocument doc;  // one per task due to FreeRTOS TLS? If not available, make it file-static in this TU
  doc.clear();
  return doc;
}
// usage
auto& doc = taskDoc();
JsonObject root = doc.to<JsonObject>();

Note: If multiple clients handled concurrently in the same task, guard with a small portMUX around serialize+send.

  1. Make logging cheap everywhere
  • Always use F("...") or constexpr format strings.
  • Avoid concatenating Strings in logs; use %d/%s with primitive args.
  • Where you must build temporary strings, reserve() once for known sizes.

Where Char vs String makes sense

  • Keep Char for:
    • Fixed-size, short identifiers frequently passed through deep call chains (names, small file keys). They’re fast, deterministic, and avoid heap fragmentation.
  • Prefer String (with reserve()) for:
    • Rare, possibly longer text (e.g., scrolling text input, status strings that can exceed 24–32 chars).
    • Places where Char<32> is repeatedly constructed in a loop but actual content is sometimes larger — using String avoids silent truncation. Remember to reserve() a tight bound to limit heap.
  • Before switching any hot-path type to String on non-PSRAM boards, try shrinking N (as above). That saves stack without new heap pressure.

Add a lightweight StackProbe to find remaining culprits
Drop this at the top of:

  • NodeManager::onUpdate (129)
  • ModuleLightsControl::onUpdate (317)
  • ModuleDevices::onUpdate (79)
  • SharedWebSocketServer::transmitData()/handler (105/66)
  • ModuleIO::onUpdate (617)
    It will show exact bytes consumed per path so we can iterate precisely.
struct StackProbe {
  const char* tag; UBaseType_t hi{};
  StackProbe(const char* t): tag(t), hi(uxTaskGetStackHighWaterMark(nullptr)) {}
  ~StackProbe() {
    UBaseType_t now = uxTaskGetStackHighWaterMark(nullptr);
    if (now < hi) EXT_LOGW("STACK", "[%s] used %u bytes (HWM now %u words)", tag,
      (hi - now) * sizeof(StackType_t), now);
  }
};
`#define` STACK_PROBE(tag) StackProbe _sp_##__LINE__{tag}

Usage:

void onUpdate(...) override {
  STACK_PROBE("NodeManager::onUpdate");
  ...
}

Quick wins summary (ordered by impact vs. effort)

  • Move logger buffer off stack (main.cpp:207) and optionally reduce to 128 bytes.
  • Wrap heavy logs with level checks and F("..."); avoid value.as() in logs.
  • Shrink obvious Char<32> to Char<12>/16 where content is bounded (“Layer %d”, short labels).
  • Defer expensive work from httpd onUpdate to SvelteKit loop (you’re already doing this for rename).
  • Optional: task-local JsonDocument for WS/httpd handlers to reduce per-call churn (stack frames + allocator).

If you want, I can prep tiny diffs for:

  • main.cpp (logger change)
  • NodeManager.h (log formatting without String + STACK_PROBE)
  • ModuleChannels.h/ModuleEffects.h/D_ParallelLEDDriver.h (Char downsizes)
  • SharedWebSocketServer.h (task-local doc)

Also, good call on moving httpd stack to 8192. With the above, it should stay well clear of the cliff even during bursty updates.


✏️ Learnings added
Learnt from: ewowi
URL: https://github.com/MoonModules/MoonLight/pull/124

Timestamp: 2026-02-14T19:58:39.107Z
Learning: ESP32-P4/httpd: Heavy onUpdate cascades (preset apply, rename, controls rebuild) can exhaust 6 KB httpd stack; 8192 bytes is the safe default. Defer heavy work from onUpdate (httpd) to the SvelteKit loop and avoid large StaticJsonDocument locals in onUpdate to preserve stack headroom.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.686Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-03T21:11:12.471Z
Learning: MoonLight ModuleDevices: To ensure group convergence despite UDP broadcast loss, updateDevices() should also apply control fields to the local device (origin "group") when a message from a group member is processed, skipping self. Also, partOfGroup() must return base == device when the base has no dots (no group prefix) instead of matching all devices.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:31:17.304Z
Learning: In MoonModules/MoonLight compareRecursive (src/MoonBase/Module.cpp): Treat an object’s "name" change as an identity change. When oldData["name"] != newData["name"], emit only the "name" UpdatedItem for that parent and short‑circuit (do not recurse into the subtree like controls). This prevents spurious controls.* updates after renames while keeping the update flow generic and not tied to specific child keys.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:08:02.954Z
Learning: MoonLight NodeManager (src/MoonBase/NodeManager.h): After switching to direct onUpdate in httpd, compareRecursive continues with its captured newData and can emit controls updates immediately after a node rename. To avoid spurious control updates, NodeManager::onUpdate must (1) guard controls array/index, and (2) skip when the current control property already equals updatedItem.value. Only "value" and "default" should drive nodeClass->updateControl/onUpdate; other control fields are UI metadata and can be ignored or just trigger requestUIUpdate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 93
File: lib/framework/ESP32SvelteKit.cpp:229-229
Timestamp: 2026-01-04T09:31:07.533Z
Learning: In lib/framework/ESP32SvelteKit.cpp, the SvelteKit loop task stack size of 6KB (6 * 1024 bytes) for non-PSRAM devices has been validated through testing, showing approximately 1500 bytes of free stack remaining under load, providing adequate ~24% headroom.

pio.ini: http stack to 8192 for heavy updates

Front end
========
Field Renderer: Pad: if the same button selected, set selected to 255

Backend
=======
- general remove as<String> in logging
- Main: Module Effects add moduleLightsControl argument, add loop20msDrivers
- Module IO: remove Wire.end()
- Module lights control: remove preset selection if pressed again
- Module effects: add onUpdate to clear preset if update happens
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Line 849: The direct assignment to _state.data["I2CReady"] bypasses the
module's update() → persistence → UI sync pipeline; replace the direct mutation
with a call to update() (or the module's state setter) to set "I2CReady" so
subscribers and persistence are notified, e.g., use update({"I2CReady": true})
or the equivalent state API used elsewhere in ModuleIO, and if you intended to
avoid triggering handlers during init, add a concise comment next to the line
explaining that behavior and why notifications are suppressed.

In `@src/MoonLight/Layers/PhysicalLayer.cpp`:
- Around line 143-150: loop20msDrivers currently calls node->loop20ms() without
synchronization or yielding; this risks concurrent access to sharedData and task
starvation. Update PhysicalLayer::loop20msDrivers so each node->loop20ms() call
is wrapped with the node's layerMutex (use xSemaphoreTake(*node->layerMutex,
portMAX_DELAY) before and xSemaphoreGive(*node->layerMutex) after) and call
addYield(10) after releasing the mutex, matching the pattern used in loopDrivers
to prevent data races and watchdog starvation.

In `@src/MoonLight/Modules/ModuleEffects.h`:
- Around line 301-319: The onUpdate override currently triggers a full
_moduleLightsControl read and recreates a stack JsonDocument on every UI update;
change onUpdate (and its NodeManager::onUpdate call) to early-return unless the
UpdatedItem indicates a meaningful change (e.g., name or on toggle) to scope the
deselect, then inside the _moduleLightsControl->read lambda check the existing
preset selected value (state.data["preset"]["selected"]) and skip calling
_moduleLightsControl->update if it is already 255 to avoid redundant cascades;
also avoid copying the entire preset/list into a stack JsonDocument by
constructing a minimal object containing only preset.select and preset.selected
(or use ModuleState update helpers) and change EXT_LOGI("remove preset") to a
verbose/debug level (EXT_LOGV/EXT_LOGD); finally remove the pointless if
(newState.size()) guard.

In `@src/MoonLight/Nodes/Drivers/D_FastLED.h`:
- Around line 347-352: The destructor ~FastLEDDriver() currently calls
events.onChannelCreated.clear() and events.onChannelEnqueued.clear() and invokes
FastLED.reset(...), which removes all listeners and assumes a singleton; to fix,
either (A) track the listener handles returned when adding listeners to
FastLED.channelEvents() (store them as members, e.g.
onChannelCreatedHandle/onChannelEnqueuedHandle) and in ~FastLEDDriver() call the
corresponding remove/unregister using those handles instead of clear(), or (B)
enforce singleton semantics on FastLEDDriver by deleting copy/move constructors
and adding a static instance guard (and document it) so
checkAndAlloc<FastLEDDriver>(name) cannot create multiple instances; choose one
approach and update the destructor and class declaration accordingly, leaving
FastLED.reset(ResetFlags::CHANNELS) only if singleton is guaranteed.
🧹 Nitpick comments (3)
src/MoonBase/Modules/ModuleIO.h (2)

580-595: Misleading comment on line 584: says "ESP32-C3" but the condition also covers S2 and S3.

The #elif covers ESP32S2, ESP32S3, and ESP32C3, but the inline comment only mentions C3. This could mislead someone picking different default pins for S2 vs S3.

Proposed fix
  `#elif` defined(CONFIG_IDF_TARGET_ESP32S2) || defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C3)
-      pinAssigner.assignPin(8, pin_I2C_SDA);  // ESP32-C3
+      pinAssigner.assignPin(8, pin_I2C_SDA);  // ESP32-S2/S3/C3
       pinAssigner.assignPin(9, pin_I2C_SCL);

930-954: updateDevices() is public and performs a full 127-address I2C bus scan — consider guarding against concurrent or repeated calls.

If updateDevices() is ever called from httpd context (e.g., via an update handler chain), the blocking scan (~127 Wire.beginTransmission/endTransmission round-trips) can stall the calling task. Combined with the re-entry issue from readPins(), this can double the scan cost. Consider:

  1. Making it private (only called internally from readPins).
  2. Adding a debounce or "already scanned" guard if it should remain public.
src/main.cpp (1)

192-196: loop20msDrivers() is gated by newFrameReady — sensor drivers won't tick when no frames are produced.

The 20ms driver loop only runs inside the if (newFrameReady) branch (line 183). During normal operation frames flow continuously so this is fine, but if effects are paused or during extended layout/mapping phases (isPositions != 0), sensor nodes like the IMU won't get their loop20ms() calls.

If future drivers need guaranteed periodic polling regardless of frame state, consider moving the 20ms block outside the if (newFrameReady) guard — or at least outside the if (layerP.lights.header.isPositions == 0) check at line 182.

Possible restructuring to decouple 20ms from frame readiness
       esp32sveltekit.lps++;
        layerP.loopDrivers();
-
-       if (millis() - last20ms >= 20) {
-         last20ms = millis();
-         layerP.loop20msDrivers();
-       }
       }
     }
 
+    // Run 20ms driver tasks regardless of frame state
+    if (millis() - last20ms >= 20) {
+      last20ms = millis();
+      layerP.loop20msDrivers();
+    }
+
     if (!mutexGiven) xSemaphoreGive(swapMutex);  // not double buffer or if conditions not met
     vTaskDelay(1);

Comment on lines 347 to 352
~FastLEDDriver() override {
auto& events = FastLED.channelEvents();
events.onChannelCreated.clear();
events.onChannelEnqueued.clear();
FastLED.reset(ResetFlags::CHANNELS); // global: ensure only one FastLEDDriver instance exists. If multiple driver nodes are possible, this destructor will tear down channels for all of them. If singleton is guaranteed by design, consider documenting that assumption at the class level.
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# First, examine the FastLEDDriver class definition to check for singleton enforcement
# (deleted copy constructor, private constructor, etc.)
grep -n "class FastLEDDriver" src/MoonLight/Nodes/Drivers/D_FastLED.h

Repository: MoonModules/MoonLight

Length of output: 109


🏁 Script executed:

#!/bin/bash
# Check if there are explicit singleton guards or design patterns in the class
head -100 src/MoonLight/Nodes/Drivers/D_FastLED.h | tail -50

Repository: MoonModules/MoonLight

Length of output: 1977


🏁 Script executed:

#!/bin/bash
# Look at the setup() method where listeners are added to understand the pattern
sed -n '60,100p' src/MoonLight/Nodes/Drivers/D_FastLED.h

Repository: MoonModules/MoonLight

Length of output: 1707


🏁 Script executed:

#!/bin/bash
# Search for how many FastLEDDriver instances are created in the codebase
rg "FastLEDDriver" --type=cpp -B2 -A2 | head -50

Repository: MoonModules/MoonLight

Length of output: 2211


🏁 Script executed:

#!/bin/bash
# Check the full destructor context and any comments about singleton design
sed -n '340,360p' src/MoonLight/Nodes/Drivers/D_FastLED.h

Repository: MoonModules/MoonLight

Length of output: 680


.clear() removes all listeners and offers no protection against multiple driver instances.

Lines 349–350 clear every subscriber on onChannelCreated and onChannelEnqueued. While the comment on line 351 acknowledges the singleton assumption for FastLED.reset(), the same caveat applies to the event clearing above it. However, the class provides no enforcement of singleton semantics—there are no deleted copy/move constructors, and the allocation pattern in ModuleDrivers.h uses checkAndAlloc<FastLEDDriver>(name), which permits multiple instances.

If a second FastLEDDriver is instantiated and then destroyed, its destructor will wipe all event listeners (including those from the first instance), breaking the first driver's operation.

Store the listener handles returned by .add() and remove only those in the destructor, or enforce singleton semantics explicitly (deleted copy/move constructors, or static instance guard).

🤖 Prompt for AI Agents
In `@src/MoonLight/Nodes/Drivers/D_FastLED.h` around lines 347 - 352, The
destructor ~FastLEDDriver() currently calls events.onChannelCreated.clear() and
events.onChannelEnqueued.clear() and invokes FastLED.reset(...), which removes
all listeners and assumes a singleton; to fix, either (A) track the listener
handles returned when adding listeners to FastLED.channelEvents() (store them as
members, e.g. onChannelCreatedHandle/onChannelEnqueuedHandle) and in
~FastLEDDriver() call the corresponding remove/unregister using those handles
instead of clear(), or (B) enforce singleton semantics on FastLEDDriver by
deleting copy/move constructors and adding a static instance guard (and document
it) so checkAndAlloc<FastLEDDriver>(name) cannot create multiple instances;
choose one approach and update the destructor and class declaration accordingly,
leaving FastLED.reset(ResetFlags::CHANNELS) only if singleton is guaranteed.

Firmware
- pio.ini: latest FastLED

backend
=======
- IO: I2S: maintain assigned pins and only change if updated -> _triggerUpdateI2S ->  _updateI2SDevices
- Physical layer: loop20ms in mutex
- Module effects: refactor onUpdate preset reset, calling triggerResetPreset
- Artnet out: yield and delay tweaks
- Effects: VU meter: circles within size
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Fix all issues with AI agents
In `@src/MoonBase/Modules/ModuleIO.h`:
- Around line 830-861: The code uses I2S-named variables for I2C logic causing
confusion; rename pinsI2SChanged to pinsI2CChanged, _triggerUpdateI2S to
_triggerUpdateI2C, and any methods like _updateI2SDevices to _updateI2CDevices
(and adjust related log messages such as "initI2C" checks) throughout the
ModuleIO class so identifiers reflect I2C usage (e.g., references in the loop
over _state.data["pins"], the Wire.begin block, and any other places that
read/update _pinI2CSDA/_pinI2CSCL); update all references/usages and tests
accordingly to preserve behavior while fixing names.

In `@src/MoonLight/Nodes/Drivers/D_ArtnetOut.h`:
- Line 186: The packet-sending loop in D_ArtnetOut (the for loop that calls
writePackage() for each light/universe) currently has addYield(10) commented
out, which can starve the IDLE task on large setups; restore yielding by
re-enabling addYield(10) immediately after the writePackage() calls (or
alternatively insert a vTaskDelay(1) or addYield(10) every N packets) so the
loop yields frequently enough while still using the existing vTaskDelayUntil
scheduling; modify the loop around writePackage() inside the ArtNet driver
(D_ArtnetOut) to reinstate yielding.

In `@src/MoonLight/Nodes/Effects/E_MoonLight.h`:
- Around line 1254-1258: The code in the gravity assignment block (when
gravityType == 2) uses integer division because sharedData.gravity.{x,y,z} and
INT16_MAX are ints, causing outputs to be only -1,0,1; change the division to
floating-point (e.g. cast numerator or divisor to float or use 32767.0f) when
assigning gravity[0], gravity[1], and gravity[2] so the calculations produce
fractional values (keep the existing sign flips and Y/Z swap logic intact).
🧹 Nitpick comments (4)
src/MoonLight/Modules/ModuleEffects.h (2)

298-316: Good: preset reset deferred from httpd to SvelteKit loop — addresses stack and frequency concerns.

Moving the preset-deselect logic out of onUpdate (httpd task) into loop() with a coalescing boolean flag is a solid improvement: it avoids httpd stack pressure from JsonDocument + cascading update() calls, and naturally deduplicates rapid slider-drag updates into a single reset per loop iteration. The selected != 255 guard at line 305 prevents redundant cross-module updates.

One remaining concern: line 310 copies the entire state.data["preset"] object (including the list array) into a stack-local JsonDocument. If the preset list grows large, this is unnecessary memory pressure. Consider constructing a minimal object with only select and selected:

Suggested minimal state construction
-             JsonDocument doc;
-             JsonObject newState = doc.to<JsonObject>();
-
-             EXT_LOGD(ML_TAG, "remove preset");
-             newState["preset"] = state.data["preset"];
-             newState["preset"]["select"] = 255;
+             JsonDocument doc;
+             JsonObject newState = doc.to<JsonObject>();
+
+             EXT_LOGD(ML_TAG, "remove preset");
+             newState["preset"]["select"] = 255;
+             newState["preset"]["selected"] = 255;

This avoids deep-copying the preset list and only sends the fields that compareRecursive needs to act on. Based on learnings, heavy onUpdate cascades can exhaust stack; keeping the payload minimal is good hygiene even in the loop context.


319-325: triggerResetPreset is a plain bool shared across tasks — benign here but worth a note.

onUpdate runs in httpd; loop() runs in the SvelteKit task. The flag is a simple set-true / read-and-clear pattern, so the worst case is one extra or one missed reset cycle — both harmless. If you ever need stricter guarantees, std::atomic<bool> is zero-cost on ESP32 and makes the cross-task intent explicit.

Also, every UI-originated onUpdate (including high-frequency slider drags) sets this flag. Thanks to the coalescing in loop() and the != 255 guard, the actual cross-module update fires at most once, which is fine. Just confirming the design is intentional — if only specific control changes (e.g., node name, on/off) should deselect the preset, scoping the trigger would further reduce unnecessary read() calls.

src/MoonLight/Nodes/Effects/E_MoonLight.h (1)

1589-1593: Potential out-of-bounds access on sharedData.bands when band exceeds NUM_GEQ_CHANNELS / 2.

The loop runs nHorizontal * nVertical = 8 iterations, and each iteration accesses sharedData.bands[2 * (band++)], giving indices 0, 2, 4, 6, 8, 10, 12, 14. This works when NUM_GEQ_CHANNELS is 16, but the user-facing bands slider (line 1574) is not consulted—changing it has no effect on this loop. If NUM_GEQ_CHANNELS were ever reduced or if nHorizontal/nVertical are made configurable, this would read out of bounds.

Consider using the bands control variable to cap the iteration or adding a bounds guard on the index.

src/MoonBase/Modules/ModuleIO.h (1)

951-955: _I2CFreq is declared but never read or written.

This member is unused — i2cFreq is read directly from _state.data["i2cFreq"] on line 850 and from Wire.getClock() in _updateI2SDevices. Remove it to avoid confusion.

  uint8_t _pinI2CSDA = UINT8_MAX;
  uint8_t _pinI2CSCL = UINT8_MAX;
- uint8_t _I2CFreq = UINT8_MAX;

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 15, 2026

@coderabbitai, regarding

line 310 copies the entire state.data["preset"] object (including the list array) into a stack-local JsonDocument. If the preset list grows large, this is unnecessary memory pressure. Consider constructing a minimal object with only select and selected:

I like the idea but I think that will not work as the whole preset object will be copied, if part of this object is missing than the compare recursive will remove that missing part, please verify and confirm and if you have another proposal , let me know

@coderabbitai
Copy link

coderabbitai bot commented Feb 15, 2026

Tip

For best results, initiate chat on the files or code changes.

@ewowi Thanks for calling this out. Short answer: a minimal patch that sets only preset.select and preset.selected will NOT remove preset.list. compareRecursive walks and applies only the keys present in the incoming object; it does not prune missing siblings. Arrays are the notable exception: when you supply a new array, that replaces the old one. For your case (object with scalar leaves), a small patch is safe.

Two low-stack patterns you can use:

  1. Patch just the two leaves (keeps preset.list and any other fields intact)
JsonDocument patch;                 // small, heap-backed; no large locals
JsonObject p = patch["preset"].to<JsonObject>();
p["select"]   = newSelect;          // e.g., 255
p["selected"] = newSelected;        // e.g., 255 or an index

update(patch.as<JsonObject>(), ModuleState::update, _moduleName);
  • compareRecursive will only visit preset.select and preset.selected. It won’t touch preset.list, preset.list[*], or other siblings under preset.
  1. Zero-allocation, in-place lambda (bypasses deep diff)
update([&](ModuleState& s) {
  s.data["preset"]["select"]   = newSelect;
  s.data["preset"]["selected"] = newSelected;
  return StateUpdateResult::CHANGED;   // notify without walking compareRecursive
}, _moduleName);
  • Use this if you don’t need per-field onUpdate side effects driven by compareRecursive. It avoids creating any temporary JsonDocument and keeps httpd stack usage minimal.

Caveats and tips:

  • Don’t build an empty replacement object for preset and then assign it wholesale; that would replace the parent and could drop siblings. Stick to leaf updates as shown.
  • Arrays behave differently: to remove/reorder list items you must provide a new array (e.g., to() then fill), which intentionally replaces the old one. That’s expected and separate from simple object-leaf patches.

If you want, I can turn this into a tiny helper like setPresetSelection(select, selected) that uses option (1) internally, so all call sites stay safe and lightweight.


✏️ Learnings added
Learnt from: ewowi
URL: https://github.com/MoonModules/MoonLight/pull/124

Timestamp: 2026-02-15T13:23:26.042Z
Learning: In MoonModules/MoonLight Module::update() + compareRecursive: passing a partial object acts as a patch — only the provided keys are visited/updated; missing siblings are preserved. Arrays are replaced only when a new array is provided. Therefore updating only preset.select and preset.selected via a small patch will not remove preset.list.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.712Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-14T19:58:39.117Z
Learning: ESP32-P4/httpd: Heavy onUpdate cascades (preset apply, rename, controls rebuild) can exhaust 6 KB httpd stack; 8192 bytes is the safe default. Defer heavy work from onUpdate (httpd) to the SvelteKit loop and avoid large StaticJsonDocument locals in onUpdate to preserve stack headroom.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-03T21:11:12.471Z
Learning: MoonLight ModuleDevices: To ensure group convergence despite UDP broadcast loss, updateDevices() should also apply control fields to the local device (origin "group") when a message from a group member is processed, skipping self. Also, partOfGroup() must return base == device when the base has no dots (no group prefix) instead of matching all devices.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:31:17.304Z
Learning: In MoonModules/MoonLight compareRecursive (src/MoonBase/Module.cpp): Treat an object’s "name" change as an identity change. When oldData["name"] != newData["name"], emit only the "name" UpdatedItem for that parent and short‑circuit (do not recurse into the subtree like controls). This prevents spurious controls.* updates after renames while keeping the update flow generic and not tied to specific child keys.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:08:02.954Z
Learning: MoonLight NodeManager (src/MoonBase/NodeManager.h): After switching to direct onUpdate in httpd, compareRecursive continues with its captured newData and can emit controls updates immediately after a node rename. To avoid spurious control updates, NodeManager::onUpdate must (1) guard controls array/index, and (2) skip when the current control property already equals updatedItem.value. Only "value" and "default" should drive nodeClass->updateControl/onUpdate; other control fields are UI metadata and can be ignored or just trigger requestUIUpdate.

@ewowi
Copy link
Collaborator Author

ewowi commented Feb 15, 2026

@coderabbitai, regarding

Short answer: a minimal patch that sets only preset.select and preset.selected will NOT remove preset.list. compareRecursive walks and applies only the keys present in the incoming object; it does not prune missing siblings.

I am not convinced so will leave it as is.

I am about to merge the PR back to main. Can you do a thorough review if all the changes in this PR. Make sure the description summarizes all what is done in this PR and give an advice if it is okay to merge back to main.
Give a list of potential open issues and make also a list with possible follow up actions for later PRs.

@coderabbitai
Copy link

coderabbitai bot commented Feb 15, 2026

@ewowi Thanks for the context. Here’s a thorough review of PR #124 and guidance on merging.

High-level summary of what this PR delivers

  • New IMU driver and gravity plumbing
    • Adds IMUDriver (MPU6050/DMP-based) with proper start/stop lifecycle and 20 ms reads.
    • Exposes gravity via SharedData.gravity; integrates in ParticlesEffect with a new gravityType selector (None/Random/Gyro). VU and several effects got small robustness fixes (clamps, min counts).
  • Centralized I2C management in ModuleIO
    • UI controls for i2cFreq and i2cBus; automatic Wire.begin()/setClock() with per-board SDA/SCL defaults and live scanning.
    • ModuleDrivers registers IMUDriver so IMU is available through the standard addNodes path.
  • Safer, shared persistence
    • SharedFSPersistence now mirrors FSPersistence semantics: registerModule defers handler enablement; begin() first reads all modules from FS, then enables update handlers only for delayedWriting modules. Avoids “defaults overwriting stored state.”
    • Adds a global delayed-writes queue guarded by a lightweight portMUX spinlock; writeToFSDelayed() drains the queue atomically, then runs callbacks outside the lock.
    • Public enable/disableUpdateHandler per module for fine control.
  • Runtime/core and stability improvements
    • HTTPD stack increased to 8192 (from 6 KB) to ride through heavy onUpdate cascades.
    • Physical layer gains loop20msDrivers() and driverTask now ticks 20 ms work alongside the normal frame loop.
    • Non‑PSRAM devices: channel limits reduced (safer headroom).
    • D_FastLED destructor cleans up FastLED channel listeners before reset; noisy logs trimmed across modules.
  • UI and module tweaks
    • Preset UI: guard for select == 255, plus FieldRenderer pad tweak to set select=255 when toggling same selection.
    • ModuleEffects is wired with ModuleLightsControl so global preset state can influence effects.
    • Devices: introduce secondary sort key (IP) for stable ordering, plus logging/maintenance cleanups.
    • Misc safety/consistency: JsonDocument usage tweaks, minor arithmetic/float delay fix in ArtNet out, tiny logging/path fixes.

Readiness to merge to main

  • Technically OK to merge provided the two items below are acknowledged:
    1. Preset overwrite path: You mentioned a follow-up commit will stop presets from directly overwriting /.config/effects.json. Until that lands, a user can Save an effect and then indirectly revert it by clicking a preset (or via presetLoop). If you’re merging now, please prioritize that follow-up right after merge.
    2. Devices duplicates: Sorting is now stable (name, then IP), but duplicates (same name+IP) can still accumulate. If you accept fixing this in the next PR, merge is fine; otherwise drop in the small dedup step before/after sorting.

If those are acceptable as post-merge follow-ups, I recommend merge. The SharedFSPersistence fix and the I2C centralization are valuable improvements and unblock the new IMU/pir/mic workstreams.

Potential open issues to be aware of

  • Preset “click” bypass (follow-up incoming)
    • Root cause: copyFile(preset → effects.json) path can overwrite saved state outside SharedFSPersistence. You already identified this and plan a change. Until then, Save→Restart can appear inconsistent if a preset click happens in between.
  • Devices duplicates (same name+IP)
    • Even with primary/secondary sort, duplicates remain unless you dedup. Symptoms: repeated “checkReOrderSwap changed” churn and noisy writes. Impacted file: ModuleDevices.h.
  • IMU driver startup assumptions
    • Ensure IMUDriver checks ModuleIO’s I2C readiness/selected pins and handles address 0x68 vs 0x69 (AD0). Right now there was a TODO about reading success from ModuleIO; make it real to prevent spurious INVALID_STATE logs when bus is not ready.
  • ModuleIO naming nit
    • Private method name _updateI2SDevices() appears to handle I2C bus scanning. Rename to _updateI2CDevices() to avoid confusion with I2S.
  • I2C device naming in i2cBus
    • i2cBus currently lists addresses only. Without a name, users can’t tell IMU vs BME vs OLED. This is a UX gap (see follow-ups).
  • ESP32‑P4 platform regression (external)
    • Manual reboot after FS writes can crash in 55.03.37; 55.03.35 is OK per your tests. Pin the platform/toolchain for P4 envs until the upstream fix ships; add a note in firmware/esp32‑p4.ini or docs.
  • 20 ms driver loop budgeting
    • loop20msDrivers() runs inside driverTask. Keep work bounded to avoid LED timing jitter; long IMU/mic reads or bus scans should be amortized or moved off the critical path.
  • httpd stack + transient stack spikes
    • You already raised to 8192. Consider a couple of surgical trims (see follow-ups) to keep headroom on D0/C3 builds.
  • Minor: float math in ArtNetOut delay, logging strings, a few Char buffers can still be tightened.

Suggested follow-ups (next PRs)

  • Presets via persistence (recommended next)
    • Option A (preferred): Apply presets in-memory only (effects module), mark Save-needed, persist on explicit Save.
    • Option B: Apply presets in-memory then persist via SharedFSPersistence (disable handler while committing a single “effects” write to avoid feedback). Eliminate direct copyFile into effects.json.
  • Devices dedup (tiny change)
    • Build a name|ip map to keep only the most recent entry (by lastSync), then emit the unique set (still sorted by name, then IP). This stops reorder flip-flop permanently.
  • I2C quality-of-life
    • Expose I2CReady flag from ModuleIO and have D_IMU gate init/reads on it.
    • Add a lightweight registry so drivers can register (addr → name) with ModuleIO; show both in i2cBus. Start with a small curated address→name table as fallback.
    • Confirm and set per-SoC default SDA/SCL pins (ESP32=21/22; S2/S3/C3=8/9; C6=23/22; P4=7/8) and allow board overrides.
  • BMI160 support and IMU abstraction
    • Add hanyazou/BMI160-Arduino (or DFRobot_BMI160) and a tiny adapter in D_IMU that normalizes accel/gyro/gravity. This keeps chip-specific code trivial and future-proofs for LSM6/ICM later.
  • SPI/I2S centralization
    • Mirror I2C: centralize SPI begin/setPins in ModuleIO with a small device registry; same for I2S microphones (INMP441/ICS43432). Ensure these do not starve the LED RMT ISRs; keep sample rates/DMA buffers conservative.
  • Stack headroom trims (low risk, high ROI)
    • Move the global log formatting buffer off stack (static + small spinlock).
    • Avoid value.as() in hot logs; serialize into a tiny fixed char[48] buffer instead.
    • Audit Char sizes used as loop locals; shrink where safe (many can be 12–24 bytes).
    • Keep heavy onUpdate work deferred to the SvelteKit loop; avoid large StaticJsonDocument locals in httpd context.
  • Naming polish
    • Rename _updateI2SDevices() → _updateI2CDevices() in ModuleIO to avoid confusion.

Release notes draft (you can paste in PR description)

  • Features
    • Added IMU driver (MPU6050/DMP) and gravity plumbing (SharedData.gravity). Updated ParticlesEffect with gravityType selector (None/Random/Gyro); various effects got safer bounds and defaults.
    • Centralized I2C in ModuleIO: per-board SDA/SCL defaults, i2cFreq control, bus scanning with i2cBus in UI.
  • Reliability & persistence
    • Migrated to SharedFSPersistence; files are read before enabling update handlers, preventing defaults from overwriting persisted state on boot. Introduced a global, mutex-protected delayed write queue; added APIs to enable/disable handlers per module.
  • Performance & stability
    • Increased HTTPD stack to 8192; added 20 ms driver loop (loop20msDrivers) in PhysicalLayer and integrated in driverTask. Reduced non‑PSRAM channel limits for safer headroom. Trimmed noisy logs; D_FastLED destructor now unsubscribes channel listeners.
  • UX & modules
    • Devices list now sorts stably by Name, then IP. Preset UI guards on select == 255; ModuleEffects is wired to ModuleLightsControl for coordinated preset behavior. Minor path/logging polish and math fixes across modules.

Recommendation

  • Merge is OK now given your plan to immediately follow with the preset‑write fix. If you’d like main to be completely clean of the device list churn, include the small dedup in this PR or make it the very next one; otherwise the current state is stable enough to land and iterate.

If you want, I can turn the follow-ups above into individual GitHub issues with checklists and backlinks to this PR.


🧠 Learnings used
Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-05T13:20:46.001Z
Learning: In MoonLight with FastLED 3.11 on ESP-IDF 5.x+ (Arduino core 3.x): FastLED defaults to RMT5 driver (wraps Espressif's led_strip component), which has known ISR handler lifecycle issues causing repeated "Handler detached" messages every ~500ms and severe performance degradation (e.g., 2 FPS). RMT5 is more prone to Wi-Fi/interrupt jitter than legacy RMT4. Solution: force legacy RMT4 by adding `-D FASTLED_RMT5=0` to build flags in platformio.ini, or switch to FastLED's parallel I/O drivers which don't use RMT. RMT4 and RMT5 cannot coexist in the same binary.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:31:32.367Z
Learning: For ESP32 LED drivers using DMA, I2S, LCD, or PARLIO: always pin driver tasks to Core 1 (APP_CPU), not Core 0. WiFi/BT runs at priority ~23 on Core 0 and would constantly preempt lower-priority driver tasks, breaking LED timing requirements. ESP-IDF explicitly recommends display/DMA processing on Core 1 to avoid WiFi interference.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 121
File: src/MoonLight/Nodes/Drivers/D_FastLED.h:176-176
Timestamp: 2026-02-08T20:43:12.857Z
Learning: In MoonLight FastLED 3.11 driver (src/MoonLight/Nodes/Drivers/D_FastLED.h): Affinity is applied per-channel via fl::ChannelOptions.mAffinity passed to fl::ChannelConfig (not via FastLED.setExclusiveDriver). Keeping setExclusiveDriver commented out leaves all drivers available; FastLED then selects the appropriate driver for each channel based on the mAffinity preference ("RMT", "I2S", "SPI", "PARLIO", or "" for auto). This allows per-channel driver preferences rather than a global restriction.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T17:07:33.344Z
Learning: FastLED library does not support direct 4-byte-per-pixel (CRGBW) buffers. It always expects CRGB (3-byte) arrays and internally generates the W channel from RGB using .setRgbw() API. Casting a 4-byte RGBW buffer to CRGB* causes misalignment (W becomes next pixel's R). For MoonLight's FastLED driver with channelsPerLight=4: must either (1) repack RGB in-place before FastLED.show() then restore after, (2) use driver-specific channelsPerLight=3 layout, or (3) allocate temporary RGB-only buffer. Option 1 (in-place repacking) is best for ESP32-D0 memory constraints.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-06T13:45:03.067Z
Learning: For MoonLight on ESP32 at extreme loads (e.g., 49K channels @ 100 fps): Task watchdog can fire when AppDrivers/AppEffects process many nodes without yielding. Even with vTaskDelay(1) at loop end, the node iteration for-loops (PhysicalLayer::loopDrivers line 108-111, PhysicalLayer::loop line 73-75) can take >5 seconds without yielding, starving IDLE task (priority 0). Solution: add taskYIELD() or vTaskDelay(1) inside the node/layer iteration loops to give IDLE task chances to run and reset watchdog. Lowering task priority won't help since IDLE is already at 0 (lowest).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-18T12:49:07.297Z
Learning: In MoonLight channel reordering architecture: Keep rgbwBufferMapping in driver task (not effect task) because: (1) Driver has spare CPU cycles waiting for DMA/I2S/Art-Net timing; effect task is CPU-bound and starvation-prone. (2) Contiguous RGB writes in effects are cache-friendly; scattered writes at final offsets are cache-hostile for non-RGB orders (GRB, BGR, etc.). (3) forEachLightIndex already handles sparse mapping iteration; moving reordering to effects adds complexity without benefit since drivers must send full frames anyway (can't skip iteration). (4) rgbwBufferMapping does brightness/gamma LUT + white extraction + reordering—moving just reordering creates code duplication. (5) Effects should express color intent (CRGB), not hardware details (offsetRed/Green/Blue). Driver-side transformation maintains separation of concerns and task load balance for scaling to 98K+ pixels.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-13T19:05:38.712Z
Learning: In MoonModules/MoonLight SharedFSPersistence (src/MoonBase/SharedFSPersistence.h): To prevent stale default values from being written to filesystem during initialization, update handlers must be enabled AFTER files are read. Implementation: (1) registerModule() stores module info without enabling handler, (2) begin() first reads all files via readFromFS(), then enables handlers only for modules with delayedWriting=true. This matches FSPersistence behavior where files are loaded before handlers become active, preventing defaults from overwriting persisted state.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-26T14:59:22.091Z
Learning: In MoonModules/MoonLight PR `#114`: ModuleDevices deadlock caused by: (1) StatefulService.h line 204-205 bug - after 500ms mutex timeout, continues WITHOUT acquiring mutex, causing race conditions and JSON corruption. (2) receiveUDP() while-loop processes own UDP broadcasts via loopback, creating circular call chain: sendUDP→broadcast→receiveUDP→updateDevices→update→handlers→sendUDP. (3) updateDevices() holds _accessMutex during expensive sort/JSON operations with 15 devices, can exceed 500ms with nested calls. Solution: Fix StatefulService to wait forever or reboot on timeout; skip own broadcasts by checking message.name == ownHostname; optimize updateDevices to sort outside mutex.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-30T20:14:11.929Z
Learning: In MoonModules/MoonLight ModuleDevices group sync: Use boolean `isOriginalSender` flag instead of sequence number for deduplication. When originator broadcasts control to group (isOriginalSender=true), members apply locally but don't re-broadcast (pass non-numeric originId to prevent addUpdateHandler from triggering sendUDP). This prevents infinite broadcast loops with zero memory overhead (no state cache needed), versus sequence number approach which requires caching last state and processing 250× more packets (each member broadcasts duplicates).
</add_learning>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 84
File: src/MoonLight/Modules/ModuleLightsControl.h:367-379
Timestamp: 2025-12-28T10:28:24.157Z
Learning: In MoonLight PR `#84`: wrapping effectTask's layerP.loop() with monitorMutex while reading channelsE for monitor emission is thread-safe and prevents partial frames, but introduces serialization between effects rendering and monitor emission. Reading from channelsD (driver/front buffer) instead—with monitorMutex held only during the brief pointer swap—is superior because it maintains full parallelism (effects ║ monitor ║ driver), shows actual LED output, and minimizes mutex hold time.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-04T09:52:08.096Z
Learning: MoonLight ModuleDevices: To reduce missed packets when many devices send at once, add a 0–20 ms non-blocking jitter only to non-control (status) UDP broadcasts (post-control apply and optionally the 10s heartbeat). Implement by scheduling the send in loop20ms instead of delaying inside addUpdateHandler/httpd context; keep control packets immediate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-16T15:53:11.236Z
Learning: For ESP32-PICO-MINI-02 (and similar ESP32 boards with PSRAM), the critical configuration to enable PSRAM is `board_build.arduino.memory_type = qio_qspi` in platformio.ini. The board JSON may already include `-DBOARD_HAS_PSRAM` and `flash_mode: qio`, but without the memory_type setting, PSRAM won't be properly initialized, causing LoadStoreError crashes when code tries to access PSRAM addresses.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T18:06:23.868Z
Learning: ESP32-P4 observed lower headroom for httpd task stack in MoonLight: NodeManager rename (create new node + rebuild controls + remove old) can overflow httpd stack when onUpdate runs in httpd. Mitigation: defer only NodeManager rename heavy work to SvelteKit loop via a small pendingRenames queue; keep identity short-circuit in compareRecursive and equality-guarded controls updates. Increasing httpd stack_size for P4 builds is an optional, broader fix.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-14T19:58:39.117Z
Learning: ESP32-P4/httpd: Heavy onUpdate cascades (preset apply, rename, controls rebuild) can exhaust 6 KB httpd stack; 8192 bytes is the safe default. Defer heavy work from onUpdate (httpd) to the SvelteKit loop and avoid large StaticJsonDocument locals in onUpdate to preserve stack headroom.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-15T12:38:33.598Z
Learning: In MoonLight GameOfLifeEffect (src/MoonLight/Nodes/Effects/E_MoonModules.h): The effect's nested loops (especially lines 257-323) iterate through all cells (e.g., 128×128 = 16,384) with nested neighbor checks (18 per cell = 294,912 operations) without yielding. This starves effectTask (priority 3) when higher-priority tasks like HTTP server (priority 5) run, causing the effect to disappear. UI actions (save, refresh) temporarily show one frame because HTTP completes and briefly releases CPU back to effectTask. Solution: Add taskYIELD() inside outer X loop to prevent task starvation. This is the same root cause as issue `#102` (Starsky glitches with many driver nodes).

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-25T10:45:58.923Z
Learning: In MoonLight ModuleDevices: Variable Length Arrays (VLAs) like `char buffer[packetSize]` on ESP32 task stack are unsafe because tasks have limited stack space (~8KB). When packetSize exceeds buffer size or stack limit, causes stack overflow and crashes (including WebSocket failures). Solution: use `char buffer[sizeof(UDPMessage)]` or static/heap allocation, and always validate packetSize before reading: `if (packetSize > sizeof(UDPMessage)) { deviceUDP.flush(); continue; }`.
</learning]>

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-08T11:39:52.878Z
Learning: In MoonLight WebSocket monitor: When browser tabs are hidden, browsers throttle JavaScript (including WebSocket onmessage handlers) but keep TCP connections open. Monitor data (48KB frames every 20-300ms) continues arriving at TCP level but isn't processed. This fills ESP32's httpd_ws_send_frame_async() buffers, causing ESP_FAIL. When tab becomes visible, hundreds of queued frames overwhelm the JavaScript main thread, freezing the UI. Solution: use Page Visibility API (client_visibility event) to stop emitting monitor data to hidden clients, preventing buffer overflow and backlog buildup.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-12T15:53:21.519Z
Learning: In MoonLight's FreeRTOS environment: `taskYIELD()` does not help with Idle Watchdog issues because it only yields to tasks with equal or higher priority. Since Effect Task and Driver Task run at priority 3 while IDLE task runs at priority 0, `taskYIELD()` will never allow IDLE to run. Only `vTaskDelay(1)` works because it blocks the calling task and allows the scheduler to run lower-priority tasks like IDLE, which can then reset the watchdog timer.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2025-12-25T13:34:40.097Z
Learning: For the MoonLight LED system: Effects task can run on Core 0 (PRO_CPU) since it only performs computation (calculating colors, writing to channels array) without timing-critical hardware interaction. WiFi preemption is acceptable for pure computation tasks. Only the driver task (using DMA, I2S, LCD, PARLIO) requires Core 1 (APP_CPU) for uninterrupted timing-critical hardware operations.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-03T21:11:12.471Z
Learning: MoonLight ModuleDevices: To ensure group convergence despite UDP broadcast loss, updateDevices() should also apply control fields to the local device (origin "group") when a message from a group member is processed, skipping self. Also, partOfGroup() must return base == device when the base has no dots (no group prefix) instead of matching all devices.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:31:17.304Z
Learning: In MoonModules/MoonLight compareRecursive (src/MoonBase/Module.cpp): Treat an object’s "name" change as an identity change. When oldData["name"] != newData["name"], emit only the "name" UpdatedItem for that parent and short‑circuit (do not recurse into the subtree like controls). This prevents spurious controls.* updates after renames while keeping the update flow generic and not tied to specific child keys.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:08:02.954Z
Learning: MoonLight NodeManager (src/MoonBase/NodeManager.h): After switching to direct onUpdate in httpd, compareRecursive continues with its captured newData and can emit controls updates immediately after a node rename. To avoid spurious control updates, NodeManager::onUpdate must (1) guard controls array/index, and (2) skip when the current control property already equals updatedItem.value. Only "value" and "default" should drive nodeClass->updateControl/onUpdate; other control fields are UI metadata and can be ignored or just trigger requestUIUpdate.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-02-15T13:23:26.042Z
Learning: In MoonModules/MoonLight Module::update() + compareRecursive: passing a partial object acts as a patch — only the provided keys are visited/updated; missing siblings are preserved. Arrays are replaced only when a new array is provided. Therefore updating only preset.select and preset.selected via a small patch will not remove preset.list.

Learnt from: ewowi
Repo: MoonModules/MoonLight PR: 0
File: :0-0
Timestamp: 2026-01-28T12:18:44.193Z
Learning: MoonLight NodeManager controls updates: After rename, compareRecursive continues with captured newData and will emit controls.* changes. Ensure NodeManager::onUpdate:
- Applies only controls.value/default to the node (updateControl + onUpdate), guarded by equality against current control state to avoid re-applying after rename.
- Propagates all other controls metadata (name/min/max/type/valid/size/…) to state/UI and triggers requestUIUpdate, but does not call nodeClass->updateControl for these metadata fields.
This prevents spurious back-application while keeping the UI in sync with the new control schema.
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@ewowi ewowi merged commit 648609b into main Feb 15, 2026
33 checks passed
@ewowi ewowi deleted the tube-pir-gyro branch February 15, 2026 13:41
@ewowi ewowi changed the title Tube gyro pir I2C support and IMU driver Feb 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants