Skip to content

[PULL REQUEST] Upgrades to debug mode#207

Open
Eric-Liu-SANDAG wants to merge 2 commits intomainfrom
debug-updgrade
Open

[PULL REQUEST] Upgrades to debug mode#207
Eric-Liu-SANDAG wants to merge 2 commits intomainfrom
debug-updgrade

Conversation

@Eric-Liu-SANDAG
Copy link
Contributor

Describe this pull request. What changes are being made?

Upgrades to debug mode to make it not completely useless

What issues does this pull request address?

Additional context

N/A

@Eric-Liu-SANDAG Eric-Liu-SANDAG self-assigned this Mar 17, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR upgrades the project’s debug mode to support running a single module for a single year by re-using a pre-existing complete run_id, while avoiding writes to the production database and saving outputs locally instead.

Changes:

  • Add a debug flag to runtime configuration/parsing and propagate it through module entrypoints from main.py.
  • When debug is enabled, skip database inserts/updates in modules and write inputs/outputs to a local debug_output/ folder.
  • Simplify debug configuration to {run_id, year, module} and validate that the referenced run is complete.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/utils.py Exposes DEBUG from parser, adds debug_output/ folder creation, expands runtime logging.
python/parsers.py Reworks debug config to single {run_id, year, module}, validates run completeness, adds module list constant.
main.py Threads debug=utils.DEBUG into all module orchestrators.
python/startup.py Adds debug parameter and skips DB insertion in debug mode.
python/staging.py Adds debug parameter and skips metadata update in debug mode.
python/hs_hh.py Adds debug parameter; writes CSVs instead of DB inserts in debug mode.
python/pop_type.py Adds debug parameter; writes CSVs instead of DB inserts in debug mode.
python/ase.py Adds debug parameter; writes CSVs instead of DB inserts in debug mode (including BULK INSERT bypass).
python/hh_characteristics.py Adds debug parameter; writes CSVs instead of DB inserts in debug mode.
python/employment.py Adds debug parameter; writes CSVs instead of DB inserts in debug mode.
config.toml Updates debug configuration shape to {run_id, year, module} with new comments.
README.md Minor TOML example quoting change for sql.staging.
.gitignore Adds debug_output/ and replaces with a more complete Python .gitignore template.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Upgrade Debug mode

2 participants