Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ jobs:
- '3.13'
steps:
- uses: actions/checkout@v6
- uses: prefix-dev/setup-pixi@v0.9.3
- uses: prefix-dev/setup-pixi@v0.9.4
with:
pixi-version: v0.62.2
pixi-version: v0.64.0
cache: true
cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}
environments: test-cpu
Expand All @@ -47,3 +47,18 @@ jobs:
uses: codecov/codecov-action@v5
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run-ty:
name: Run ty
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: prefix-dev/setup-pixi@v0.9.4
with:
pixi-version: v0.64.0
cache: true
cache-write: ${{ github.event_name == 'push' && github.ref_name == 'main' }}
frozen: true
environments: py314
- name: Run ty
run: pixi run -e py314 ty
shell: bash -el {0}
6 changes: 3 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Claude Code
.claude/

# Distribution / packaging
*.egg
*.egg-info/
Expand All @@ -13,7 +16,6 @@ wheels/

# Documentation
docs/_build/
_build/

# IDE
.idea/
Expand Down Expand Up @@ -48,5 +50,3 @@ htmlcov/

# Version file (generated by hatch-vcs)
src/*/_version.py

.claude
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ repos:
hooks:
- id: check-hooks-apply
- id: check-useless-excludes
- repo: https://github.com/tox-dev/tox-toml-fmt
rev: v1.2.2
- repo: https://github.com/tox-dev/pyproject-fmt
rev: v2.12.1
hooks:
- id: tox-toml-fmt
- id: pyproject-fmt
- repo: https://github.com/lyz-code/yamlfix
rev: 1.19.1
hooks:
Expand All @@ -17,7 +17,7 @@ repos:
hooks:
- id: check-added-large-files
args:
- --maxkb=50000
- --maxkb=10000
- id: check-ast
- id: check-case-conflict
- id: check-docstring-first
Expand Down Expand Up @@ -47,7 +47,7 @@ repos:
hooks:
- id: yamllint
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.14
rev: v0.14.11
hooks:
- id: ruff-check
args:
Expand All @@ -62,7 +62,7 @@ repos:
- pyi
- python
- repo: https://github.com/kynan/nbstripout
rev: 0.9.0
rev: 0.8.2
hooks:
- id: nbstripout
args:
Expand Down
3 changes: 0 additions & 3 deletions .yamllint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,8 @@ rules:
quoted-strings: disable
trailing-spaces: enable
truthy:
check-keys: false
level: warning
yaml-files:
- '*.yaml'
- '*.yml'
- .yamllint
ignore:
- src/skillmodels/test_data/simplest_augmented_model.yaml
48 changes: 36 additions & 12 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ filter-based maximum likelihood estimation following Cunha, Heckman, Schennach (

```bash
# Run tests
pixi run tests
pixi run -e test-cpu tests

# Run tests with coverage
pixi run tests-with-cov
pixi run -e test-cpu tests-with-cov

# Run a single test file
pixi run -e test-cpu pytest tests/test_kalman_filters.py
Expand All @@ -30,8 +30,8 @@ pixi run ty
# Quality checks (linting, formatting)
prek run --all-files

# Build documentation (from docs/ directory)
make html
# Build documentation (mystmd, from docs/ directory)
myst build
```

## Command Rules
Expand All @@ -40,24 +40,24 @@ Always use these command mappings:

- **Python**: Use `pixi run python` instead of `python` or `python3`
- **Type checker**: Use `pixi run ty` instead of running ty/mypy/pyright directly
- **Tests**: Use `pixi run tests` instead of `pytest` directly
- **Tests**: Use `pixi run -e test-cpu tests` instead of `pytest` directly
- **Linting/formatting**: Use `prek run --all-files` instead of `ruff` directly
- **All quality checks**: Use `prek run --all-files`

Before finishing any task that modifies code, always run:

1. `pixi run ty` (type checker)
1. `pixi run tests` (tests)
1. `pixi run -e test-cpu tests` (tests)
1. `prek run --all-files` (quality checks)

## Architecture

### Core Pipeline Flow

```
Model Dict + Data
ModelSpec + Data
process_model() → Validates/extends model specification
process_model() → Validates/extends model specification → ProcessedModel
process_data() → Transforms data to estimation format
Expand All @@ -70,8 +70,12 @@ get_filtered_states() → Extract estimated latent factors

### Key Modules

- **process_model.py**: Model specification validation and preprocessing. Handles
dimensions, labels, stagemap, anchoring, and endogenous factors.
- **model_spec.py**: User-facing frozen dataclasses for model specification
(`ModelSpec`, `FactorSpec`, `AnchoringSpec`, `EstimationOptions`, `Normalizations`).
- **types.py**: Internal frozen dataclasses (`ProcessedModel`, `Labels`, `Dimensions`,
`Anchoring`, `ParsingInfo`, `ParsedParams`, etc.) and immutability utilities.
- **process_model.py**: Model specification validation and preprocessing. Converts
`ModelSpec` into `ProcessedModel`.
- **kalman_filters.py**: Core Kalman filter implementation (predict/update steps). Uses
square-root form for numerical stability.
- **likelihood_function.py**: Log-likelihood computation using Kalman filtering.
Expand All @@ -94,8 +98,10 @@ The codebase uses:

### Public API

The main package exports three functions:
The main package exports model specification classes and core functions:

- `ModelSpec`, `FactorSpec`, `AnchoringSpec`, `EstimationOptions`, `Normalizations`:
Frozen dataclasses for defining models
- `get_maximization_inputs()`: Prepare optimization problem for parameter estimation
- `get_filtered_states()`: Extract filtered latent factor estimates
- `simulate_dataset()`: Generate synthetic data from model specification (accepts
Expand All @@ -105,13 +111,31 @@ The main package exports three functions:

- Require Python 3.14
- Uses Ruff for linting (target: Python 3.14, line length: 88)
- Google-style docstrings
- Google-style docstrings with imperative mood ("Return" not "Returns")
- Use MyST syntax in docstrings (single backticks `like this`), not reStructuredText (no
double backticks, no `:ref:`, `:func:`, etc.)
- Dataclass attributes use inline docstrings (docstring on the line after the field):
```python
name: str
"""Description of name."""
```
- Pre-commit hooks enforce formatting and linting
- Type checking via `ty` with strict rules
- Do not use `from __future__ import annotations`
- Use modern numpy random API: `rng = np.random.default_rng(seed)` instead of
`np.random.seed()` or legacy functions like `np.random.randn()`

### Immutability Conventions

- All model configuration and internal data structures use frozen dataclasses
- Dict fields on internal dataclasses use `MappingProxyType` (not `Mapping`); wrap at
the call site with `MappingProxyType(...)`
- Dict fields on user-facing dataclasses (`AnchoringSpec`, `Normalizations`) use
`Mapping` with `__post_init__` conversion via `ensure_containers_are_immutable()`
- List fields use `tuple`, set fields use `frozenset`
- `ensure_containers_are_immutable()` recursively converts dict→MappingProxyType,
list→tuple, set→frozenset

## Testing

- pytest with markers: `wip`, `unit`, `integration`, `end_to_end`
Expand Down
4 changes: 2 additions & 2 deletions pixi.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading