Skip to content

enhancement(transforms): add rate_field and dynamic-field fallback support to sample#24977

Closed
jh7459-gh wants to merge 3 commits intovectordotdev:masterfrom
jh7459-gh:enhancement/sample-rate-field-v2
Closed

enhancement(transforms): add rate_field and dynamic-field fallback support to sample#24977
jh7459-gh wants to merge 3 commits intovectordotdev:masterfrom
jh7459-gh:enhancement/sample-rate-field-v2

Conversation

@jh7459-gh
Copy link

Summary

This PR improves the sample transform for per-event sampling use cases by adding dynamic field-driven sampling while preserving safe fallback behavior.

Changes included:

  • Add ratio_field support for per-event ratio sampling ((0, 1]).
  • Add rate_field support for per-event 1/N sampling (positive integer).
  • Requires static sampling configuration (rate or ratio), as a default value. i.e. dynamic-only mode is not allowed.
  • Disallow configuring ratio_field and rate_field together (mirrors the existing rate vs ratio exclusivity pattern).
  • Keeps sample_rate_key populated with the effective sampling value used for each sampled event.
  • Add validation and transform tests for fallback behavior and invalid configs.
  • Update generated transform reference metadata and changelog fragment.

Vector configuration

[transforms.sample_dynamic]
type = "sample"
inputs = ["in"]

# required static fallback (exactly one of these)
ratio = 0.01
# rate = 100

# optional dynamic field (choose one)
ratio_field = "sample_ratio"
# rate_field = "sample_rate_n"

# optional deterministic/keyed sampling controls
key_field = "trace_id"
group_by = "{{ service }}"

How did you test this PR?

Ran locally:

  • cargo fmt --all
  • cargo test --no-default-features --features transforms-sample sample:: --lib
  • cargo clippy --no-default-features --features transforms-sample --lib --tests -- -D warnings
  • ./scripts/check_changelog_fragments.sh

Test coverage added/updated includes:

  • reject dynamic ratio-only config
  • reject dynamic rate-only config
  • reject config that sets both dynamic fields
  • accept static + dynamic config
  • dynamic ratio overrides static
  • dynamic rate overrides static
  • dynamic ratio fallback to static ratio/rate when missing
  • dynamic rate fallback to static ratio when missing

Change Type

  • Bug fix
  • New feature
  • Dependencies
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

@jh7459-gh jh7459-gh requested review from a team as code owners March 21, 2026 21:49
@github-actions github-actions bot added domain: transforms Anything related to Vector's transform components domain: external docs Anything related to Vector's external, public documentation labels Mar 21, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 21, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@jh7459-gh
Copy link
Author

I have read the CLA Document and I hereby sign the CLA

@jh7459-gh
Copy link
Author

recheck

1 similar comment
@jh7459-gh
Copy link
Author

recheck

@jh7459-gh
Copy link
Author

CLA is signed (the bot comment says ‘All contributors have signed the CLA’), but the check still fails due a bot-side writeback error:\n\n\n\nCould a maintainer please rerun/resolve or override this required CLA check? This appears unrelated to PR code/content.

@jh7459-gh
Copy link
Author

Follow-up with exact error text:

CLA is signed (bot comment confirms all contributors have signed), but the CLAAssistant workflow run still fails with:
Could not update the JSON file: Could not update file: Changes must be made through a pull request.

Could a maintainer please rerun/resolve or override the required CLA check for this PR? This appears to be a CLA bot permissions/config issue, not a code issue in this PR.

@jh7459-gh
Copy link
Author

@vectordotdev/vector could someone please take a quick look at the CLA check on this PR?

The CLA is already signed (bot comment confirms "All contributors have signed the CLA"), but the required CLAAssistant check still fails with:

Could not update the JSON file: Could not update file: Changes must be made through a pull request.

Latest failing run:
https://github.com/vectordotdev/vector/actions/runs/23389690157/job/68042237701

This appears to be a CLA bot/repo-permissions issue rather than a PR/code issue.

@jh7459-gh
Copy link
Author

Closing in favor of #24976 (reopened), where CLA and required lightweight checks are currently passing on the existing head SHA.

@jh7459-gh jh7459-gh closed this Mar 21, 2026
@github-actions github-actions bot locked and limited conversation to collaborators Mar 21, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

domain: external docs Anything related to Vector's external, public documentation domain: transforms Anything related to Vector's transform components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant