feat: Update Sourcer interface for propagating totalPartitions

[vaibhavtiwari33]

Sourcer changes:

• Deprecate Remove partitions_handler in Sourcer
• Replace partitions_handler with active_partitions_handler in Sourcer
• Maintain backward compatibility
• Add support for optional total_partitions

To maintain backward compatibility (in terms of code compilation on SDK version upgrade), I had made all the following Sourcer methods optional:

• partitions_handler
• active_partitions_handler
• total_partitions_handler

But since the partitions_handler method was previously an @abstractmethod, I didn't like the fact that we were removing the constraint of implementing either of the .*partitions_handler methods.

Closes #334

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.68%. Comparing base (fd0f49a) to head (9f44725).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #343   +/-   ##
=======================================
  Coverage   92.67%   92.68%           
=======================================
  Files          67       67           
  Lines        3509     3513    +4     
  Branches      229      229           
=======================================
+ Hits         3252     3256    +4     
  Misses        193      193           
  Partials       64       64           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vigith]

is this expected?

[vaibhavtiwari33]

Seems like when I installed dependencies using the newly added uv manager, it downloaded the latest version of grpcio-tools:

uv.lock
_________
...
[[package]]
name = "grpcio-tools"
version = "1.78.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
    { name = "grpcio" },
    { name = "protobuf" },
    { name = "setuptools" },
]

I'm guessing since the dependencies allow versions >= lowest supported version:

pyproject.toml
_______________
...
dependencies = [
    "grpcio>=1.48.1",
    "grpcio-tools>=1.48.1",
...

Let me force the usage of v1.75.0 grpcio and generate again.

[vaibhavtiwari33]

Wait, but I see the uv.lock already updated with the 1.78.0 version for grpcio-tools. Maybe we never regenerated proto after moving package managers? The PR for migrating from poetry to uv shows version 1.78.0

[vaibhavtiwari33]

Ya, the previous poetry.lock in the PR shows v1.75.0

[vigith]

let's do in it 2 PRs if you want to update gRPC?

[vaibhavtiwari33]

Ok, let me use the previous grpc version

[vaibhavtiwari33]

Reverted the grpcio version to 1.75.0
PR is much cleaner now. Ty!

[kohlisid]

@vaibhavtiwari33 just a drop in note to update the examples with the updated signatures as well! Thanks
Will review the rest when moved from draft

[vigith]

do we really need that check? doesn't python default to None and None is the right value?

[vaibhavtiwari33]

you're right, the proto initializer for PartitionsResponse accepts _Optional[int]

def __init__(self, partitions: _Optional[_Iterable[int]] = ..., total_partitions: _Optional[int] = ...) -> None: ...

I can remove the check. Let me do that.

[vaibhavtiwari33]

Updated