Skip to content

LLM client code refactor#178

Merged
adambalogh merged 18 commits intomainfrom
aba/code
Mar 12, 2026
Merged

LLM client code refactor#178
adambalogh merged 18 commits intomainfrom
aba/code

Conversation

@adambalogh
Copy link
Collaborator

@adambalogh adambalogh commented Mar 12, 2026

Summary

Refactors the LLM client to use async/await and introduces the TEE Registry for dynamic endpoint discovery.

  • Async LLM client: All LLM request methods (chat, completion) are now async, with streaming using AsyncGenerator instead of threaded TextGenerationStream
  • TEE Registry integration: LLM endpoints and TLS certificates are now discovered on-chain via the TEERegistry contract instead of hardcoded server URLs. The pinned TLS cert from the registry replaces TOFU trust
  • TEE metadata in responses: TextGenerationOutput and StreamChunk now include tee_id, tee_endpoint, and tee_payment_address fields for full provenance
  • Removed og_llm_streaming_server_url: Unified behind a single optional og_llm_server_url override, with registry-based discovery as the default
  • Removed custom exceptions.py: Cleaned up unused exception classes
  • Simplified Makefile: Consolidated individual test targets into a single make test that runs all tests via pytest tests/
  • New llm_test.py: Comprehensive unit tests for the async LLM client
  • Updated examples and docs: All examples and generated docs reflect the async API and TEE registry changes

Test plan

  • make test — all unit tests pass
  • make check — formatting and type checks pass
  • make integrationtest — integration tests pass against devnet
  • Manual: make chat and make chat-stream work with live TEE endpoints

@adambalogh adambalogh changed the title LLM related code refactor LLM client code refactor Mar 12, 2026
@adambalogh adambalogh marked this pull request as ready for review March 12, 2026 01:27
@adambalogh adambalogh merged commit bf76c9f into main Mar 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant