-
-
Notifications
You must be signed in to change notification settings - Fork 8
feat: Support configuring the external endpoint of Trino clusters for ackUri rewriting #100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Pushed the current version as |
|
Updated |
|
Customer confirmed this solves their issues :) |
trino-lb-core/src/trino_api.rs
Outdated
| trino_lb_addr: &Url, | ||
| external_trino_addr: Option<&Url>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
question: Why are these parameters called *_addr if they contain URLs? I think their names should be updated to reflect this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See 54b235d
trino-lb-core/src/trino_api.rs
Outdated
| ) -> Result<(), Error> { | ||
| // Point nextUri to trino-lb | ||
| if let Some(next_uri) = &self.next_uri { | ||
| let next_uri = Url::parse(next_uri).context(ParseNextUriFromTrinoSnafu)?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: Immediately parsing this as a URL when deserializing TrinoQueryApiResponse might be a better idea and also gets rid of the to_string call below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
trino-lb-core/src/trino_api.rs
Outdated
| #[instrument( | ||
| fields(next_uri = %next_uri, trino_lb_addr = %trino_lb_addr), | ||
| )] | ||
| fn change_next_uri_to_trino_lb(next_uri: &Url, trino_lb_addr: &Url) -> Url { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: We should not take &Url for trino_lb_addr here if the first thing we do is to clone it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The trino_lb_addr comes directly from the pared YAML config of trino-lb, so we always need to clone it.
This way it's lazy and we don't eagerly clone it somewhere up the chain and don't actually use it.
But I don't really care, so picked in f40cabd ;)
| /// Endpoint of the Trino cluster the query is running on. | ||
| pub trino_endpoint: Url, | ||
|
|
||
| /// (Optionally, if configured) public endpoint of the Trino cluster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggestion: Slight reword. The fact that this option is optional implies that it only takes effect when configured.
| /// (Optionally, if configured) public endpoint of the Trino cluster. | |
| /// Optional public endpoint of the Trino cluster. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer my variant as it makes it a bit more clear.
Every Trino always has a public endpoint. The question is, if trino-lb was configured to know about it or not
| Url::parse(&trino_external_endpoint) | ||
| .context(ParseClusterExternalEndpointFromStoredQuerySnafu)?, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: Such a bummer that have to do this every time. Can we maybe natively support Url to be used as part of query!?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the same problem as in #100 (comment)?
| /// special handling. | ||
| #[instrument( | ||
| name = "GET /v1/statement/executing/{queryId}/{slug}/{token}", | ||
| name = "GET (or HEAD) /v1/statement/executing/{queryId}/{slug}/{token}", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: Ideally we set this to the appropriate HTTP method in the function itself, but I'm unsure if this is supported (without using the special otel.name field/attribute).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
| name = "GET (or HEAD) /v1/statement/executing/{queryId}/{slug}/{token}", | ||
| skip(state, headers) | ||
| )] | ||
| pub async fn get_trino_executing_statement( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: This function needs to be renamed because it now now only handles GET requests, but HEAD as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was the very surprising implementation details of axum, which caused the "bug" in the first place.
Even if you register it via get(), is still get's HEADs.
We register the fn with
.route(
"/v1/statement/executing/{query_id}/{slug}/{token}",
get(v1::statement::get_trino_executing_statement),
)We do this many times with lot's of functions, so we would need to change all of them.
And I currently find the registration code readable
| uri: Uri, | ||
| ) -> Result<(HeaderMap, Json<TrinoQueryApiResponse>), Error> { | ||
| ) -> Result<Response, Error> { | ||
| if method == http::Method::HEAD { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: I would like to see a match here instead to explicitly handle GET and HEAD and error/warn on all other methods.
note: It might additionally also make sense to move GET and HEAD code into separate functions respectively.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moved to match in e9b150b
| Ok((trino_headers, Json(trino_query_api_response))) | ||
| } | ||
|
|
||
| async fn handle_head_request_to_trino( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: I think this function is poorly named. It implies it handles the complete request and response while it only returns a list of headers. This should be made clear.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean there is really nothing to return except the headers, as HEAD requests never return a body ^^
The handle_query_running_on_trino fn also only returns the headers and the Json from the body.
Co-authored-by: Techassi <git@techassi.dev>
Part of #97
Additionally, sometimes the trino-client sends a HEAD request, which we need to proxy as HEAD (ant not GET) as well: