OpenAPI: Promote the S3 signing endpoint to the main spec#15450
Open
adutra wants to merge 3 commits intoapache:mainfrom
Open
OpenAPI: Promote the S3 signing endpoint to the main spec#15450adutra wants to merge 3 commits intoapache:mainfrom
adutra wants to merge 3 commits intoapache:mainfrom
Conversation
Dev ML discussion: https://lists.apache.org/thread/2kqdqb46j7jww36wwg4txv6pl2hqq9w7 This commit promotes the S3 remote signing endpoint from an AWS-specific implementation to a first-class REST catalog API endpoint. This enables other storage providers (GCS, Azure, etc.) to eventually reuse the same signing endpoint pattern without duplicating the API definition. Summary of changes: - Added `/v1/{prefix}/namespaces/{namespace}/tables/{table}/sign/{provider}` endpoint to the main REST catalog OpenAPI spec. - Defined `RemoteSignRequest`, `RemoteSignResult` and `RemoteSignResponse` schemas. - Defined a new `provider` request body parameter in order to disambiguate requests from different storage providers. - Deprecated the separate `s3-signer-open-api.yaml` spec from the AWS module (for removal). - Updated the Python client.
adutra
added a commit
to adutra/iceberg
that referenced
this pull request
Feb 26, 2026
Dev ML discussion: https://lists.apache.org/thread/2kqdqb46j7jww36wwg4txv6pl2hqq9w7 This commit adapts the code base to the REST spec changes in apache#15450. Summary of changes: - Added new signer endpoint to `Endpoint` and `ResourcePaths` - Added new remote signing properties to `RESTCatalogProperties` - Introduced `RemoteSignRequest`, `RemoteSignRequestParser`, `RemoteSignResponse`, `RemoteSignResponseParser` - Deprecated `S3SignRequest`, `S3SignRequestParser`, `S3SignResponse`, `S3SignResponseParser` for removal - Deprecated `S3ObjectMapper` for removal - Added new serializers to `RESTSerializers` - Adapted `S3V4RestSignerClient`: - Deprecated public fields - Changed access methods and `check()` method to account for new properties and deprecated ones. - Included new `provider` request body parameter Test changes: - Refactored `S3SignerServlet` to extract a parent abstract class, `RemoteSignerServlet` (it can now be reused to test other providers) - Moved JSON parser tests from AWS module to Core module - Enhanced `TestS3V4RestSignerClient`
adutra
added a commit
to adutra/iceberg
that referenced
this pull request
Feb 26, 2026
Dev ML discussion: https://lists.apache.org/thread/2kqdqb46j7jww36wwg4txv6pl2hqq9w7 This commit adapts the code base to the REST spec changes in apache#15450. Summary of changes: - Added new signer endpoint to `Endpoint` and `ResourcePaths` - Added new remote signing properties to `RESTCatalogProperties` - Introduced `RemoteSignRequest`, `RemoteSignRequestParser`, `RemoteSignResponse`, `RemoteSignResponseParser` - Deprecated `S3SignRequest`, `S3SignRequestParser`, `S3SignResponse`, `S3SignResponseParser` for removal - Deprecated `S3ObjectMapper` for removal - Added new serializers to `RESTSerializers` - Adapted `S3V4RestSignerClient`: - Deprecated public fields - Changed access methods and `check()` method to account for new properties and deprecated ones. - Included new `provider` request body parameter Test changes: - Refactored `S3SignerServlet` to extract a parent abstract class, `RemoteSignerServlet` (it can now be reused to test other providers) - Moved JSON parser tests from AWS module to Core module - Enhanced `TestS3V4RestSignerClient`
adutra
added a commit
to adutra/iceberg
that referenced
this pull request
Feb 26, 2026
Dev ML discussion: https://lists.apache.org/thread/2kqdqb46j7jww36wwg4txv6pl2hqq9w7 This commit adapts the code base to the REST spec changes in apache#15450. Summary of changes: - Added new signer endpoint to `Endpoint` and `ResourcePaths` - Added new remote signing properties to `RESTCatalogProperties` - Introduced `RemoteSignRequest`, `RemoteSignRequestParser`, `RemoteSignResponse`, `RemoteSignResponseParser` - Deprecated `S3SignRequest`, `S3SignRequestParser`, `S3SignResponse`, `S3SignResponseParser` for removal - Deprecated `S3ObjectMapper` for removal - Added new serializers to `RESTSerializers` - Adapted `S3V4RestSignerClient`: - Deprecated public fields - Changed access methods and `check()` method to account for new properties and deprecated ones. - Included new `provider` request body parameter Test changes: - Refactored `S3SignerServlet` to extract a parent abstract class, `RemoteSignerServlet` (it can now be reused to test other providers) - Moved JSON parser tests from AWS module to Core module - Enhanced `TestS3V4RestSignerClient`
Contributor
Author
nastra
changed the title
nastra
reviewed
Feb 26, 2026
open-api/rest-catalog-open-api.yaml
Outdated
| - s3 | ||
| - gcs | ||
| - adls | ||
| default: s3 |
Contributor
There was a problem hiding this comment.
maybe we should omit the default here?
Contributor
There was a problem hiding this comment.
my understanding is its for backward compatibility ? we only supported s3 as the provider supported, we can spec it out too
Contributor
There was a problem hiding this comment.
my main concern here is that the java impl never defaults to that value, so I would rather just omit the default value here
Contributor
Author
There was a problem hiding this comment.
My intent is to update the Java impl to reflect this default value.
nastra
approved these changes
Feb 26, 2026
Contributor
nastra
left a comment
nastra
left a comment
There was a problem hiding this comment.
overall LGTM, just one comment around the default provider being used/defined
dimas-b
approved these changes
Feb 26, 2026
singhpk234
approved these changes
Feb 26, 2026
Contributor
singhpk234
left a comment
singhpk234
left a comment
There was a problem hiding this comment.
LGTM too, thanks @adutra
huaxingao
approved these changes
Feb 26, 2026
adutra
added a commit
to adutra/iceberg
that referenced
this pull request
Mar 2, 2026
Dev ML discussion: https://lists.apache.org/thread/2kqdqb46j7jww36wwg4txv6pl2hqq9w7 This commit adapts the code base to the REST spec changes in apache#15450. Summary of changes: - Added new signer endpoint to `Endpoint` and `ResourcePaths` - Added new remote signing properties to `RESTCatalogProperties` - Introduced `RemoteSignRequest`, `RemoteSignRequestParser`, `RemoteSignResponse`, `RemoteSignResponseParser` - Deprecated `S3SignRequest`, `S3SignRequestParser`, `S3SignResponse`, `S3SignResponseParser` for removal - Deprecated `S3ObjectMapper` for removal - Added new serializers to `RESTSerializers` - Adapted `S3V4RestSignerClient`: - Deprecated public fields - Changed access methods and `check()` method to account for new properties and deprecated ones. - Included new `provider` request body parameter Test changes: - Refactored `S3SignerServlet` to extract a parent abstract class, `RemoteSignerServlet` (it can now be reused to test other providers) - Moved JSON parser tests from AWS module to Core module - Enhanced `TestS3V4RestSignerClient`
dimas-b
approved these changes
Mar 2, 2026
danielcweeks
reviewed
Mar 4, 2026
open-api/rest-catalog-open-api.yaml
Outdated
| signed, such as the S3 DeleteObjects call. | ||
| provider: | ||
| type: string | ||
| enum: |
Contributor
There was a problem hiding this comment.
I'm concerned the enumeration here is too strict and should be left as just a string value. We're blocking out other provides that support HMAC-SHA256 or similar signing.
We shouldn't bake explicit cloud or vendor products into the spec.
Contributor
Author
|
@danielcweeks PTAL |
open-api/rest-catalog-open-api.yaml
Outdated
| signed, such as the S3 DeleteObjects call. | ||
| provider: | ||
| type: string | ||
| description: The storage provider for which the request is to be signed. For backwards compatibility, if |
Contributor
There was a problem hiding this comment.
Do we want to add some context in the description here as to what we should expect for these values? I don't want to have a hard coded enumeration of cloud providers (limits extensibility), but I'm ok with saying something like
The provider should correspond to the scheme used for a storage native URI. For example
s3for AWS S3 paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dev ML discussion: https://lists.apache.org/thread/2kqdqb46j7jww36wwg4txv6pl2hqq9w7
This commit promotes the S3 remote signing endpoint from an AWS-specific implementation to a first-class REST catalog API endpoint.
This enables other storage providers (GCS, Azure, etc.) to eventually reuse the same signing endpoint pattern without duplicating the API definition.
Summary of changes:
/v1/{prefix}/namespaces/{namespace}/tables/{table}/sign/{provider}endpoint to the main REST catalog OpenAPI spec.RemoteSignRequest,RemoteSignResultandRemoteSignResponseschemas.providerrequest body parameter in order to disambiguate requests from different storage providers.s3-signer-open-api.yamlspec from the AWS module (for removal).