Skip to content

Test for grpc metadata headers#16046

Draft
v-pratap wants to merge 1 commit intogoogleapis:mainfrom
v-pratap:trace-header
Draft

Test for grpc metadata headers#16046
v-pratap wants to merge 1 commit intogoogleapis:mainfrom
v-pratap:trace-header

Conversation

@v-pratap
Copy link
Contributor

To run this script

export CLOUD_STORAGE_ENABLE_TRACING=rpc,http,raw-client,rpc-streams,gRPC
export GOOGLE_CLOUD_CPP_ENABLE_CLOG=yes
export GRPC_VERBOSITY=DEBUG
export GRPC_TRACE=rpc,http

bazel run --compilation_mode=dbg --copt=-g //google/cloud/storage/tests:async_client_integration_test-default > grpc-test.log 2>&1

@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the async_client_integration_test.cc file by removing a substantial portion of existing async client integration tests. It introduces new infrastructure, including a ThreadPool class and associated helper functions, to facilitate more advanced and potentially concurrent testing. A new test case for appendable object uploads is added, which, based on the PR title and description, is intended to lay the groundwork for testing gRPC metadata headers.

Highlights

  • Refactored Async Client Integration Tests: A large suite of existing async client integration tests, including ObjectCRUD, ComposeObject, StreamingRead, and various upload/rewrite tests, were removed from async_client_integration_test.cc to streamline the test file.
  • Introduced Thread Pool Utility: A new ThreadPool class, along with helper functions MakeOptions, MakeGrpcClient, MakeAsyncClient, and ReadRangeTask, was added, indicating preparation for more complex, potentially concurrent, testing scenarios related to gRPC.
  • New Appendable Upload Test: A new integration test, StartAppendableUploadEmpty, was implemented, focusing on appendable object uploads. This test appears to be a foundational step for validating gRPC metadata headers, aligning with the PR's title and description.
Changelog
  • google/cloud/storage/tests/async_client_integration_test.cc
    • Removed include for google/cloud/storage/async/options.h.
    • Added include for google/cloud/opentelemetry_options.h.
    • Added standard library includes for <queue> and <thread>.
    • Removed the SetUp method and bucket_name_ member from the AsyncClientIntegrationTest class.
    • Deleted numerous existing integration test cases (e.g., ObjectCRUD, ComposeObject, StreamingRead, various upload and rewrite tests).
    • Introduced new helper functions: MakeOptions, MakeGrpcClient, and MakeAsyncClient.
    • Implemented a ThreadPool class to manage concurrent task execution.
    • Defined the ReadRangeTask function for handling asynchronous object range reads.
    • Added a new integration test named StartAppendableUploadEmpty.
Activity
  • The PR description provides detailed instructions for running a script with specific environment variables (CLOUD_STORAGE_ENABLE_TRACING, GOOGLE_CLOUD_CPP_ENABLE_CLOG, GRPC_VERBOSITY, GRPC_TRACE) to enable gRPC tracing and logging, suggesting active debugging and testing of gRPC metadata headers.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request refactors async_client_integration_test.cc by removing several old test cases and introducing a new ThreadPool class and related helper functions (MakeOptions, MakeGrpcClient, MakeAsyncClient, ReadRangeTask) to support asynchronous operations. The AsyncClientIntegrationTest class's SetUp method and bucket_name_ member were removed. Review comments suggest improving the new test by replacing hardcoded bucket/object names and project IDs with dynamically generated ones, removing a large block of commented-out code, adhering to the Google C++ Style Guide by removing unnecessary this-> qualifiers in the ThreadPool class, passing offset and limit parameters by value in ReadRangeTask for better clarity and efficiency, and removing a debugging std::cout statement.

Comment on lines +238 to +239
auto bucket_name = std::string{"gcs-grpc-team-fastbyte-bajajnehaa-test-us-west4"};
auto object_name = "vaibhav-test-file-111";

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using hardcoded bucket and object names in integration tests can lead to flakiness and conflicts, especially when tests are run in parallel or by different users. Please use the test fixture's helper functions to generate random names, such as MakeRandomBucketName() and MakeRandomObjectName(). The project ID should also be retrieved from the environment rather than being hardcoded.

Comment on lines +272 to +308
// auto close = writer.Close();

// auto object_metadata = client.GetObjectMetadata(bucket_name, object_name);
// auto m = *object_metadata;
// auto generation = m.generation();

// auto w1 = async.ResumeAppendableObjectUpload(BucketName(bucket_name), object_name, generation)
// .get();

// ASSERT_STATUS_OK(w1);

// AsyncWriter writer1;
// AsyncToken token1;
// std::tie(writer1, token1) = *std::move(w1);

// for (int i = 0; i < kBlockCount; ++i) {
// // std::cout << "Writing data iteration #" << i << std::endl;
// auto p = writer1.Write(std::move(token1), WritePayload(block)).get();
// ASSERT_STATUS_OK(p);
// token1 = *std::move(p);
// }

// // auto object_metadata1 = client.GetObjectMetadata(bucket_name, object_name);
// // auto m1 = *object_metadata1;
// // // auto generation1 = m1.generation();
// // std::cout << "Object metadata1: " << m << std::endl;

// auto metadata = writer1.Finalize(std::move(token1)).get();
// ASSERT_STATUS_OK(metadata);
// // // ScheduleForDelete(*metadata);

// EXPECT_EQ(metadata->bucket(), BucketName(bucket_name).FullName());
// EXPECT_EQ(metadata->name()," object_name");
// EXPECT_EQ(metadata->size(), kBlockCount * kBlockSize);
// EXPECT_EQ("dddd", "Sdfs");
// std::cout << "Test completed successfully" << std::endl;
// client.DeleteObject(bucket_name, object_name);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This large block of commented-out code should be removed to improve the test's readability and maintainability. If this code is intended for future work, it should be tracked in a separate issue or branch rather than being left in the codebase.

Comment on lines +125 to +130
std::unique_lock<std::mutex> lock(this->queue_mutex_);

// Wait for a task to be available or for the pool to stop.
this->condition_.wait(lock, [this] {
return this->stop_ || !this->tasks_.empty();
});

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The Google C++ Style Guide, which this repository follows, recommends avoiding this-> unless it's necessary for disambiguation. In this context, it's not required and can be removed for cleaner code. This applies to other uses of this-> throughout the ThreadPool class.

                        std::unique_lock<std::mutex> lock(queue_mutex_);

                        // Wait for a task to be available or for the pool to stop.
                        condition_.wait(lock, [this] {
                            return stop_ || !tasks_.empty();
                        });
References
  1. The Google C++ Style Guide advises against using this-> unless it is required, for example, to disambiguate a member variable from a local variable. (link)

Comment on lines +203 to +205
void ReadRangeTask(std::shared_ptr<ObjectDescriptor> descriptor,
std::int64_t& offset,
std::int64_t& limit) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The parameters offset and limit are passed by non-const reference but are not modified within the function. For plain-old-data types like std::int64_t, it is clearer and generally more efficient to pass them by value.

void ReadRangeTask(std::shared_ptr<ObjectDescriptor>  descriptor,
                   std::int64_t offset,
                   std::int64_t limit) {

std::tie(writer, token) = *std::move(w);
for (int i = 0; i != kBlockCount; ++i) {
for (int i = 0; i < kBlockCount; ++i) {
std::cout << "Writing data iteration #" << i << std::endl;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This std::cout statement appears to be for debugging. It should be removed from the test code before merging. If logging is needed for diagnostics, consider using the library's logging facilities, which provide more control.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant