Sync eng/common directory with azure-sdk-tools for PR 14420#7003
Sync eng/common directory with azure-sdk-tools for PR 14420#7003
Conversation
azure-sdk
added
EngSys
There was a problem hiding this comment.
Pull request overview
Syncs eng/common content from azure-sdk-tools (PR 14420) by introducing a reusable Azure Pipelines job template intended to run azsdk-cli benchmark scenarios in CI.
Changes:
- Adds a new
benchmark-job.ymlpipeline job template undereng/common. - Uses sparse checkout to fetch only the benchmark project and
eng/commonbefore runningdotnet runwith scenario/tag selection and optional model override.
| - Name: $(Build.Repository.Name) | ||
| Commitish: $(Build.SourceVersion) | ||
| WorkingDirectory: $(System.DefaultWorkingDirectory)/$(Build.Repository.Name) | ||
| Paths: | ||
| - '${{ parameters.BenchmarkProject }}/**' |
There was a problem hiding this comment.
Using $(Build.Repository.Name) in WorkingDirectory and later in the PowerShell task path will produce a nested path like .../Azure/azure-sdk-for-cpp/... (see other pipelines splitting Build.Repository.Name on /). The sparse-checkout step creates the directory with New-Item and may fail when intermediate folders (e.g., .../Azure) don't already exist. Prefer checking out into a stable folder name (e.g., $(System.DefaultWorkingDirectory)/$(TargetRepoName) like ai-evals-tests.yml does) or just $(System.DefaultWorkingDirectory) and adjust the later workingDirectory accordingly.
|
|
||
| jobs: | ||
| - job: Run_Benchmark | ||
| variables: | ||
| - template: /eng/pipelines/templates/variables/globals.yml | ||
| - template: /eng/pipelines/templates/variables/image.yml | ||
| - group: 'AzSDK_Eval_Variable_group' |
There was a problem hiding this comment.
This template hardcodes a variable group (AzSDK_Eval_Variable_group). Since eng/common templates are intended to be reused across repositories, this will break consumers that don't have that variable group defined/authorized. Consider making the variable group name a parameter (defaulting to empty) and only including it when provided, or moving variable-group binding to the calling pipeline instead of the shared template.
| jobs: | |
| - job: Run_Benchmark | |
| variables: | |
| - template: /eng/pipelines/templates/variables/globals.yml | |
| - template: /eng/pipelines/templates/variables/image.yml | |
| - group: 'AzSDK_Eval_Variable_group' | |
| - name: VariableGroupName | |
| type: string | |
| default: 'AzSDK_Eval_Variable_group' | |
| jobs: | |
| - job: Run_Benchmark | |
| variables: | |
| - template: /eng/pipelines/templates/variables/globals.yml | |
| - template: /eng/pipelines/templates/variables/image.yml | |
| - ${{ if ne(parameters.VariableGroupName, '') }}: | |
| - group: ${{ parameters.VariableGroupName }} |
| env: | ||
| DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1 | ||
| DOTNET_CLI_TELEMETRY_OPTOUT: 1 | ||
| DOTNET_MULTILEVEL_LOOKUP: 0 | ||
| COPILOT_GITHUB_TOKEN: $(azuresdk-copilot-github-pat) |
There was a problem hiding this comment.
COPILOT_GITHUB_TOKEN is sourced from a repo-specific secret variable $(azuresdk-copilot-github-pat), which may not exist in consuming repos and also bakes a specific secret name into a shared template. Consider making the token value a parameter (or allowing it to be omitted) so callers can decide how to provide credentials (e.g., via a variable group, runtime parameters, or System.AccessToken when applicable).
| - name: ScenarioName | ||
| type: string | ||
| default: ' ' | ||
| - name: Tags | ||
| type: string | ||
| default: ' ' | ||
| - name: Model | ||
| type: string | ||
| default: ' ' |
There was a problem hiding this comment.
The string parameters default to a single space (' ') and are then trimmed in the script. This works, but it’s non-obvious and can lead to confusing behavior when values are inspected/logged. Prefer defaulting these to an empty string ('') and simplifying the checks accordingly.
| - '${{ parameters.BenchmarkProject }}/**' | ||
| - 'eng/common/**' | ||
|
|
||
| - template: /eng/pipelines/templates/steps/install-dotnet.yml |
There was a problem hiding this comment.
/eng/pipelines/templates/steps/install-dotnet.yml is referenced here, but that template file does not exist in this repository (there is no eng/pipelines/templates/steps/install-dotnet.yml). As-is, any pipeline that consumes this job template will fail at template expansion. Consider inlining the needed UseDotNet@2 steps here (similar to eng/common/pipelines/templates/jobs/ai-eval-job.yml) or adding the missing install-dotnet.yml to eng/pipelines/templates/steps.
| - template: /eng/pipelines/templates/steps/install-dotnet.yml | |
| - task: UseDotNet@2 | |
| displayName: 'Install .NET SDK' | |
| inputs: | |
| useGlobalJson: true |
Sync eng/common directory with azure-sdk-tools for PR Azure/azure-sdk-tools#14420 See eng/common workflow