-
Notifications
You must be signed in to change notification settings - Fork 284
Devops: push-env-key.sh script to update env configs on remote #5962
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| REMOTE_HOST="jack" | ||
| REMOTE_FILE="env.json" | ||
| LOCAL_ENV="env.json" | ||
| REMOTE_BASE='\$HOME/jenkins-files' | ||
|
|
||
| usage() { | ||
| cat <<EOF | ||
| Pull the remote env.json from $REMOTE_HOST and update the local copy. | ||
|
|
||
| Usage: $(basename "$0") [-f FOLDER] | ||
|
|
||
| -f FOLDER Remote folder under ~/jenkins-files (default: master) | ||
| -h Show this help message | ||
| EOF | ||
| exit 1 | ||
| } | ||
|
|
||
| TARGET_FOLDER="" | ||
| while getopts "f:h" opt; do | ||
| case $opt in | ||
| f) TARGET_FOLDER="$OPTARG" ;; | ||
| h) usage ;; | ||
| *) usage ;; | ||
| esac | ||
| done | ||
| shift $((OPTIND - 1)) | ||
|
|
||
| TARGET_FOLDER="${TARGET_FOLDER:-master}" | ||
| TARGET_FOLDER="${TARGET_FOLDER#/}" | ||
| TARGET_FOLDER="${TARGET_FOLDER%/}" | ||
| [ -z "$TARGET_FOLDER" ] && TARGET_FOLDER="master" | ||
|
|
||
| REMOTE_REPO="$REMOTE_BASE/$TARGET_FOLDER" | ||
|
|
||
| REMOTE_TMP="$(mktemp)" | ||
| trap 'rm -f "$REMOTE_TMP"' EXIT | ||
|
|
||
| echo "Pulling latest env.json from $REMOTE_HOST ($TARGET_FOLDER)..." | ||
| ssh "$REMOTE_HOST" "cd \"$(eval echo $REMOTE_REPO)\" && git pull --ff-only" >&2 | ||
| scp -q "$REMOTE_HOST:$(ssh $REMOTE_HOST eval echo $REMOTE_REPO)/$REMOTE_FILE" "$REMOTE_TMP" | ||
|
|
||
| backup_local_env() { | ||
| local base="${LOCAL_ENV}.bak" | ||
| local candidate="$base" | ||
| if [ -e "$candidate" ]; then | ||
| local counter=1 | ||
| while [ -e "${base}-${counter}" ]; do | ||
| counter=$((counter + 1)) | ||
| done | ||
| candidate="${base}-${counter}" | ||
| fi | ||
|
|
||
| cp "$LOCAL_ENV" "$candidate" | ||
| echo "$candidate" | ||
| } | ||
|
|
||
| if [ ! -f "$LOCAL_ENV" ]; then | ||
| echo "No local $LOCAL_ENV found; creating from remote." | ||
| cp "$REMOTE_TMP" "$LOCAL_ENV" | ||
| exit 0 | ||
| fi | ||
|
|
||
| if diff -q "$LOCAL_ENV" "$REMOTE_TMP" >/dev/null 2>&1; then | ||
| echo "Local $LOCAL_ENV already matches remote; nothing to do." | ||
| exit 0 | ||
| fi | ||
|
|
||
| BACKUP_PATH=$(backup_local_env) | ||
| echo "Backed up local $LOCAL_ENV to $BACKUP_PATH" | ||
|
|
||
| cp "$REMOTE_TMP" "$LOCAL_ENV" | ||
| echo "Updated $LOCAL_ENV with contents from $REMOTE_HOST/$TARGET_FOLDER." |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,162 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| REMOTE_HOST="jack" | ||
| REMOTE_FILE="env.json" | ||
| LOCAL_ENV="env.json" | ||
|
|
||
| usage() { | ||
| cat <<EOF | ||
| Push an env.json key to the Jenkins env.json on $REMOTE_HOST. | ||
| Inserts the key alphabetically within the correct section. | ||
|
|
||
| Usage: $(basename "$0") [-m MESSAGE] [-f FOLDER] <KEY> [JSON_VALUE] | ||
|
|
||
| KEY Top-level key in env.json (e.g. XGRAM_INIT) | ||
| JSON_VALUE JSON value to set (any valid JSON). If omitted, extracted from local env.json. | ||
| -m MESSAGE Custom commit message (default: "Set <KEY> in env.json") | ||
| -f FOLDER Remote folder under ~/jenkins-files (default: master) | ||
|
|
||
| Examples: | ||
| $(basename "$0") XGRAM_INIT | ||
| $(basename "$0") -f testnet XGRAM_INIT '{"apiKey": "abc123"}' | ||
| $(basename "$0") -m "Enable xgram exchange plugin" -f master XGRAM_INIT | ||
| EOF | ||
| exit 1 | ||
| } | ||
|
|
||
| COMMIT_MSG="" | ||
| TARGET_FOLDER="" | ||
| while getopts "m:f:h" opt; do | ||
| case $opt in | ||
| m) COMMIT_MSG="$OPTARG" ;; | ||
| h) usage ;; | ||
| *) usage ;; | ||
| esac | ||
| done | ||
| shift $((OPTIND - 1)) | ||
|
|
||
| [ $# -lt 1 ] && usage | ||
| KEY="$1" | ||
| shift | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" | ||
| REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" | ||
|
|
||
| if [ $# -gt 0 ]; then | ||
| VALUE="$1" | ||
| echo "$VALUE" | jq . > /dev/null 2>&1 || { echo "Error: Invalid JSON value" >&2; exit 1; } | ||
| else | ||
| VALUE=$(jq --arg key "$KEY" '.[$key]' "$REPO_ROOT/$LOCAL_ENV") | ||
| if [ "$VALUE" = "null" ]; then | ||
| echo "Error: Key '$KEY' not found in local $LOCAL_ENV" >&2 | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| TARGET_FOLDER="${TARGET_FOLDER:-master}" | ||
| TARGET_FOLDER="${TARGET_FOLDER#/}" | ||
| TARGET_FOLDER="${TARGET_FOLDER%/}" | ||
| [ -z "$TARGET_FOLDER" ] && TARGET_FOLDER="master" | ||
| REMOTE_BASE='$HOME/jenkins-files' | ||
| REMOTE_REPO="$REMOTE_BASE/$TARGET_FOLDER" | ||
|
|
||
| [ -z "$COMMIT_MSG" ] && COMMIT_MSG="Set $KEY in $REMOTE_FILE" | ||
|
|
||
| echo "Key: $KEY" | ||
| echo "Value: $VALUE" | ||
| echo "Folder: $TARGET_FOLDER" | ||
| echo "Message: $COMMIT_MSG" | ||
| echo "" | ||
|
|
||
| REMOTE_TMP=$(mktemp) | ||
| VALUE_TMP=$(mktemp) | ||
| RESULT_TMP=$(mktemp) | ||
| trap 'rm -f "$REMOTE_TMP" "$VALUE_TMP" "$RESULT_TMP"' EXIT | ||
|
|
||
| echo "Pulling latest on $REMOTE_HOST..." | ||
| ssh "$REMOTE_HOST" "cd $REMOTE_REPO && git pull --ff-only" >&2 | ||
|
|
||
| echo "Fetching remote $REMOTE_FILE..." | ||
| scp -q "$REMOTE_HOST:$REMOTE_REPO/$REMOTE_FILE" "$REMOTE_TMP" | ||
|
|
||
| printf '%s' "$VALUE" > "$VALUE_TMP" | ||
|
|
||
| echo "Merging $KEY into correct section..." | ||
| python3 - "$KEY" "$VALUE_TMP" "$REMOTE_TMP" "$REPO_ROOT/$LOCAL_ENV" > "$RESULT_TMP" <<'PYEOF' | ||
| import json | ||
| import sys | ||
|
|
||
| SECTION_MARKER = "--------" | ||
|
|
||
| key = sys.argv[1] | ||
| with open(sys.argv[2]) as f: | ||
| value = json.loads(f.read()) | ||
| with open(sys.argv[3]) as f: | ||
| remote = json.loads(f.read()) | ||
| with open(sys.argv[4]) as f: | ||
| local = json.loads(f.read()) | ||
|
|
||
| # Find which section the key belongs to using local env.json key order | ||
| target_section = None | ||
| current_section = None | ||
| for k in local: | ||
| if SECTION_MARKER in k: | ||
| current_section = k | ||
| elif k == key: | ||
| target_section = current_section | ||
| break | ||
|
|
||
| if target_section is None: | ||
| print(f"Warning: '{key}' not in local env.json; appending to last section", | ||
| file=sys.stderr) | ||
|
|
||
| # Split remote keys into sections: [(header, [key, ...])] | ||
| sections = [] | ||
| cur_header = None | ||
| cur_keys = [] | ||
| for k in remote: | ||
| if SECTION_MARKER in k: | ||
| sections.append((cur_header, cur_keys)) | ||
| cur_header = k | ||
| cur_keys = [] | ||
| else: | ||
| cur_keys.append(k) | ||
| sections.append((cur_header, cur_keys)) | ||
|
|
||
| # Remove key from wherever it currently sits | ||
| for i, (h, keys) in enumerate(sections): | ||
| sections[i] = (h, [k for k in keys if k != key]) | ||
|
|
||
| # Insert key into the target section | ||
| added = False | ||
| for i, (h, keys) in enumerate(sections): | ||
| if h == target_section: | ||
| keys.append(key) | ||
|
Comment on lines
+133
to
+135
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
If a key is not found in local Useful? React with 👍 / 👎. |
||
| added = True | ||
| break | ||
| if not added: | ||
| sections[-1][1].append(key) | ||
|
|
||
| # Sort keys alphabetically within each section | ||
| for i, (h, keys) in enumerate(sections): | ||
| sections[i] = (h, sorted(keys)) | ||
|
|
||
| # Reconstruct ordered dict, pulling values from remote (or using new value) | ||
| result = {} | ||
| for h, keys in sections: | ||
| if h is not None: | ||
| result[h] = remote[h] | ||
| for k in keys: | ||
| result[k] = value if k == key else remote[k] | ||
|
|
||
| print(json.dumps(result, indent=2)) | ||
| PYEOF | ||
|
|
||
| echo "Pushing to $REMOTE_HOST..." | ||
| scp -q "$RESULT_TMP" "$REMOTE_HOST:$REMOTE_REPO/$REMOTE_FILE" | ||
|
|
||
| ESCAPED_MSG=$(printf '%q' "$COMMIT_MSG") | ||
| ssh "$REMOTE_HOST" "cd $REMOTE_REPO && git add $REMOTE_FILE && git diff --cached --stat && git commit -m $ESCAPED_MSG && git push" | ||
|
|
||
| echo "Done." | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-foption in argument parserThe script advertises
-f FOLDERand includesfingetopts, but thecasestatement never handlesf, so any invocation with-ffalls into*and exits viausage. This means users cannot target non-default remote folders (liketestnet) even though that is documented as a primary workflow.Useful? React with 👍 / 👎.