diff --git a/docs/2-getting-started/environment-best-practices.md b/docs/2-getting-started/environment-best-practices.md new file mode 100644 index 0000000..3845d66 --- /dev/null +++ b/docs/2-getting-started/environment-best-practices.md @@ -0,0 +1,193 @@ +--- +title: Environment Best Practices +--- + +# Environment Best Practices + +Unreliable comparison environments produce misleading validation results. When source data drifts, branches fall behind, or environments collide, you cannot trust what Recce reports. + +This guide covers strategies to prepare reliable, efficient environments for Recce data validation. Recce compares a *base environment* (production or staging, representing your main branch) against a *current environment* (representing your pull request branch). + +## When to use this guide + +- Setting up CI/CD for Recce for the first time +- Seeing inconsistent diff results across PRs +- Managing warehouse costs from accumulated PR environments +- Troubleshooting validation results that don't match expectations + +## Challenges this guide addresses + +Several factors can affect comparison accuracy: + +- Source data updates continuously +- Transformations take time to run +- Other pull requests (PRs) merge into the base branch +- Generated environments accumulate in the warehouse + +## Use per-PR schemas + +Each PR should have its own isolated schema. This prevents interference between concurrent PRs and makes cleanup straightforward. + +```yaml +# profiles.yml +ci: + schema: "{{ env_var('CI_SCHEMA') }}" + +# CI workflow +env: + CI_SCHEMA: "pr_${{ github.event.pull_request.number }}" +``` + +Benefits: + +- Complete isolation between PRs +- Parallel validation without conflicts +- Easy cleanup by dropping the schema + +See [Environment Setup](environment-setup.md) for detailed configuration. + +## Prepare a single base environment + +Use one consistent base environment for all PRs to compare against. Options: + +| Base Environment | Characteristics | Best For | +|------------------|-----------------|----------| +| Production | Latest merged code, full data | Accurate production comparison | +| Staging | Latest merged code, limited data | Faster comparisons, lower cost | + +If using staging as base: + +- Ensure transformed results reflect the latest commit of the base branch +- Use the same source data as PR environments +- Use the same transformation logic as PR environments + +The staging environment should match PR environments as closely as possible, differing only in git commit. + +## Limit source data range + +Most data is temporal. Using only recent data reduces transformation time while still validating correctness. + +**Strategy:** Use data from the last month, excluding the current week. This ensures consistent results regardless of when transformations run. + +```sql +SELECT * +FROM {{ source('your_source_name', 'orders') }} +{% if target.name != 'prod' %} +WHERE + order_date >= DATEADD(month, -1, CURRENT_DATE) + AND order_date < DATE_TRUNC('week', CURRENT_DATE) +{% endif %} +``` + +![Diagram showing how limiting data to the previous month excluding current week creates consistent comparison windows](../assets/images/7-cicd/prep-env-limit-data-range.png){: .shadow} + +Benefits: + +- Faster transformation execution +- Consistent comparison results +- Reduced warehouse costs + +## Reduce source data volatility + +If source data updates frequently (hourly or more), comparison results can vary based on timing rather than code changes. + +**Strategies:** + +- **Zero-copy clone** (Snowflake, BigQuery, Databricks): Freeze source data at a specific point in time +- **Weekly snapshots**: Update source data weekly to reduce variability + +![Diagram showing zero-copy clone creating a frozen snapshot of source data for consistent CI comparisons](../assets/images/7-cicd/prep-env-clone-source.png){: .shadow} + +## Keep base environment current + +The base environment can become outdated in two scenarios: + +1. **New source data**: If you update data weekly, update the base environment at least weekly +2. **PRs merged to main**: Trigger base environment update on merge events + +Configure your CD workflow to run: + +- On merge to main (immediate update) +- On schedule (e.g., daily at 2 AM UTC) + +See [Setup CD](setup-cd.md) for workflow configuration. + +## Obtain artifacts for environments + +Recce uses base and current environment artifacts (`manifest.json`, `catalog.json`) to find corresponding tables in the data warehouse for comparison. + +**Recommended approaches:** + +- **Recce Cloud** - Automatic artifact management via `recce-cloud upload`. See [Setup CD](setup-cd.md) and [Setup CI](setup-ci.md). +- **dbt Cloud** - Download artifacts from dbt Cloud jobs. See dbt Cloud Setup (separate guide). + +**Alternative approaches** (for custom setups): + +- **Cloud storage** - Upload artifacts to S3, GCS, or Azure Blob in CI +- **GitHub Actions artifacts** - Use `gh run download` to retrieve from workflow runs +- **Stateless** - Checkout the base branch and run `dbt docs generate` on-demand + +## Keep PR branch in sync with base + +If a PR runs after other PRs merge to main, the comparison mixes: + +- Changes from the current PR +- Changes from other merged PRs + +This produces comparison results that don't accurately reflect the current PR's impact. + +![Diagram showing how an outdated PR branch mixes changes from other merged PRs into comparison results](../assets/images/7-cicd/prep-env-pr-outdated.png){: .shadow} + +**GitHub**: Enable [branch protection](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/keeping-your-pull-request-in-sync-with-the-base-branch) to show when PRs are outdated. + +**CI check**: Add a workflow step to verify the PR is up-to-date: + +```yaml +- name: Check if PR is up-to-date + if: github.event_name == 'pull_request' + run: | + git fetch origin main + UPSTREAM=${GITHUB_BASE_REF:-'main'} + HEAD=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}} + if [ "$(git rev-list --left-only --count ${HEAD}...origin/${UPSTREAM})" -eq 0 ]; then + echo "Branch is up-to-date" + else + echo "Branch is not up-to-date" + exit 1 + fi +``` + +## Clean up PR environments + +As PRs accumulate, so do generated schemas. Implement cleanup to manage warehouse storage. + +**On PR close**: Create a workflow that drops the PR schema when the PR closes. + +```jinja +{% macro clear_schema(schema_name) %} +{% set drop_schema_command = "DROP SCHEMA IF EXISTS " ~ schema_name ~ " CASCADE;" %} +{% do run_query(drop_schema_command) %} +{% endmacro %} +``` + +Run the cleanup: + +```shell +dbt run-operation clear_schema --args "{'schema_name': 'pr_123'}" +``` + +**Scheduled cleanup**: Remove schemas not used for a week. + +## Example configuration + +| Environment | Schema | When to Run | Count | Data Range | +|-------------|--------|-------------|-------|------------| +| Production | `public` | Daily | 1 | All | +| Staging | `staging` | Daily + on merge | 1 | 1 month, excluding current week | +| PR | `pr_` | On push | # of open PRs | 1 month, excluding current week | + +## Next steps + +- [Environment Setup](environment-setup.md) - Technical configuration for profiles.yml and CI/CD +- [Setup CD](setup-cd.md) - Configure automatic baseline updates +- [Setup CI](setup-ci.md) - Configure PR validation diff --git a/docs/2-getting-started/environment-setup.md b/docs/2-getting-started/environment-setup.md new file mode 100644 index 0000000..11655d2 --- /dev/null +++ b/docs/2-getting-started/environment-setup.md @@ -0,0 +1,195 @@ +--- +title: Environment Setup +--- + +# Environment Setup + +Configure your dbt profiles and CI/CD environment variables for Recce data validation. + +## Goal + +Set up isolated schemas for base vs current comparison. After completing this guide, your CI/CD workflows automatically create per-PR schemas and compare them against production. + +## Prerequisites + +- [x] **dbt project**: A working dbt project with `profiles.yml` configured +- [x] **CI/CD platform**: GitHub Actions, GitLab CI, or similar +- [x] **Warehouse access**: Credentials with permissions to create schemas dynamically + +## Why separate schemas matter + +Recce compares two sets of data to validate changes: + +- **Base**: The production state (main branch) +- **Current**: The PR branch with your changes + +For accurate validation, these must point to different schemas in your warehouse. Without separation, you would compare identical data and miss meaningful differences. + +## How CI/CD works with Recce + +Recce uses both continuous delivery (CD) and continuous integration (CI) to automate data validation: + +- **CD (Continuous Delivery)**: Runs after merge to main. Updates baseline artifacts with latest production state. +- **CI (Continuous Integration)**: Runs on PR. Validates proposed changes against baseline. + +**Set up CD first**, then CI. CD establishes your baseline (production artifacts), which CI uses for comparison. + +## Configure profiles.yml + +Your `profiles.yml` file defines how dbt connects to your warehouse. Add a `ci` target with a dynamic schema for PR isolation. + +```yaml +jaffle_shop: + target: dev + outputs: + dev: + type: snowflake + account: "{{ env_var('SNOWFLAKE_ACCOUNT') }}" + user: "{{ env_var('SNOWFLAKE_USER') }}" + password: "{{ env_var('SNOWFLAKE_PASSWORD') }}" + database: analytics + warehouse: COMPUTE_WH + schema: dev + threads: 4 + + # CI environment with dynamic schema per PR + ci: + type: snowflake + account: "{{ env_var('SNOWFLAKE_ACCOUNT') }}" + user: "{{ env_var('SNOWFLAKE_USER') }}" + password: "{{ env_var('SNOWFLAKE_PASSWORD') }}" + database: analytics + warehouse: COMPUTE_WH + schema: "{{ env_var('CI_SCHEMA') }}" + threads: 4 + + prod: + type: snowflake + account: "{{ env_var('SNOWFLAKE_ACCOUNT') }}" + user: "{{ env_var('SNOWFLAKE_USER') }}" + password: "{{ env_var('SNOWFLAKE_PASSWORD') }}" + database: analytics + warehouse: COMPUTE_WH + schema: public + threads: 4 +``` + +After saving, your profile supports three targets: `dev` for local development, `ci` for PR validation, and `prod` for production. + +Key points: + +- The `ci` target uses `env_var('CI_SCHEMA')` for dynamic schema assignment +- The `prod` target uses a fixed schema (`public`) for consistency +- Adapt this pattern for other warehouses (BigQuery uses `dataset` instead of `schema`) + +## Set CI/CD environment variables + +Your CI/CD workflow sets the schema dynamically for each PR. The key configuration: + +**GitHub Actions:** + +```yaml +env: + CI_SCHEMA: "pr_${{ github.event.pull_request.number }}" +``` + +**GitLab CI:** + +```yaml +variables: + CI_SCHEMA: "mr_${CI_MERGE_REQUEST_IID}" +``` + +This creates schemas like `pr_123`, `pr_456` for each PR automatically. When a PR opens, the workflow sets `CI_SCHEMA` and dbt writes to that isolated schema. + +For complete workflow examples, see [Setup CD](setup-cd.md) and [Setup CI](setup-ci.md). + +## Recommended pattern: Schema-per-PR + +Create an isolated schema for each PR. This is the recommended approach for teams. + +| Base Schema | Current Schema | Example | +|-------------|----------------|---------| +| `public` (prod) | `pr_123` | PR #123 gets its own schema | + +**Why this pattern:** + +- Complete isolation between PRs +- Multiple PRs can run validation in parallel without conflicts +- Easy cleanup by dropping the schema when PR closes +- Clear audit trail of what data each PR produced + +## Alternative patterns + +### Using staging as base + +Instead of comparing against production, compare against a staging environment with limited data. + +| Base Schema | Current Schema | Use Case | +|-------------|----------------|----------| +| `staging` | `pr_123` | Teams wanting faster comparisons | + +**Pros:** + +- Faster diffs with limited data ranges +- Consistent source data between base and current +- Reduced warehouse costs + +**Cons:** + +- Staging may drift from production +- Issues caught in staging might not reflect production behavior +- Requires maintaining an additional environment + +See [Environment Best Practices](environment-best-practices.md) for strategies on limiting data ranges. + +### Shared development schema (not recommended) + +Using a single `dev` schema for all development work. + +| Base Schema | Current Schema | Use Case | +|-------------|----------------|----------| +| `public` (prod) | `dev` | Solo developers only | + +**Why this is not recommended:** + +- Multiple PRs overwrite each other's data +- Cannot run parallel validations +- Comparison results may include changes from other work +- Difficult to isolate issues to specific PRs + +Only use this pattern for individual local development, not for CI/CD automation. + +## Verification + +After configuring your setup, verify that both base and current schemas are accessible. + +### Check configuration locally + +```shell +dbt debug --target ci +``` + +### Verify in Recce interface + +Launch Recce and check **Environment Info** in the top-right corner. You should see: + +- **Base**: Your production schema (e.g., `public`) +- **Current**: Your PR-specific schema (e.g., `pr_123`) + +## Troubleshooting + +| Issue | Solution | +|-------|----------| +| Schema creation fails | Verify your CI credentials have `CREATE SCHEMA` permissions | +| Environment variable not found | Check that secrets are configured in your CI/CD platform settings | +| Base and current show same schema | Ensure `--target ci` is used in CI, not `--target dev` | +| Profile not found | Verify `profiles.yml` is accessible in CI (check path or use `DBT_PROFILES_DIR`) | +| Connection timeout | Check warehouse IP allowlists include CI runner IP ranges | + +## Next steps + +- [Get Started with Recce Cloud](start-free-with-cloud.md) - Complete onboarding guide +- [Environment Best Practices](environment-best-practices.md) - Strategies for source data and schema management +- [Setup CD](setup-cd.md) - CD workflow for GitHub Actions and GitLab CI +- [Setup CI](setup-ci.md) - CI workflow for GitHub Actions and GitLab CI diff --git a/docs/7-cicd/setup-cd.md b/docs/2-getting-started/setup-cd.md similarity index 87% rename from docs/7-cicd/setup-cd.md rename to docs/2-getting-started/setup-cd.md index 075d0d5..2a6cd11 100644 --- a/docs/7-cicd/setup-cd.md +++ b/docs/2-getting-started/setup-cd.md @@ -4,7 +4,9 @@ title: Setup CD # Setup CD - Auto-Update Baseline -Set up automatic updates for your Recce Cloud base sessions. Keep your data comparison baseline current every time you merge to main, with no manual work required. +Manually updating your Recce Cloud baseline after every merge is tedious and error-prone. This guide shows you how to automate baseline updates so your data comparison stays current without manual intervention. + +After completing this guide, your continuous deployment (CD) workflow automatically uploads dbt artifacts to Recce Cloud whenever code merges to main. ## What This Does @@ -12,15 +14,22 @@ Set up automatic updates for your Recce Cloud base sessions. Keep your data comp - **Triggers**: Merge to main + scheduled updates + manual runs - **Action**: Auto-update base Recce session with latest production artifacts -- **Benefit**: Current comparison baseline for all future PRs/MRs +- **Benefit**: Current comparison baseline for all future PRs ## Prerequisites Before setting up CD, ensure you have: - [x] **Recce Cloud account** - [Start free trial](https://cloud.reccehq.com/) -- [x] **Repository connected** to Recce Cloud - [Connect Git Provider](../2-getting-started/start-free-with-cloud.md#2-connect-git-provider) +- [x] **Repository connected** to Recce Cloud - [Connect Git Provider](start-free-with-cloud.md#2-connect-git-provider) - [x] **dbt artifacts** - Know how to generate `manifest.json` and `catalog.json` from your dbt project +- [x] **Environment configured** - [Environment Setup](environment-setup.md) with `prod` target for base artifacts + +## Environment strategy + +This workflow uses the **main branch** with the `prod` target as the base environment. The base artifacts represent your production state, which PRs compare against. + +See [Environment Setup](environment-setup.md) for profiles.yml configuration. ## Setup @@ -170,11 +179,11 @@ Look for these indicators: **GitHub:** -![Recce Cloud showing updated base sessions](../assets/images/7-cicd/verify-setup-github-cd.png){: .shadow} +![Recce Cloud Sessions page displaying a successfully uploaded production baseline from GitHub Actions](../assets/images/7-cicd/verify-setup-github-cd.png){: .shadow} **GitLab:** -![Recce Cloud showing updated base sessions](../assets/images/7-cicd/verify-setup-gitlab-cd.png){: .shadow} +![Recce Cloud Sessions page displaying a successfully uploaded production baseline from GitLab CI](../assets/images/7-cicd/verify-setup-gitlab-cd.png){: .shadow} ### Expected Output @@ -293,7 +302,7 @@ prod-build: 1. Verify your repository is connected in [Recce Cloud settings](https://cloud.reccehq.com/settings) 2. **For GitHub**: Ensure `GITHUB_TOKEN` is passed explicitly to the upload step and the job has `contents: read` permission 3. **For GitLab**: Verify project has GitLab integration configured - - Check that you've created a [Personal Access Token](../2-getting-started/gitlab-pat-guide.md) + - Check that you've created a [Personal Access Token](gitlab-pat-guide.md) - Ensure the token has appropriate scope (`api` or `read_api`) - Verify the project is connected in Recce Cloud settings @@ -344,4 +353,4 @@ prod-build: ## Next Steps -**[Setup CI](./setup-ci.md)** to automatically validate PR/MR changes against your updated base session. This completes your CI/CD pipeline by adding automated data validation for every pull request or merge request. +**[Setup CI](setup-ci.md)** to automatically validate PR/MR changes against your updated base session. This completes your CI/CD pipeline by adding automated data validation for every pull request or merge request. diff --git a/docs/7-cicd/setup-ci.md b/docs/2-getting-started/setup-ci.md similarity index 79% rename from docs/7-cicd/setup-ci.md rename to docs/2-getting-started/setup-ci.md index 9310ee0..2115bb8 100644 --- a/docs/7-cicd/setup-ci.md +++ b/docs/2-getting-started/setup-ci.md @@ -2,26 +2,35 @@ title: Setup CI --- -# Setup CI - Auto-Validate PRs/MRs +# Setup CI - Auto-Validate PRs -Automatically validate your data changes in every pull request or merge request using Recce Cloud. Catch data issues before they reach production, with validation results right in your PR/MR. +Manual data validation before merging is error-prone and slows down PR reviews. This guide shows you how to set up continuous integration (CI) that automatically validates data changes in every pull request (PR). + +After completing this guide, your CI workflow validates every PR against your production baseline, with results appearing in Recce Cloud. ## What This Does -**Automated PR/MR Validation** prevents data regressions before merge: +**Automated PR Validation** prevents data regressions before merge: -- **Triggers**: PR/MR opened or updated against main +- **Triggers**: PR opened or updated against main - **Action**: Auto-update Recce session for validation -- **Benefit**: Automated data validation and comparison visible in your PR/MR +- **Benefit**: Automated data validation and comparison visible in your PR ## Prerequisites Before setting up CI, ensure you have: - [x] **Recce Cloud account** - [Start free trial](https://cloud.reccehq.com/) -- [x] **Repository connected** to Recce Cloud - [Connect Git Provider](../2-getting-started/start-free-with-cloud.md#2-connect-git-provider) +- [x] **Repository connected** to Recce Cloud - [Connect Git Provider](start-free-with-cloud.md#2-connect-git-provider) - [x] **dbt artifacts** - Know how to generate `manifest.json` and `catalog.json` from your dbt project -- [x] **CD configured** - [Setup CD](./setup-cd.md) to establish baseline for comparisons +- [x] **CD configured** - [Setup CD](setup-cd.md) to establish baseline for comparisons +- [x] **Environment configured** - [Environment Setup](environment-setup.md) with `ci` target for per-PR schemas + +## Environment strategy + +This workflow uses **per-PR schemas** with the `ci` target as the current environment. Each PR gets an isolated schema (e.g., `pr_123`) that compares against the base artifacts from CD. + +See [Environment Setup](environment-setup.md) for profiles.yml configuration and why per-PR schemas are recommended. ## Setup @@ -150,7 +159,7 @@ recce-upload: ## Verification -### Test with a PR/MR +### Test with a PR **GitHub:** @@ -169,16 +178,16 @@ recce-upload: Look for these indicators: - [x] **Workflow/Pipeline completes** without errors -- [x] **PR/MR session created** in [Recce Cloud](https://cloud.reccehq.com) +- [x] **PR session created** in [Recce Cloud](https://cloud.reccehq.com) - [x] **Session URL** appears in workflow/pipeline output **GitHub:** -![Recce Cloud showing PR validation session](../assets/images/7-cicd/verify-setup-github-ci.png){: .shadow} +![Recce Cloud Sessions page displaying a PR validation session created from GitHub Actions](../assets/images/7-cicd/verify-setup-github-ci.png){: .shadow} **GitLab:** -![Recce Cloud showing MR validation session](../assets/images/7-cicd/verify-setup-gitlab-ci.png){: .shadow} +![Recce Cloud Sessions page displaying a MR validation session created from GitLab CI](../assets/images/7-cicd/verify-setup-gitlab-ci.png){: .shadow} ### Expected Output @@ -232,12 +241,12 @@ Artifacts from: "/builds/your-org/your-project/target" Change request: https://gitlab.com/your-org/your-project/-/merge_requests/4 ``` -### Review PR/MR Session +### Review PR Session To analyze the changes in detail: 1. Go to your [Recce Cloud](https://cloud.reccehq.com) -2. Find the PR/MR session that was created +2. Find the PR session that was created 3. Launch Recce instance to explore data differences ## Advanced Options @@ -269,18 +278,17 @@ If CI is not working, the issue is likely in your CD setup. Most problems are sh - Upload errors - Sessions not appearing -**→ See the [Setup CD Troubleshooting section](./setup-cd.md#troubleshooting)** for detailed solutions. +**→ See the [Setup CD Troubleshooting section](setup-cd.md#troubleshooting)** for detailed solutions. **CI-specific tip:** If CD works but CI doesn't, verify: -1. PR/MR trigger conditions in your workflow configuration -2. The PR/MR is targeting the correct base branch (usually `main`) -3. You're looking at PR/MR sessions in Recce Cloud (not production sessions) +1. PR trigger conditions in your workflow configuration +2. The PR is targeting the correct base branch (usually `main`) +3. You're looking at PR sessions in Recce Cloud (not production sessions) ## Next Steps -After setting up CI, explore these workflow guides: +After setting up CI, explore these guides: -- [PR/MR review workflow](./scenario-pr-review.md) - Collaborate with teammates using Recce -- [Preset checks](./preset-checks.md) - Configure automatic validation checks -- [Best practices](./best-practices-prep-env.md) - Environment preparation tips +- [Environment Best Practices](environment-best-practices.md) - Strategies for source data and schema management +- [Get Started with Recce Cloud](start-free-with-cloud.md) - Complete onboarding guide diff --git a/docs/2-getting-started/start-free-with-cloud.md b/docs/2-getting-started/start-free-with-cloud.md index 996c354..b493414 100644 --- a/docs/2-getting-started/start-free-with-cloud.md +++ b/docs/2-getting-started/start-free-with-cloud.md @@ -92,14 +92,19 @@ This step adds CI/CD workflow files to your repository. The web agent detects yo #### Choose your setup 1. How do you run dbt? - - **You own your dbt run** - - **GitHub Actions**: Continue with this guide - - **GitLab CI, CircleCI**: See [Setup CD](setup-cd.md) and [Setup CI](setup-ci.md) - - **You run dbt on a platform** (dbt Cloud, Paradime, etc.): See [dbt Cloud Setup](dbt-cloud-setup.md) + + - **You own your dbt run** (GitHub Actions, GitLab CI, CircleCI): Continue with this guide + - **You run dbt on a platform** (dbt Cloud, Paradime, etc.): See [dbt Cloud Setup](dbt-cloud-setup.md) 2. How complex is your environment? - - **Simple** (prod and dev targets): Continue with this guide. We use per-PR schemas for fast setup. To learn why, see [Environment Setup](environment-setup.md). - - **Advanced** (multiple schemas, staging environments): See [Environment Setup](environment-setup.md) + + - **Simple** (prod and dev targets): Continue with this guide. We use per-PR schemas for fast setup. To learn why, see [Environment Setup](environment-setup.md). + - **Advanced** (multiple schemas, staging environments): See [Environment Setup](environment-setup.md) + +3. What's your CI/CD platform? + + - **GitHub Actions**: Continue with this guide + - **Other platforms** (GitLab CI, CircleCI, etc.): See [Setup CD](setup-cd.md) and [Setup CI](setup-ci.md) #### Set Up Profile.yml @@ -340,7 +345,7 @@ You can now: ## Related Resources -- [CI/CD Getting Started](../7-cicd/ci-cd-getting-started.md) -- [Setup CD](../7-cicd/setup-cd.md) -- [Setup CI](../7-cicd/setup-ci.md) -- [Best Practices for Preparing Environments](../7-cicd/best-practices-prep-env.md) +- [Environment Setup](environment-setup.md) +- [Setup CD](setup-cd.md) +- [Setup CI](setup-ci.md) +- [Environment Best Practices](environment-best-practices.md) diff --git a/docs/7-cicd/ci-cd-getting-started.md b/docs/7-cicd/ci-cd-getting-started.md index eda3981..1bfda5c 100644 --- a/docs/7-cicd/ci-cd-getting-started.md +++ b/docs/7-cicd/ci-cd-getting-started.md @@ -65,14 +65,14 @@ Before setting up, ensure you have: Both GitHub and GitLab follow the same simple pattern: ### 1. Setup CD - Auto-update baseline -[**Setup CD Guide**](./setup-cd.md) - Configure automatic baseline updates when you merge to main +[**Setup CD Guide**](../2-getting-started/setup-cd.md) - Configure automatic baseline updates when you merge to main - Updates your production baseline artifacts automatically - Runs on merge to main + optional scheduled updates - Works with both GitHub Actions and GitLab CI/CD ### 2. Setup CI - Auto-validate PRs/MRs -[**Setup CI Guide**](./setup-ci.md) - Enable automatic validation for every PR/MR +[**Setup CI Guide**](../2-getting-started/setup-ci.md) - Enable automatic validation for every PR/MR - Validates data changes in every pull request or merge request - Catches issues before they reach production @@ -84,9 +84,9 @@ Start with **CD first** to establish your baseline (production artifacts), then ## Next Steps -1. **[Setup CD](./setup-cd.md)** - Establish automatic baseline updates -2. **[Setup CI](./setup-ci.md)** - Enable PR/MR validation -3. Review [best practices](./best-practices-prep-env.md) for environment preparation +1. **[Setup CD](../2-getting-started/setup-cd.md)** - Establish automatic baseline updates +2. **[Setup CI](../2-getting-started/setup-ci.md)** - Enable PR/MR validation +3. Review [Environment Best Practices](../2-getting-started/environment-best-practices.md) for environment preparation ## Related workflows diff --git a/mkdocs.yml b/mkdocs.yml index 10c1c93..d92d7b8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -55,6 +55,10 @@ nav: - 2-getting-started/oss-vs-cloud.md - 2-getting-started/start-free-with-cloud.md - dbt Cloud Setup: 2-getting-started/dbt-cloud-setup.md + - Environment Setup: 2-getting-started/environment-setup.md + - Environment Best Practices: 2-getting-started/environment-best-practices.md + - Setup CD: 2-getting-started/setup-cd.md + - Setup CI: 2-getting-started/setup-ci.md #- 2-getting-started/cloud-5min-tutorial.md - 2-getting-started/installation.md - Claude Plugin: 2-getting-started/claude-plugin.md @@ -83,15 +87,11 @@ nav: - 6-collaboration/checklist.md - 6-collaboration/share.md - CI/CD: - - 7-cicd/ci-cd-getting-started.md - - 7-cicd/setup-cd.md - - 7-cicd/setup-ci.md - 7-cicd/pr-mr-summary.md #- 7-cicd/recce-debug.md # content outdated - - 7-cicd/scenario-dev.md + - 7-cicd/scenario-dev.md - 7-cicd/scenario-pr-review.md - 7-cicd/preset-checks.md - - 7-cicd/best-practices-prep-env.md - Technical Concepts: - 8-technical-concepts/state-file.md @@ -156,6 +156,12 @@ theme: plugins: - search + - redirects: + redirect_maps: + '7-cicd/setup-cd.md': '2-getting-started/setup-cd.md' + '7-cicd/setup-ci.md': '2-getting-started/setup-ci.md' + '7-cicd/ci-cd-getting-started.md': '2-getting-started/environment-setup.md' + '7-cicd/best-practices-prep-env.md': '2-getting-started/environment-best-practices.md' - glightbox: skip_classes: - skip-glightbox diff --git a/requirements.txt b/requirements.txt index 7359048..44e20f7 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,4 @@ mkdocs-material +mkdocs-redirects mkdocs-glightbox mkdocs-material[imaging] \ No newline at end of file