Add URL validation and improve scope flag parsing#50
Merged
YusukeHirao merged 5 commits intomainfrom Mar 9, 2026
Merged
Conversation
- --scope フラグのカンマ区切りで空文字列をフィルタリング - --list-file で空リストの場合にエラーメッセージを表示 - URL 引数を new URL() で検証し、無効な場合にエラーを表示 Closes #17 https://claude.ai/code/session_01JZboa1PjwHA9voNa2Gx8KZ
- validateUrls ヘルパーを抽出し、--list / --list-file / 単一URL引数の全パスで適用 - --list-file 内の無効URLを検出するテストを追加 - --list 内の無効URLを検出するテストを追加 - scope 全空文字列のエッジケーステストを追加 https://claude.ai/code/session_01JZboa1PjwHA9voNa2Gx8KZ
crawl コマンドの URL 引数にプロトコル付き完全形式が必要であることを明記。 https://claude.ai/code/session_01JZboa1PjwHA9voNa2Gx8KZ
エクスポートされたトップレベル定数に JSDoc が欠落していたため追記。 https://claude.ai/code/session_01JZboa1PjwHA9voNa2Gx8KZ
- titleOnly → metadataOnly に用語を修正（実装に合わせる） - Archive.write() の説明に snapshot の zip 圧縮ステップを追記 https://claude.ai/code/session_01JZboa1PjwHA9voNa2Gx8KZ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds comprehensive URL validation to the crawl command and improves the handling of the
--scopeflag by filtering out empty values.closes: #17
Key Changes
URL Validation: Added a new
validateUrls()function that validates all URLs using the URL constructor before processing. This ensures:--list-file,--list, and positional arguments are caught early with helpful error messagesScope Flag Parsing: Enhanced
mapFlagsToCrawlConfig()to filter out empty strings and whitespace-only values when parsing comma-separated scope values, preventing invalid empty scope entriesDocumentation Updates:
https://example.com) are requiredtitleOnlytometadataOnlyTest Coverage: Added 6 new test cases covering:
Implementation Details
The
validateUrls()function attempts to construct a URL object for each provided URL string, catching any parsing errors and throwing a descriptive error message. This validation is called at three key points in the crawl command flow: when using--list-file, when combining--listwith arguments, and when using a single positional argument.https://claude.ai/code/session_01JZboa1PjwHA9voNa2Gx8KZ