diff --git a/README.md b/README.md index cb49f54..6d542d8 100644 --- a/README.md +++ b/README.md @@ -45,24 +45,23 @@ The following are potential goals we are not yet certain of: Both of these potential goals could pose challenges to interoperability, so we want to investigate more how important such functionality is to developers to find the right tradeoff. -### API Updates: Deprecations and Renaming +## Experiments and Updates -To improve API clarity, consistency, and address inconsistencies in parameter support across various models, the LanguageModel API has been updated. +### Sampling Parameters -**Deprecated Features:** +Developers have expressed the value of tuning language model sampling parameters for testing and optimizing task-specific model behavior. At the same time, web standards engagements have highlighted the need for more interoperable API shapes for sampling parameters among different models. -The following features of the LanguageModel API are **deprecated** and their functionality is now restricted to web extension contexts only: +The API was initially made available in extension contexts with the following sampling parameter options and attributes: -* The static method `LanguageModel.params()` -* The instance attributes `languageModel.topK` and `languageModel.temperature` -* The `LanguageModelParams` interface and all its attributes (`defaultTopK`, `maxTopK`, `defaultTemperature`, `maxTemperature`) -* The `topK` and `temperature` options within `LanguageModel.create()` +* The static method `LanguageModel.params()`, which exposes default and maximum values for sampling parameters: `defaultTemperature`, `maxTemperature`, `defaultTopK`, `maxTopK`. +* The `temperature` and `topK` options, which may be provided to `LanguageModel.create()` to control the sampling behavior of individual language model sessions. +* The `temperature` and `topK` attributes on `LanguageModel` session instances, which expose the current values of the sampling parameters for that session. -These features may be completely removed in the future. This change is intended to simplify the API and address inconsistencies in parameter support across various models. +Access to these features is limited to extension and experimental web contexts. Ongoing experimentation and community engagement will explore different API shapes that satisfy developer requirements and address interoperability concerns. -**Renamed Features:** +### Renamed Features -The following features have been renamed. The old names are now deprecated and will only function as aliases within Chrome Extension contexts: +The following features have been recently renamed. The legacy aliases are deprecated, and clients should update their code to use the new names. The legacy aliases will be removed from extension contexts in a future release. | Old Name (Deprecated in Extensions, Removed in Web) | New Name (Available in All Contexts) | | :-------------------------------------------------- | :----------------------------------------| @@ -71,7 +70,6 @@ The following features have been renamed. The old names are now deprecated and w | `languageModel.measureInputUsage()` | `languageModel.measureContextUsage()` | | `languagemodel.onquotaoverflow` | `languagemodel.oncontextoverflow`. | -**Note:** Developers using any of the deprecated features within an extension context will receive warnings in the DevTools Issues tab. These deprecated features and aliases may be completely removed in a future Chrome release. ## Examples @@ -454,23 +452,26 @@ Note that `append()` can also cause [overflow](#tokenization-context-window-leng ### Configuration of per-session parameters -In addition to the `initialPrompts` option shown above, in extension contexts, the currently-configurable model parameters are [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [top-K](https://huggingface.co/blog/how-to-generate#top-k-sampling). The `params()` API gives the default and maximum values for these parameters. +Tuning language model sampling parameters can be useful for both testing and adjusting task-specific model behavior. Common sampling parameters include [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [topK](https://huggingface.co/blog/how-to-generate#top-k-sampling). -**Deprecation Notice:** The `topK` and `temperature` options for `LanguageModel.create()`, the `LanguageModel.params()` static method, and the `languageModel.topK` and `languageModel.temperature` instance attributes are now **deprecated**. These features are only functional within web extension contexts and will be ignored or unavailable in standard web page contexts. They may be completely removed in a future release. - -The `LanguageModel.params()` API, only available in extensions, can be used to query the default and maximum values for these parameters. +**Notice:** Sampling parameter features are currently only available within extension and experimental contexts. While they are useful for exploring model behavior, the current fields are not guaranteed to be supported or interpreted consistently across all models or user agents. _The limited applicability and non-universal nature of these sampling hyperparameters are discussed further in [issue #42](https://github.com/webmachinelearning/prompt-api/issues/42): sampling hyperparameters are not universal among models._ +In extension and experimental contexts: +* The `LanguageModel.params()` static method provides default and maximum values for temperature and topK parameters, once the user agent has ascertained or downloaded the specific underlying model. +* The `temperature` and `topK` instance attributes provide the current values for these parameters for a given session. +* Sampling parameters can also be configured at session creation time via the `temperature` and `topK` options for `LanguageModel.create()` + + + ```js -// The topK and temperature members of the options object are deprecated. They will only be considered when -// LanguageModel.create() is called from within a Chrome Extension. In web page contexts, they are ignored. +// Sampling parameter support is limited to extension and experimental web contexts. +// Accessors are undefined, and options are ignored, outside of those contexts. const customSession = await LanguageModel.create({ temperature: 0.8, topK: 10 }); -// This interface and all its attributes (`defaultTopK`, `maxTopK`, `defaultTemperature`, `maxTemperature`) -// are now only available within Chrome Extension contexts. Web pages can no longer call this method. const params = await LanguageModel.params(); const conditionalSession = await LanguageModel.create({ temperature: isCreativeTask ? params.defaultTemperature * 1.1 : params.defaultTemperature * 0.8, @@ -831,7 +832,7 @@ Note that although the API is not exposed to web platform workers, a browser cou To actually get a response back from the model given a prompt, the following possible stages are involved: 1. Download the model, if necessary. -2. Establish a session, including configuring per-session options and parameters. +2. Establish a session, including configuring per-session options. 3. Add an initial prompt to establish context. (This will not generate a response.) 4. Execute a prompt and receive a response. diff --git a/index.bs b/index.bs index 812520d..447787b 100644 --- a/index.bs +++ b/index.bs @@ -37,7 +37,7 @@ These APIs are part of a family of APIs expected to be powered by machine learni interface LanguageModel : EventTarget { static Promise create(optional LanguageModelCreateOptions options = {}); static Promise availability(optional LanguageModelCreateCoreOptions options = {}); - // **DEPRECATED**: This method is only available in extension contexts. + // **EXPERIMENTAL**: Only available in extension and experimental contexts. static Promise params(); // These will throw "NotSupportedError" DOMExceptions if role = "system" @@ -75,16 +75,16 @@ interface LanguageModel : EventTarget { // **DEPRECATED**: This attribute is only available in extension contexts. attribute EventHandler onquotaoverflow; - // **DEPRECATED**: This attribute is only available in extension contexts. + // **EXPERIMENTAL**: Only available in extension and experimental contexts. readonly attribute unsigned long topK; - // **DEPRECATED**: This attribute is only available in extension contexts. + // **EXPERIMENTAL**: Only available in extension and experimental contexts. readonly attribute float temperature; Promise clone(optional LanguageModelCloneOptions options = {}); undefined destroy(); }; -// **DEPRECATED**: This interface and its attributes are only available in extension contexts. +// **EXPERIMENTAL**: Only available in extension and experimental contexts. [Exposed=Window, SecureContext] interface LanguageModelParams { readonly attribute unsigned long defaultTopK; @@ -109,9 +109,9 @@ dictionary LanguageModelTool { dictionary LanguageModelCreateCoreOptions { // Note: these two have custom out-of-range handling behavior, not in the IDL layer. // They are unrestricted double so as to allow +Infinity without failing. - // **DEPRECATED**: This option is only allowed in extension contexts. + // **EXPERIMENTAL**: Only available in extension and experimental contexts. unrestricted double topK; - // **DEPRECATED**: This option is only allowed in extension contexts. + // **EXPERIMENTAL**: Only available in extension and experimental contexts. unrestricted double temperature; sequence expectedInputs;