Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 22 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,24 +45,23 @@ The following are potential goals we are not yet certain of:

Both of these potential goals could pose challenges to interoperability, so we want to investigate more how important such functionality is to developers to find the right tradeoff.

### API Updates: Deprecations and Renaming
## Experiments and Updates

To improve API clarity, consistency, and address inconsistencies in parameter support across various models, the LanguageModel API has been updated.
### Sampling Parameters

**Deprecated Features:**
Developers have expressed the value of tuning language model sampling parameters for testing and optimizing task-specific model behavior. At the same time, web standards engagements have highlighted the need for more interoperable API shapes for sampling parameters among different models.

The following features of the LanguageModel API are **deprecated** and their functionality is now restricted to web extension contexts only:
The API was initially made available in extension contexts with the following sampling parameter options and attributes:

* The static method `LanguageModel.params()`
* The instance attributes `languageModel.topK` and `languageModel.temperature`
* The `LanguageModelParams` interface and all its attributes (`defaultTopK`, `maxTopK`, `defaultTemperature`, `maxTemperature`)
* The `topK` and `temperature` options within `LanguageModel.create()`
* The static method `LanguageModel.params()`, which exposes default and maximum values for sampling parameters: `defaultTemperature`, `maxTemperature`, `defaultTopK`, `maxTopK`.
* The `temperature` and `topK` options, which may be provided to `LanguageModel.create()` to control the sampling behavior of individual language model sessions.
* The `temperature` and `topK` attributes on `LanguageModel` session instances, which expose the current values of the sampling parameters for that session.

These features may be completely removed in the future. This change is intended to simplify the API and address inconsistencies in parameter support across various models.
Access to these features is limited to extension and experimental web contexts. Ongoing experimentation and community engagement will explore different API shapes that satisfy developer requirements and address interoperability concerns.

**Renamed Features:**
### Renamed Features

The following features have been renamed. The old names are now deprecated and will only function as aliases within Chrome Extension contexts:
The following features have been recently renamed. The legacy aliases are deprecated, and clients should update their code to use the new names. The legacy aliases will be removed from extension contexts in a future release.

| Old Name (Deprecated in Extensions, Removed in Web) | New Name (Available in All Contexts) |
| :-------------------------------------------------- | :----------------------------------------|
Expand All @@ -71,7 +70,6 @@ The following features have been renamed. The old names are now deprecated and w
| `languageModel.measureInputUsage()` | `languageModel.measureContextUsage()` |
| `languagemodel.onquotaoverflow` | `languagemodel.oncontextoverflow`. |

**Note:** Developers using any of the deprecated features within an extension context will receive warnings in the DevTools Issues tab. These deprecated features and aliases may be completely removed in a future Chrome release.

## Examples

Expand Down Expand Up @@ -454,23 +452,26 @@ Note that `append()` can also cause [overflow](#tokenization-context-window-leng

### Configuration of per-session parameters

In addition to the `initialPrompts` option shown above, in extension contexts, the currently-configurable model parameters are [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [top-K](https://huggingface.co/blog/how-to-generate#top-k-sampling). The `params()` API gives the default and maximum values for these parameters.
Tuning language model sampling parameters can be useful for both testing and adjusting task-specific model behavior. Common sampling parameters include [temperature](https://huggingface.co/blog/how-to-generate#sampling) and [topK](https://huggingface.co/blog/how-to-generate#top-k-sampling).

**Deprecation Notice:** The `topK` and `temperature` options for `LanguageModel.create()`, the `LanguageModel.params()` static method, and the `languageModel.topK` and `languageModel.temperature` instance attributes are now **deprecated**. These features are only functional within web extension contexts and will be ignored or unavailable in standard web page contexts. They may be completely removed in a future release.

The `LanguageModel.params()` API, only available in extensions, can be used to query the default and maximum values for these parameters.
**Notice:** Sampling parameter features are currently only available within extension and experimental contexts. While they are useful for exploring model behavior, the current fields are not guaranteed to be supported or interpreted consistently across all models or user agents.

_The limited applicability and non-universal nature of these sampling hyperparameters are discussed further in [issue #42](https://github.com/webmachinelearning/prompt-api/issues/42): sampling hyperparameters are not universal among models._

In extension and experimental contexts:
* The `LanguageModel.params()` static method provides default and maximum values for temperature and topK parameters, once the user agent has ascertained or downloaded the specific underlying model.
* The `temperature` and `topK` instance attributes provide the current values for these parameters for a given session.
* Sampling parameters can also be configured at session creation time via the `temperature` and `topK` options for `LanguageModel.create()`



```js
// The topK and temperature members of the options object are deprecated. They will only be considered when
// LanguageModel.create() is called from within a Chrome Extension. In web page contexts, they are ignored.
// Sampling parameter support is limited to extension and experimental web contexts.
// Accessors are undefined, and options are ignored, outside of those contexts.
const customSession = await LanguageModel.create({
temperature: 0.8,
topK: 10
});
// This interface and all its attributes (`defaultTopK`, `maxTopK`, `defaultTemperature`, `maxTemperature`)
// are now only available within Chrome Extension contexts. Web pages can no longer call this method.
const params = await LanguageModel.params();
const conditionalSession = await LanguageModel.create({
temperature: isCreativeTask ? params.defaultTemperature * 1.1 : params.defaultTemperature * 0.8,
Expand Down Expand Up @@ -831,7 +832,7 @@ Note that although the API is not exposed to web platform workers, a browser cou
To actually get a response back from the model given a prompt, the following possible stages are involved:

1. Download the model, if necessary.
2. Establish a session, including configuring per-session options and parameters.
2. Establish a session, including configuring per-session options.
3. Add an initial prompt to establish context. (This will not generate a response.)
4. Execute a prompt and receive a response.

Expand Down
12 changes: 6 additions & 6 deletions index.bs
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ These APIs are part of a family of APIs expected to be powered by machine learni
interface LanguageModel : EventTarget {
static Promise<LanguageModel> create(optional LanguageModelCreateOptions options = {});
static Promise<Availability> availability(optional LanguageModelCreateCoreOptions options = {});
// **DEPRECATED**: This method is only available in extension contexts.
// **EXPERIMENTAL**: Only available in extension and experimental contexts.
static Promise<LanguageModelParams?> params();

// These will throw "NotSupportedError" DOMExceptions if role = "system"
Expand Down Expand Up @@ -75,16 +75,16 @@ interface LanguageModel : EventTarget {
// **DEPRECATED**: This attribute is only available in extension contexts.
attribute EventHandler onquotaoverflow;

// **DEPRECATED**: This attribute is only available in extension contexts.
// **EXPERIMENTAL**: Only available in extension and experimental contexts.
readonly attribute unsigned long topK;
// **DEPRECATED**: This attribute is only available in extension contexts.
// **EXPERIMENTAL**: Only available in extension and experimental contexts.
readonly attribute float temperature;

Promise<LanguageModel> clone(optional LanguageModelCloneOptions options = {});
undefined destroy();
};

// **DEPRECATED**: This interface and its attributes are only available in extension contexts.
// **EXPERIMENTAL**: Only available in extension and experimental contexts.
[Exposed=Window, SecureContext]
interface LanguageModelParams {
readonly attribute unsigned long defaultTopK;
Expand All @@ -109,9 +109,9 @@ dictionary LanguageModelTool {
dictionary LanguageModelCreateCoreOptions {
// Note: these two have custom out-of-range handling behavior, not in the IDL layer.
// They are unrestricted double so as to allow +Infinity without failing.
// **DEPRECATED**: This option is only allowed in extension contexts.
// **EXPERIMENTAL**: Only available in extension and experimental contexts.
unrestricted double topK;
// **DEPRECATED**: This option is only allowed in extension contexts.
// **EXPERIMENTAL**: Only available in extension and experimental contexts.
unrestricted double temperature;

sequence<LanguageModelExpected> expectedInputs;
Expand Down
Loading