You must not set both endpoint_name and cluster_id.
param databricks_uri: str = 'databricks'¶
The databricks URI. Only used when using a serving endpoint.
param endpoint_name: Optional[str] = None¶
Name of the model serving endpoint.
You must specify the endpoint name to connect to a model serving endpoint.
You must not set both endpoint_name and cluster_id.
param extra_params: Dict[str, Any] [Optional]¶
Any extra parameters to pass to the endpoint.
param host: str [Optional]¶
Databricks workspace hostname.
If not provided, the default value is determined by
the DATABRICKS_HOST environment variable if present, or
the hostname of the current Databricks workspace if running inside
a Databricks notebook attached to an interactive cluster in “single user”
or “no isolation shared” mode.
param max_tokens: Optional[int] = None¶
The maximum number of tokens to generate.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_kwargs: Optional[Dict[str, Any]] = None¶
Deprecated. Please use extra_params instead. Extra parameters to pass to
the endpoint.
param n: int = 1¶
The number of completion choices to generate.
param stop: Optional[List[str]] = None¶
The stop sequence.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.0¶
The sampling temperature.
param transform_input_fn: Optional[Callable] = None¶
A function that transforms {prompt, stop, **kwargs} into a JSON-compatible
request object that the endpoint accepts.
For example, you can apply a prompt template to the input prompt.