Open Source Models

Open source model components provide access to locally-hosted and open source language models.

Ollama

This component generates text using Ollama's language models.

To use this component in a flow, connect BroxiAI to your locally running Ollama server and select a model.

1. In the Ollama component, in the Base URL field, enter the address for your locally running Ollama server. This value is set as the OLLAMA_HOST environment variable in Ollama. The default base URL is http://127.0.0.1:11434.

2. To refresh the server's list of models, click .

3. In the Model Name field, select a model. This example uses llama3.2:latest.

4. Connect the Ollama model component to a flow. For example, this flow connects a local Ollama server running a Llama 3.2 model as the custom model for an Agent component.

Ollama model as Agent custom model

For more information, see the Ollama documentation.

Inputs

Name	Display Name	Info
Base URL	Base URL	Endpoint of the Ollama API.
Model Name	Model Name	The model name to use.
Temperature	Temperature	Controls the creativity of model responses.

Outputs

Name	Type	Description
model	LanguageModel	An instance of an Ollama model configured with the specified parameters.

LMStudio

This component generates text using LM Studio's local language models.

For more information, see LM Studio documentation.

Inputs

Name	Type	Description
base_url	String	The URL where LM Studio is running. Default: "http://localhost:1234".
max_tokens	Integer	Maximum number of tokens to generate in the response. Default: 512.
temperature	Float	Controls randomness in the output. Range: [0.0, 2.0]. Default: 0.7.
top_p	Float	Controls diversity via nucleus sampling. Range: [0.0, 1.0]. Default: 1.0.
stop	List[String]	List of strings that will stop generation when encountered (advanced).
stream	Boolean	Whether to stream the response. Default: False.
presence_penalty	Float	Penalizes repeated tokens. Range: [-2.0, 2.0]. Default: 0.0.
frequency_penalty	Float	Penalizes frequent tokens. Range: [-2.0, 2.0]. Default: 0.0.

Outputs

Name	Type	Description
model	LanguageModel	An instance of LMStudio configured with the specified parameters.

Hugging Face API

This component sends requests to the Hugging Face API to generate text using the model specified in the Model ID field.

The Hugging Face API is a hosted inference API for models hosted on Hugging Face, and requires a Hugging Face API token to authenticate.

Inputs

Name	Type	Description
model_id	String	The model ID from Hugging Face Hub. For example, "gpt2", "facebook/bart-large".
huggingfacehub_api_token	SecretString	Your Hugging Face API token for authentication.
temperature	Float	Controls randomness in the output. Range: [0.0, 1.0]. Default: 0.7.
max_new_tokens	Integer	Maximum number of tokens to generate. Default: 512.
top_p	Float	Nucleus sampling parameter. Range: [0.0, 1.0]. Default: 0.95.
top_k	Integer	Top-k sampling parameter. Default: 50.
model_kwargs	Dictionary	Additional keyword arguments to pass to the model.

Outputs

Name	Type	Description
model	LanguageModel	An instance of HuggingFaceHub configured with the specified parameters.

Usage Notes

• Local Deployment: These models run locally, providing privacy and control over your data

• Cost Effective: No API costs after initial setup, ideal for development and testing

• Customization: Full control over model parameters and configurations

• Offline Capability: Can work without internet connection once models are downloaded