How to create an agent to your cluster

Note

This page is about the in-app chatbot agent you configure in Neos Studio for the end users of a cluster. For the AI coding assistant that helps developers build Neos clusters from Claude Code or GitHub Copilot, see Neos Copilot.

Configure AI Models

You must configure AI models in Neos Studio.

A model must have an unique name, the model ID and the provider.

Identifier

When using a model hosted on Microsoft Foundry, the model ID must be the deployment ID. By default, the deployment ID is the same as the model ID, but the same model can be deployed multiple times with different deployment IDs.

Provider

We support the following providers : OpenAI, Anthropic, Google, Mistral, Hugging Face, Ollama.

You can use models hosted on Microsoft Foundry, AWS Bedrock, OpenRouter, ...

Type

The type defines how the model is used. A model has a single type, so you must configure one model per usage.

Type	Description	Supported providers
`Chat`	Default value. Required to use the model in an agent, in the chatbot or to invoke it by code.	All providers
`Embeddings`	Required to generate embeddings (vector representations of text), for example to implement semantic search.	OpenAI, Google, Hugging Face, Ollama

When the type is Embeddings, you must also set the vector dimension, which is the size of the vectors generated by the model (for example 3072 for text-embedding-3-large).

Optional properties

The organization ID is deprecated and is specific to the official OpenAI provider: Previously, the API key generated on the developer portal was not linked to an organization. Now that it is, there is no longer any need to provide it.

Configure secrets

There are 2 secrets to set to be able to use AI models:

The API key which is always required except when using Ollama since the model will be hosted locally.
The endpoint which is optional when using the provider's official endpoint, but required when using an alternative endpoint such as Microsoft Foundry.

Development

This is the recommended way in development mode.

To set user secrets, you must execute the following commands:

dotnet user-secrets set "AI:[Provider]:ApiKey" "[ApiKey]" --id "[ClusterRootNamespace].AspNetCore"
dotnet user-secrets set "AI:[Provider]:ApiEndpoint" "[ApiEndpoint]" --id "[ClusterRootNamespace].AspNetCore"

The cluster root namespace can be found in the cluster configuration file.

For example, to use OpenAI models hosted on Microsoft Foundry on TechnicalDemos, you will need to execute:

dotnet user-secrets set "AI:OpenAI:ApiKey" "..." --id "TechnicalDemos.AspNetCore"
dotnet user-secrets set "AI:OpenAI:ApiEndpoint" "https://neosazopenaisweden.openai.azure.com/openai/v1" --id "TechnicalDemos.AspNetCore"

You can override the secrets for a specific model by adding the model name instead of the provider name:

dotnet user-secrets set "AI:NeosGpt6Luna:ApiKey" "..." --id "TechnicalDemos.AspNetCore"
dotnet user-secrets set "AI:NeosGpt6Luna:ApiEndpoint" "https://neosazopenaisweden.openai.azure.com/openai/v1" --id "TechnicalDemos.AspNetCore"

Production

In Kubernetes, the recommended way is to set the secrets in a store.

Key	Value
AI:[Provider]:ApiKey	The API key (requires)
AI:[Provider]:ApiEndpoint	The endpoint (optional)

For example, with a OpenAI model hosted on Microsoft Foundry:

Key	Value
AI:OpenAI:ApiKey	...
AI:OpenAI:ApiEndpoint	https://neosazopenaisweden.openai.azure.com/openai/v1

You can override the secrets for a specific model by adding the model name instead of the provider name:

Key	Value
AI:NeosGpt6Luna:ApiKey	...
AI:NeosGpt6Luna:ApiEndpoint	https://neosazopenaisweden.openai.azure.com/openai/v1

Network timeout

The calls made to the provider have a network timeout of 5 minutes by default, so that reasoning models have enough time to think before answering.

It can be changed, in seconds, with the same keys as the secrets: the model name overrides the provider name, which overrides the value shared by all the models.

Key	Value
AI:[ModelName]:NetworkTimeout	The timeout of a model in seconds (optional)
AI:[Provider]:NetworkTimeout	The timeout of the models of a provider in seconds (optional)
AI:NetworkTimeout	The timeout of all the models in seconds (optional)

For example, to give 10 minutes to the NeosGpt6Luna model:

dotnet user-secrets set "AI:NeosGpt6Luna:NetworkTimeout" "600" --id "TechnicalDemos.AspNetCore"

The timeout applies to each call sent to the provider, not to the whole answer: an agent which calls skills sends several calls to the model, and the provider SDK can retry a call which failed.

Create an Agent

Then, you must configure an agent in Neos Studio.

Property	Description	Example
Identifier	A unique name	OrderCreator
Name	The displayed name in the chatbot	Bob
ModelName	The model name configured in the previous step	NeosGpt6Luna
IconName	The name of the icon displayed in the chatbot	OrderIcon
Temperature	What sampling temperature to use. Higher values mean the model will take more risks. Try 0.9 for more creative applications and 0 (argmax sampling) for ones with a well-defined answer.	0.9
TopP	An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with the top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered	1
MaxTokens	The maximum number of tokens to generate in the completion.
ReasoningEffort	The reasoning effort to use for the agent, on models that support reasoning. The higher the effort, the more the model reasons before answering. Minimal is sent as Low, XHigh and Max are both sent as the extra high level.	Medium
IncludeSkillInstructions	If true, the instructions of the skills used by the agent will be included in the prompt sent to the model. If false you can include them by using handlebars in the agent prompt	True
Description	A localized string to describe the agent	en: Order creator, fr: Créateur de commande
Instructions	A system-level instruction to guide your model's behavior throughout the conversation. The assistant refers to himself in the first person singular.	I'm an assistant that helps the user to create orders
Skills	The skills of your agent. You should select one or more skills provided by Neos or your own skills	OrderCreation

Note

When you create an agent, you need to generate the cluster to apply the changes. When you edit an agent except its name, you do not need to generate the cluster, the changes are applied immediately after saving.

Warning

The agent is not visible in the chatbot if the API Key or API endpoint are not configured.

Skills provided by Neos

Neos provides several skills that you can use to extend the capabilities of your agent. You don't need to reference the functions of these skills in the instructions of the agent, they are automatically available for the assistant.

The skills that read or create data return the value of a property with a masking type masked, as the API does: the model never receives the real value.

EntitiesExplorer Skill

Provides functionalities to explore entities.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IEntitiesExplorerSkill interface.

You must include this skill if the agent needs to explore entities to find an entity needed to perform an action asked by the user. Only entities with read permission will be provided.

Functions provided by this skill:

Function name	C# Method name	Description
GetAllEntities	Task GetAllEntitiesAsync()	Provides a markdown list of all entities with their description. Only entities with read permission will be provided.
GetEntityProperties	Task GetEntityPropertiesAsync(string entityName)	Provides a markdown list of all entity properties with their caption and description for the specified entity. Navigation properties to unauthorized entities will not be provided. Properties without caption and AI description will not be provided, so you can hide a property to an agent.
GetEnumValues	string GetEnum(string enumName)	Provides a markdown list of all enum members with their caption for the specified enum.

EntitiesQuerierSkill

Provides functionalities to query entities. You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IEntitiesQuerierSkill interface.

You must include this skill if the agent needs to query entities with OData queries.

Functions provided by this skill:

Function name	C# Method name	Description
QueryExpressionEvaluator	Task QueryExpressionEvaluatorAsync(string entityName, string? filter, string? apply, string? select, string? expand, string? orderBy, int? top)	Executes an OData query on an entity and returns its rows in JSON.

The agent queries an entity with the OData version 4 query options $filter, $apply (aggregations), $select, $expand, $orderby and $top. The query options are translated into a LINQ query, which is translated into SQL by Entity Framework: no code is compiled or loaded in memory.

Only the entities and the properties described by the entity metadata, the ones returned by the EntitiesExplorer skill, can be queried. Two entities can only be queried together through a navigation property, and a property of type LocalizableString is filtered or sorted with a language code, for example Caption/en, while selecting it returns all its translations. A navigation property to a single entity can be expanded, but a collection navigation property cannot, because its related rows would not be limited: the agent queries the related entity with a filter on its navigation property, or aggregates the collection with $apply.

A property with a masking type is returned masked in the rows. $filter and $orderby can still use it, on the server, but a query whose $apply groups, aggregates or computes it is refused with an error: the resulting value would no longer be tied to the property, so it could not be masked.

Note

The QueryExpressionEvaluator function can only query entities with read permission, otherwise an exception is thrown. The read permission is checked on the queried entity and on every entity used by the query through its navigation properties. So you must check if the entity is authorized if your cluster used the user permissions module.

Note

Performance considerations
To avoid loading a whole table in memory, a query returns at most 1000 rows by default. If a query returns more rows, an error is returned to the agent, which must then filter or aggregate the data in its query. You can change this value in the Yaml configuration file of your cluster, for example to set the maximum number of rows to 500:

AI:
  EntityQuerierSkill:
    MaxResultCount: 500

A query has a maximum execution time of 30 seconds by default. When it is exceeded, the database query is cancelled and an error is returned to the agent. You can change this value in the Yaml configuration file of your cluster, for example to set the maximum execution time to 10 seconds:

AI:
  EntityQuerierSkill:
    ExecutionTimeout: 10

With the YAML persistence, used by Neos Studio, the queries are executed synchronously and cannot be interrupted: the timeout is only checked between the returned rows.

EntityViewsExplorer Skill

Provides functionalities to explore entity views.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IEntitiesExplorerSkill interface.

You must include this skill if the agent needs to explore entity views to find an entity view needed to perform an action asked by the user. For example to create an order, the agent needs to explore the entity views in order to find the appropriate entity view to create an Order.

Functions provided by this skill:

Function name	Description
string GetReadableEntityViews()	Provides a markdown list of all entity views with their description. Only entity views with read permission will be provided.
string GetWritableEntityViews()	Provides a markdown list of all entity views with their description. Only entity views with create, update or delete permission will be provided.
string GetEntityProperties(string entityViewName)	Provides a markdown list of all entity view property with their description for the specified entity View.

EntityViewQuerierSkill

Provides functionalities to query entity views.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IEntitiesQuerierSkill interface.

You must include this skill if the agent needs to query entity views. For example to get the average price of a product, the agent needs to query the entity views to get the price of the product.

As with the EntitiesQuerierSkill, a masked property can be filtered and sorted, but a query whose $apply groups, aggregates or computes it is refused with an error. A query that only counts the rows ignores its $apply: it returns the number of rows that match its $filter.

Functions provided by this skill:

Function name	Description
string HowToDoODataQuery()	Provide instructions to create an ODataQuery

DataCreatorSkill

Provides functionalities to create data.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IDataCreatorSkill interface.

You must include this skill if the agent needs to create data. For example to create an employee, the agent needs to create an JSON schema to create a JSON string and send it to the API.

Functions provided by this skill:

Function name	Description
string? GetEntityViewJsonSchema(string entityViewName)	Provide a Json schema of POST API Model for the specified API
Task CreateEntityViewAsync(string? entityViewName, string? jsonString)	Create an entity view with the specified entity view name and JSON string

MermaidGraphSkill

Provides functionalities to create mermaid graphs.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IMermaidGraphSkill interface.

You must include this skill if the agent needs to create mermaid graphs.

Functions provided by this skill:

Function name	Description
string HowToDoMermaidGraph()	Provide instructions to build mermaid charts and diagrams

UIInteractionSkill

Provides functionalities to interact with the UI.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IUIInteractionSkill interface.

You must include this skill if the agent needs to interact with UI. This skill can open a UI view in creation mode with initial data and can open an UI view in edition mode.

Functions provided by this skill:

Function name	Description
string CreateNewEntityAsync(string? uiViewName, string? jsonString)	Opens the specified UI view with the initial data as JSON
string OpenUIViewNameAsync(string? uiViewName, string? identifiers, string? parameters)	Opens the UI view with the identifiers of an item and with parameters

DataProposalSkill

Provides functionalities to propose new data in a UI form.

You can reference this skill as a dependency of other skills using GroupeIsa.Neos.Application.AI.Skills.IDataProposalSkill interface.

Functions provided by this skill:

Function name	Description
string CreateNewEntityAsync(string? uiViewName, string? jsonString)	Open the specified UI view with the initial data as JSON

Create your own skill

See Create your own skill.

How to Monitor the Chatbot

You can create a class that implements the IChatBotInterceptor interface to get information about the conversation with the chatbot. Example:

public class ChatBotInterceptor : IChatBotInterceptor
{
    /// <inheritdoc />
    public Task OnFunctionInvokedAsync(string agentId, string threadId, string functionName, Dictionary<string, object?> parameters, object? result)
    {
        // Do what you want with the information about the function call.
    }

    /// <inheritdoc />
    public Task OnPromptInvokedAsync(string agentId, string threadId, string question, string answer)
    {
        // Do what you want with the exchange between the user and the chatbot.
    }
}

Once you have created the class, you need to register it in the Startup class:

public static class Startup
{
    public static void ConfigureServices(IServiceCollection services)
    {
        services.AddScoped<IChatBotInterceptor, ChatBotInterceptor>();
    }
}

Table of Contents

How to create an agent to your cluster

Note

Configure AI Models

Identifier

Provider

Type

Optional properties

Configure secrets

Development

Production

Network timeout

Create an Agent

Note

Warning

Skills provided by Neos

EntitiesExplorer Skill

EntitiesQuerierSkill

Note

Note

EntityViewsExplorer Skill

EntityViewQuerierSkill

DataCreatorSkill

MermaidGraphSkill

UIInteractionSkill

DataProposalSkill

Create your own skill

How to Monitor the Chatbot