Provider Development

This section covers how to develop custom model provider plugins to integrate new LLM platforms into Astrsomn.

Overview

Astrsomn loads model provider plugins via SPI (Service Provider Interface). Each provider needs to:

1. Implement ModelProviderHandler (recommended: extend AbstractModelProviderHandler)
2. Register SPI service files
3. Create an AstroExtensionDescriptor for plugin metadata

Architecture

A model provider's role is to bridge LangChain4j model interfaces (ChatModel / StreamingChatModel / EmbeddingModel) with a specific LLM platform's API.

Development Steps

1. Create Maven Module

<project>
    <groupId>com.astrsomn</groupId>
    <artifactId>astrsomn-provider-custom</artifactId>
    <version>0.2.0-SNAPSHOT</version>

    <dependencies>
        <!-- Core API dependency -->
        <dependency>
            <groupId>com.astrsomn</groupId>
            <artifactId>astrsomn-api-runtime</artifactId>
            <version>0.2.0-SNAPSHOT</version>
        </dependency>

        <!-- LangChain4j model interfaces -->
        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j</artifactId>
        </dependency>
    </dependencies>
</project>

2. Implement ModelProviderHandler

Extend AbstractModelProviderHandler — it provides template methods for model type dispatch, parameter validation, and more:

package com.astrsomn.provider.custom;

import com.astrsomn.api.runtime.common.constant.AiModelEnum;
import com.astrsomn.api.runtime.common.entity.AiModelEntity;
import com.astrsomn.api.runtime.common.langchain.buildParam.AstroChatParam;
import com.astrsomn.api.runtime.common.langchain.extension.model.AbstractModelProviderHandler;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.StreamingChatModel;

import java.util.List;

public class CustomProviderHandler extends AbstractModelProviderHandler {

    @Override
    public AiModelEnum.ProviderEnum getProvider() {
        return AiModelEnum.ProviderEnum.CUSTOM;  // Register enum first
    }

    @Override
    public <T> T createModel(Class<T> modelClass, AstroChatParam<?> param) {
        // Use parent's template method for dispatch
        return dispatchModel(modelClass, param);
    }

    @Override
    protected ChatModel getChatModel(AstroChatParam<?> param) {
        // Build synchronous chat model
        return CustomChatModel.builder()
                .apiKey(param.getApiKey())
                .baseUrl(param.getBaseUrl())
                .modelName(resolveModelKey(param))
                .temperature(param.getModelSetting().getTemperature())
                .maxTokens(param.getModelSetting().getMaxTokens())
                .build();
    }

    @Override
    protected StreamingChatModel getStreamModel(AstroChatParam<?> param) {
        // Build streaming chat model
        return CustomStreamingChatModel.builder()
                .apiKey(param.getApiKey())
                .baseUrl(param.getBaseUrl())
                .modelName(resolveModelKey(param))
                .build();
    }

    @Override
    public List<AiModelEntity> getAvailableModels(String apiKey, String apiSecret) {
        // Call vendor API to fetch available models
        return fetchModelsFromApi(apiKey, apiSecret);
    }
}

About AbstractModelProviderHandler

The base class provides:

• dispatchModel() — auto-dispatches to getChatModel / getStreamModel / getEmbeddingModel based on modelClass
• resolveModelKey() — resolves model name from parameters
• validateParams() — null-safety validation
• toJson() — object to JSON serialization

3. Create Extension Descriptor

Create an AstroExtensionDescriptor subclass for plugin metadata:

package com.astrsomn.provider.custom;

import com.astrsomn.api.runtime.common.constant.AiModelEnum;
import com.astrsomn.api.system.entity.SystemExtensionEnum;
import com.astrsomn.starter.system.AstroExtensionDescriptor;

public class CustomExtensionDescriptor extends AstroExtensionDescriptor {

    @Override
    public String getExtensionKey() {
        return AiModelEnum.ProviderEnum.CUSTOM.getCode();
    }

    @Override
    public String getExtensionCode() {
        return AiModelEnum.ProviderEnum.CUSTOM.getCode();
    }

    @Override
    public SystemExtensionEnum.ExtensionTypeEnum getExtensionType() {
        return SystemExtensionEnum.ExtensionTypeEnum.MODEL_PROVIDER;
    }
}

4. Create Extension Properties File

Create src/main/resources/extension-custom.properties:

name=Custom Model Provider
version=0.2.0-SNAPSHOT
description=Custom LLM provider plugin
changeLog=Initial release
minServerVersion=0.2.0

5. Register SPI

Create two files under src/main/resources/META-INF/services/:

File 1: com.astrsomn.api.runtime.common.langchain.extension.model.ModelProviderHandler

com.astrsomn.provider.custom.CustomProviderHandler

File 2: com.astrsomn.api.runtime.common.langchain.extension.AstroExtensionDescriptor

com.astrsomn.provider.custom.CustomExtensionDescriptor

Interface Reference

ModelProviderHandler

Package: com.astrsomn.api.runtime.common.langchain.extension.model

Method	Return Type	Description
getProvider()	AiModelEnum.ProviderEnum	Returns provider enum identifier
createModel(Class<T>, AstroChatParam<?>)	<T> T	Creates model instance (Chat / Streaming / Embedding)
getAvailableModels(String, String)	List<AiModelEntity>	Fetches available model list
getVersion()	String	Plugin version (default: "1.0.0")
getAuthor()	String	Author info (default: "Astrsomn")

AbstractModelProviderHandler

Package: com.astrsomn.api.runtime.common.langchain.extension.model

Method	Description
dispatchModel(Class<T>, AstroChatParam<?>)	Template method — dispatches by type to getChatModel / getStreamModel / getEmbeddingModel
resolveModelKey(AstroChatParam<?>)	Resolves model name from parameters
validateParams(Class<?>, AstroChatParam<?>)	Validates non-null parameters
getChatModel(AstroChatParam<?>)	Override — creates synchronous ChatModel
getStreamModel(AstroChatParam<?>)	Override — creates StreamingChatModel
getEmbeddingModel(AstroChatParam<?>)	Override — creates EmbeddingModel

Build & Deploy

# Package the plugin
mvn clean package -DskipTests

# Copy JAR to Astrsomn Server plugins directory
cp target/astrsomn-provider-custom-0.2.0-SNAPSHOT.jar /path/to/astrsomn/plugins/

Auto-Discovery

The framework auto-scans the plugins/ directory at startup and loads all extensions via SPI. No framework source modifications needed.

Reference Implementations

Study existing provider source code:

• DeepSeek — astrsomn-plugins/astrsomn-providers/astrsomn-provider-deepseek/ — OpenAI-compatible protocol, supports Chat / Stream / Embedding
• Zhipu — astrsomn-plugins/astrsomn-providers/astrsomn-provider-zhipu/
• OpenAI — astrsomn-plugins/astrsomn-providers/astrsomn-provider-openai/

Related Documentation

• Interface Guide — Detailed API specifications
• Examples — Complete end-to-end examples
• Vector Store Development — Vector store plugin guide
• MCP Development — MCP protocol plugin guide