seedproject-web/api/.memory/llm.md

LLM Integration Documentation

Architecture

app/LLM/
├── LLMProvider.php     — Abstract base class (contract for all providers)
├── LLMManager.php      — Factory: loads org config from DB, returns provider instance
├── svcOpenAI.php       — OpenAI (fully implemented)
├── svcAnthropic.php    — Anthropic/Claude (fully implemented)
├── svcGemini.php       — Google Gemini (fully implemented)
├── svcMistral.php      — Mistral AI (fully implemented)
└── svcDeepSeek.php     — DeepSeek (fully implemented)

Database Storage — sp_orgs_meta

LLM configs are stored per-org in sp_orgs_meta:

column	description
orgid	Foreign key to sp_orgs
keyval	Provider key (e.g. llmOpenAI)
metval	JSON config blob
active	1 = enabled, 0 = disabled

keyval names

Provider	keyval
OpenAI	llmOpenAI
Anthropic	llmAnthropic
Gemini	llmGemini
Mistral	llmMistral
DeepSeek	llmDeepSeek

metval JSON structure (all providers)

{
    "secretKey":   "sk-...",
    "model":       "gpt-4o",
    "max_tokens":  4096,
    "temperature": 0.7
}

LLMManager — Factory Class

use App\LLM\LLMManager;

// Get a configured, ready-to-use provider instance
$llm = LLMManager::forOrg($orgId, 'openai');

// List all configured providers for an org
$providers = LLMManager::getAvailableProviders($orgId);
// returns: ['openai', 'gemini']

// Save or update a provider config
LLMManager::saveOrgConfig($orgId, 'openai', [
    'secretKey'   => 'sk-...',
    'model'       => 'gpt-4o',
    'max_tokens'  => 4096,
    'temperature' => 0.7,
]);

LLMProvider — Base Class Interface

All providers expose the same methods:

// Send a prompt, get full response
$response = $llm->sendPrompt(array $messages, array $options = []);
// Returns: ['success' => bool, 'content' => string, 'usage' => array, 'error' => string]

// Stream a prompt, callback per chunk
$result = $llm->streamPrompt(array $messages, array $options, callable $callback);
// Returns: ['success' => bool, 'error' => string]

// Get available models
$models = $llm->getModels();
// Returns: [['id' => 'gpt-4o', 'label' => 'GPT-4o'], ...]

// Test API key + connectivity
$status = $llm->testConnection();
// Returns: ['success' => bool, 'latency_ms' => int, 'error' => string]

Fluent setters (chainable)

$llm->setModel('gpt-4o')
    ->setMaxTokens(2048)
    ->setTemperature(0.5)
    ->setSystemPrompt('You are a helpful assistant.')
    ->setTimeout(60);

Usage Examples (PHP)

Basic chat

$llm      = LLMManager::forOrg($orgId, 'openai');
$response = $llm->sendPrompt([
    ['role' => 'user', 'content' => 'Summarize this contract: ...']
]);

if ($response['success']) {
    echo $response['content'];
    // $response['usage'] = ['prompt_tokens' => 120, 'completion_tokens' => 80, 'total_tokens' => 200]
}

With system prompt and custom options

$llm = LLMManager::forOrg($orgId, 'anthropic');
$llm->setSystemPrompt('You are a legal document assistant.');

$response = $llm->sendPrompt(
    [['role' => 'user', 'content' => 'Draft an NDA for two parties.']],
    ['model' => 'claude-sonnet-4-6', 'max_tokens' => 2048, 'temperature' => 0.3]
);

Streaming (PHP — CLI or long-running process)

$llm = LLMManager::forOrg($orgId, 'gemini');
$llm->streamPrompt(
    [['role' => 'user', 'content' => 'Write a report on...']],
    [],
    function(string $chunk) {
        echo $chunk;
        flush();
    }
);

Multi-turn conversation

$messages = [
    ['role' => 'user',      'content' => 'My name is Carlos.'],
    ['role' => 'assistant', 'content' => 'Nice to meet you, Carlos!'],
    ['role' => 'user',      'content' => 'What is my name?'],
];

$response = $llm->sendPrompt($messages);

API Endpoints — appi/controllers/llm.php

All endpoints are authenticated via Auth::API(). Internal AJAX requests (X-Requested-With: XMLHttpRequest) pass through automatically. External requests require ?apikey= param.

POST /appi/llm/chat

Full response.

// Request
{
    "provider": "openai",
    "org_id":   100,
    "messages": [{"role": "user", "content": "Hello"}],
    "options":  {"model": "gpt-4o", "max_tokens": 1024, "system_prompt": "..."}
}

// Response
{
    "success":  true,
    "provider": "openai",
    "content":  "Hello! How can I help you?",
    "usage":    {"prompt_tokens": 10, "completion_tokens": 8, "total_tokens": 18}
}

POST /appi/llm/stream

SSE streaming. Each chunk sent as:

event: chunk
data: {"chunk":"partial text here"}

event: done
data: "[DONE]"

event: error
data: {"error":"something went wrong"}

GET /appi/llm/models?provider=openai&org_id=100

{
    "success":  true,
    "provider": "openai",
    "models":   [{"id": "gpt-4o", "label": "GPT-4o"}, ...]
}

GET /appi/llm/providers?org_id=100

{
    "success":   true,
    "org_id":    100,
    "providers": ["openai", "gemini"]
}

POST /appi/llm/test

// Request
{"provider": "openai", "org_id": 100}

// Response
{"success": true, "provider": "openai", "latency_ms": 342, "error": ""}

Frontend JS Client — public/assets/js/llm-client.js

Load in any page that needs LLM functionality:

$this->JavaScript[] = '/public/assets/js/llm-client.js';
$this->view->JavaScript = $this->JavaScript;

LLMClient.chat()

const res = await LLMClient.chat({
    provider: 'openai',
    orgId:    100,
    messages: [{ role: 'user', content: 'Hello' }],
    options:  { model: 'gpt-4o', system_prompt: 'You are helpful.' }
});
console.log(res.content);

LLMClient.stream()

const output = document.getElementById('output');

await LLMClient.stream({
    provider: 'anthropic',
    orgId:    100,
    messages: [{ role: 'user', content: 'Write a report on...' }],
    options:  { model: 'claude-sonnet-4-6' },
    onChunk:  (chunk) => { output.innerHTML += chunk; },
    onDone:   ()      => { console.log('Stream complete'); },
    onError:  (err)   => { console.error('Error:', err); }
});

LLMClient.getModels()

const { models } = await LLMClient.getModels('openai', 100);
// models = [{ id: 'gpt-4o', label: 'GPT-4o' }, ...]

LLMClient.getProviders()

const { providers } = await LLMClient.getProviders(100);
// providers = ['openai', 'gemini']

LLMClient.test()

const { success, latency_ms, error } = await LLMClient.test('openai', 100);

Provider API Differences (internals)

Provider	Auth Header	System Prompt	Role names	Quirks
OpenAI	Authorization: Bearer	role: system msg	user / assistant	No temperature on o1/o3 models
Anthropic	x-api-key	Top-level system key	user / assistant	Requires anthropic-version header
Gemini	?key= query param	systemInstruction	user / model	Messages use parts: [{text}]
Mistral	Authorization: Bearer	role: system msg	user / assistant	None
DeepSeek	Authorization: Bearer	role: system msg	user / assistant	No temperature on R1 (reasoner)

Available Models

OpenAI

• gpt-4o — GPT-4o (default)
• gpt-4o-mini — GPT-4o Mini
• gpt-4-turbo — GPT-4 Turbo
• o1 — o1
• o1-mini — o1 Mini
• o3-mini — o3 Mini

Anthropic

• claude-sonnet-4-6 — Claude Sonnet 4.6 (default)
• claude-opus-4-6 — Claude Opus 4.6
• claude-haiku-4-5-20251001 — Claude Haiku 4.5

Gemini

• gemini-2.5-flash — Gemini 2.5 Flash (default)
• gemini-2.5-flash-lite — Gemini 2.5 Flash Lite
• gemini-2.0-flash — Gemini 2.0 Flash
• gemini-1.5-pro — Gemini 1.5 Pro

Mistral

• mistral-large-latest — Mistral Large (default)
• mistral-small-latest — Mistral Small
• codestral-latest — Codestral
• open-mistral-nemo — Mistral Nemo

DeepSeek

• deepseek-chat — DeepSeek Chat V3 (default)
• deepseek-reasoner — DeepSeek Reasoner R1

Adding a New Provider

1. Create app/LLM/svcNewProvider.php extending LLMProvider
2. Implement: __construct(array $config), sendPrompt(), streamPrompt(), getModels(), testConnection()
3. Register in LLMManager.php:
   • Add to $providers array: 'newprovider' => svcNewProvider::class
   • Add to $metaKeys array: 'newprovider' => 'llmNewProvider'
4. Add JSON config row to sp_orgs_meta with keyval = 'llmNewProvider'