The official TypeScript/JavaScript SDK for the Graphlit Platform - build AI-powered applications with knowledge retrieval in minutes.
Graphlit is a cloud platform that handles the complex parts of building AI applications:
- Ingest any content - PDFs, websites, audio, video, and more
- Chat with your data - Using RAG (Retrieval-Augmented Generation)
- Extract insights - Summaries, entities, and metadata
- Build knowledge graphs - Automatically connect related information
- Real-time streaming - Watch AI responses appear word-by-word across 9 different providers
- Tool calling - Let AI execute functions and retrieve data
- Extended provider support - Native streaming integration with OpenAI, Anthropic, Google, Groq, Cerebras, Cohere, Mistral, AWS Bedrock, and Deepseek
- Better performance - Optimized streaming with provider-specific SDKs
- Network resilience - Automatic retry logic for transient failures
- Quick Start
- Installation
- Setting Up
- Network Resilience
- Streaming Provider Support
- Basic Examples
- Common Use Cases
- Advanced Agent Features
- Advanced Workflows
- API Reference
- Testing & Examples
- Support
Get started in 2 minutes:
# Install the SDK
npm install graphlit-client
# Set your credentials (get free account at https://portal.graphlit.dev)
export GRAPHLIT_ORGANIZATION_ID=your_org_id
export GRAPHLIT_ENVIRONMENT_ID=your_env_id
export GRAPHLIT_JWT_SECRET=your_secret
import { Graphlit, Types } from "graphlit-client";
const client = new Graphlit(); // Uses env vars: GRAPHLIT_ORGANIZATION_ID, GRAPHLIT_ENVIRONMENT_ID, GRAPHLIT_JWT_SECRET
// First, create a specification (or use your project default)
const spec = await client.createSpecification({
name: "Assistant",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
},
});
// Start chatting with AI
await client.streamAgent(
"Tell me a joke",
(event) => {
if (event.type === "message_update") {
console.log(event.message.message);
}
},
undefined, // conversationId (optional)
{ id: spec.createSpecification.id }, // specification
);
npm install graphlit-client
Install the LLM SDK for streaming responses:
# For OpenAI streaming
npm install openai
# For Anthropic streaming
npm install @anthropic-ai/sdk
# For Google streaming
npm install @google/generative-ai
# For Groq streaming (OpenAI-compatible)
npm install groq-sdk
# For Cerebras streaming (OpenAI-compatible)
npm install openai
# For Cohere streaming
npm install cohere-ai
# For Mistral streaming
npm install @mistralai/mistralai
# For AWS Bedrock streaming (Claude models)
npm install @aws-sdk/client-bedrock-runtime
# For Deepseek streaming (OpenAI-compatible)
npm install openai
Create a .env
file in your project:
GRAPHLIT_ORGANIZATION_ID=your_org_id
GRAPHLIT_ENVIRONMENT_ID=your_env_id
GRAPHLIT_JWT_SECRET=your_secret
# Optional: For streaming with specific providers
OPENAI_API_KEY=your_key
ANTHROPIC_API_KEY=your_key
GOOGLE_API_KEY=your_key
# Additional streaming providers
GROQ_API_KEY=your_key # For Groq models (Llama, Mixtral)
CEREBRAS_API_KEY=your_key # For Cerebras models
COHERE_API_KEY=your_key # For Cohere Command models
MISTRAL_API_KEY=your_key # For Mistral models
DEEPSEEK_API_KEY=your_key # For Deepseek models
# For AWS Bedrock streaming (requires AWS credentials)
AWS_REGION=us-east-2
AWS_ACCESS_KEY_ID=your_key
AWS_SECRET_ACCESS_KEY=your_secret
The SDK now includes automatic retry logic for network errors and transient failures:
By default, the client will automatically retry on these status codes:
429
- Too Many Requests502
- Bad Gateway503
- Service Unavailable504
- Gateway Timeout
const client = new Graphlit(); // Uses default retry configuration
Configure retry behavior to match your needs:
const client = new Graphlit({
organizationId: "your_org_id",
environmentId: "your_env_id",
jwtSecret: "your_secret",
retryConfig: {
maxAttempts: 10, // Maximum retry attempts (default: 5)
initialDelay: 500, // Initial delay in ms (default: 300)
maxDelay: 60000, // Maximum delay in ms (default: 30000)
jitter: true, // Add randomness to delays (default: true)
retryableStatusCodes: [429, 500, 502, 503, 504], // Custom status codes
onRetry: (attempt, error, operation) => {
console.log(`Retry attempt ${attempt} for ${operation.operationName}`);
console.log(`Error: ${error.message}`);
},
},
});
Change retry behavior on the fly:
// Start with default configuration
const client = new Graphlit();
// Later, update for more aggressive retries
client.setRetryConfig({
maxAttempts: 20,
initialDelay: 100,
retryableStatusCodes: [429, 500, 502, 503, 504, 521, 522, 524],
});
For testing or specific scenarios:
const client = new Graphlit({
organizationId: "your_org_id",
environmentId: "your_env_id",
jwtSecret: "your_secret",
retryConfig: {
maxAttempts: 1, // No retries
},
});
The Graphlit SDK supports real-time streaming responses from 9 different LLM providers. Each provider requires its specific SDK and API key:
Provider | Models | SDK Required | API Key |
---|---|---|---|
OpenAI | GPT-4, GPT-4o, GPT-4.1, O1, O3, O4 | openai |
OPENAI_API_KEY |
Anthropic | Claude 3, Claude 3.5, Claude 3.7, Claude 4 | @anthropic-ai/sdk |
ANTHROPIC_API_KEY |
Gemini 1.5, Gemini 2.0, Gemini 2.5 | @google/generative-ai |
GOOGLE_API_KEY |
|
Groq | Llama 4, Llama 3.3, Mixtral, Deepseek R1 | groq-sdk |
GROQ_API_KEY |
Cerebras | Llama 3.3, Llama 3.1 | openai |
CEREBRAS_API_KEY |
Cohere | Command R+, Command R, Command R7B, Command A | cohere-ai |
COHERE_API_KEY |
Mistral | Mistral Large, Medium, Small, Nemo, Pixtral | @mistralai/mistralai |
MISTRAL_API_KEY |
AWS Bedrock | Nova Premier/Pro, Claude 3.7, Llama 4 | @aws-sdk/client-bedrock-runtime |
AWS credentials |
Deepseek | Deepseek Chat, Deepseek Reasoner | openai |
DEEPSEEK_API_KEY |
Each provider requires both the SDK installation and proper client setup:
import { Graphlit } from "graphlit-client";
const client = new Graphlit();
// Example: Set up multiple streaming providers
if (process.env.OPENAI_API_KEY) {
const { OpenAI } = await import("openai");
client.setOpenAIClient(new OpenAI());
}
if (process.env.COHERE_API_KEY) {
const { CohereClient } = await import("cohere-ai");
client.setCohereClient(
new CohereClient({ token: process.env.COHERE_API_KEY }),
);
}
if (process.env.GROQ_API_KEY) {
const { Groq } = await import("groq-sdk");
client.setGroqClient(new Groq({ apiKey: process.env.GROQ_API_KEY }));
}
// Then create specifications for any provider
const spec = await client.createSpecification({
name: "Multi-Provider Assistant",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.Cohere, // or any supported provider
cohere: {
model: Types.CohereModels.CommandRPlus,
temperature: 0.7,
},
});
- OpenAI-Compatible: Groq, Cerebras, and Deepseek use OpenAI-compatible APIs
- AWS Bedrock: Requires AWS credentials and uses the Converse API for streaming
- Cohere: Supports both chat and tool calling with Command models
- Google: Includes advanced multimodal capabilities with Gemini models
- Mistral: Supports both text and vision models (Pixtral)
Simple conversation with streaming responses:
import { Graphlit, Types } from "graphlit-client";
const client = new Graphlit(); // Uses env vars: GRAPHLIT_ORGANIZATION_ID, GRAPHLIT_ENVIRONMENT_ID, GRAPHLIT_JWT_SECRET
// Create a specification for the AI model
const spec = await client.createSpecification({
name: "Assistant",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
temperature: 0.7,
},
});
// Chat with streaming
await client.streamAgent(
"What can you help me with?",
(event) => {
if (event.type === "message_update") {
// Print the AI's response as it streams
process.stdout.write(event.message.message);
}
},
undefined, // conversationId
{ id: spec.createSpecification.id }, // specification
);
Upload a PDF and ask questions about it:
import { Graphlit, Types } from "graphlit-client";
const client = new Graphlit(); // Uses env vars: GRAPHLIT_ORGANIZATION_ID, GRAPHLIT_ENVIRONMENT_ID, GRAPHLIT_JWT_SECRET
// Create a specification
const spec = await client.createSpecification({
name: "Document Q&A",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
},
});
// Upload a PDF synchronously to ensure it's ready
const content = await client.ingestUri(
"https://arxiv.org/pdf/1706.03762.pdf", // Attention Is All You Need paper
"AI Research Paper", // name
undefined, // id
true, // isSynchronous - waits for processing
);
console.log(`β
Uploaded: ${content.ingestUri.id}`);
// Wait a moment for content to be fully indexed
await new Promise((resolve) => setTimeout(resolve, 5000));
// Create a conversation that filters to this specific content
const conversation = await client.createConversation({
filter: { contents: [{ id: content.ingestUri.id }] },
});
// Ask questions about the PDF
await client.streamAgent(
"What are the key innovations in this paper?",
(event) => {
if (event.type === "message_update") {
console.log(event.message.message);
}
},
conversation.createConversation.id, // conversationId with content filter
{ id: spec.createSpecification.id }, // specification
);
Extract content from websites:
// Scrape a website (waits for processing to complete)
const webpage = await client.ingestUri(
"https://en.wikipedia.org/wiki/Artificial_intelligence", // uri
"AI Wikipedia Page", // name
undefined, // id
true, // isSynchronous
);
// Wait for content to be indexed
await new Promise((resolve) => setTimeout(resolve, 5000));
// Create a conversation filtered to this content
const conversation = await client.createConversation({
filter: { contents: [{ id: webpage.ingestUri.id }] },
});
// Ask about the specific content
const response = await client.promptAgent(
"Summarize the key points about AI from this Wikipedia page",
conversation.createConversation.id, // conversationId with filter
{ id: spec.createSpecification.id }, // specification (create one as shown above)
);
console.log(response.message);
Compare responses from different LLM providers:
import { Graphlit, Types } from "graphlit-client";
const client = new Graphlit();
// Set up multiple providers
if (process.env.OPENAI_API_KEY) {
const { OpenAI } = await import("openai");
client.setOpenAIClient(new OpenAI());
}
if (process.env.COHERE_API_KEY) {
const { CohereClient } = await import("cohere-ai");
client.setCohereClient(
new CohereClient({ token: process.env.COHERE_API_KEY }),
);
}
if (process.env.GROQ_API_KEY) {
const { Groq } = await import("groq-sdk");
client.setGroqClient(new Groq({ apiKey: process.env.GROQ_API_KEY }));
}
// Create specifications for different providers
const providers = [
{
name: "OpenAI GPT-4o",
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: { model: Types.OpenAiModels.Gpt4O_128K },
},
{
name: "Cohere Command R+",
serviceType: Types.ModelServiceTypes.Cohere,
cohere: { model: Types.CohereModels.CommandRPlus },
},
{
name: "Groq Llama",
serviceType: Types.ModelServiceTypes.Groq,
groq: { model: Types.GroqModels.Llama_3_3_70B },
},
];
// Compare responses
for (const provider of providers) {
console.log(`\nπ€ ${provider.name}:`);
const spec = await client.createSpecification({
...provider,
type: Types.SpecificationTypes.Completion,
});
await client.streamAgent(
"Explain quantum computing in simple terms",
(event) => {
if (event.type === "message_update") {
process.stdout.write(event.message.message);
}
},
undefined,
{ id: spec.createSpecification.id },
);
}
Let AI call functions to get real-time data:
import { Graphlit, Types } from "graphlit-client";
const client = new Graphlit(); // Uses env vars: GRAPHLIT_ORGANIZATION_ID, GRAPHLIT_ENVIRONMENT_ID, GRAPHLIT_JWT_SECRET
// Define a weather tool
const weatherTool: Types.ToolDefinitionInput = {
name: "get_weather",
description: "Get current weather for a city",
schema: JSON.stringify({
type: "object",
properties: {
city: { type: "string", description: "City name" },
},
required: ["city"],
}),
};
// Tool implementation
const toolHandlers = {
get_weather: async (args: { city: string }) => {
// Call your weather API here
return {
city: args.city,
temperature: 72,
condition: "sunny",
};
},
};
// Create a specification for tool calling
const spec = await client.createSpecification({
name: "Weather Assistant",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
},
});
// Chat with tools
await client.streamAgent(
"What's the weather in San Francisco?",
(event) => {
if (event.type === "tool_update" && event.status === "completed") {
console.log(`π§ Called ${event.toolCall.name}`);
} else if (event.type === "message_update") {
console.log(event.message.message);
}
},
undefined, // conversationId
{ id: spec.createSpecification.id }, // specification
[weatherTool], // tools
toolHandlers, // handlers
);
Create an AI that answers questions from your documents:
import { Graphlit, Types } from "graphlit-client";
class KnowledgeAssistant {
private client: Graphlit;
private conversationId?: string;
private specificationId?: string;
private contentIds: string[] = [];
constructor() {
this.client = new Graphlit(); // Uses env vars: GRAPHLIT_ORGANIZATION_ID, GRAPHLIT_ENVIRONMENT_ID, GRAPHLIT_JWT_SECRET
}
async initialize() {
// Create a specification for the assistant
const spec = await this.client.createSpecification({
name: "Knowledge Assistant",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
temperature: 0.7,
},
});
this.specificationId = spec.createSpecification?.id;
}
async uploadDocuments(urls: string[]) {
console.log("π Uploading documents...");
for (const url of urls) {
const content = await this.client.ingestUri(
url, // uri
url.split("/").pop() || "Document", // name
undefined, // id
true, // isSynchronous - wait for processing
);
this.contentIds.push(content.ingestUri.id);
}
console.log("β
Documents uploaded!");
// Wait for content to be indexed
await new Promise((resolve) => setTimeout(resolve, 5000));
}
async ask(question: string) {
// Create conversation with content filter if not exists
if (!this.conversationId && this.contentIds.length > 0) {
const conversation = await this.client.createConversation({
filter: { contents: this.contentIds.map((id) => ({ id })) },
});
this.conversationId = conversation.createConversation?.id;
}
await this.client.streamAgent(
question,
(event) => {
if (event.type === "conversation_started" && !this.conversationId) {
this.conversationId = event.conversationId;
} else if (event.type === "message_update") {
process.stdout.write(event.message.message);
}
},
this.conversationId, // Maintains conversation context
{ id: this.specificationId! }, // specification
);
}
}
// Usage
const assistant = new KnowledgeAssistant();
await assistant.initialize();
// Upload your documents
await assistant.uploadDocuments([
"https://arxiv.org/pdf/2103.15348.pdf",
"https://arxiv.org/pdf/1706.03762.pdf",
]);
// Ask questions
await assistant.ask("What are these papers about?");
await assistant.ask("How do they relate to each other?");
Extract specific information from uploaded content:
// Upload a document synchronously
const document = await client.ingestUri(
"https://example.com/document.pdf", // uri
"Document #12345", // name
undefined, // id
true, // isSynchronous
);
// Wait for content to be indexed
await new Promise((resolve) => setTimeout(resolve, 5000));
// Extract specific data
const extraction = await client.extractContents(
"Extract the key information from this document",
undefined, // tools
undefined, // specification
{ contents: [{ id: document.ingestUri.id }] }, // filter
);
console.log("Extracted data:", extraction.extractContents);
Create summaries across multiple files:
// Upload multiple documents synchronously
const ids: string[] = [];
for (const url of documentUrls) {
const content = await client.ingestUri(
url, // uri
url.split("/").pop() || "Document", // name
undefined, // id
true, // isSynchronous
);
ids.push(content.ingestUri.id);
}
// Generate a summary across all documents
const summary = await client.summarizeContents(
[
{
type: Types.SummarizationTypes.Custom,
prompt: "Create an executive summary of these documents",
},
], // summarizations
{ contents: ids.map((id) => ({ id })) }, // filter
);
console.log("Summary:", summary.summarizeContents);
// Option 1: Synchronous processing (simpler)
const content = await client.ingestUri(
"https://example.com/large-document.pdf", // uri
undefined, // name
undefined, // id
true, // isSynchronous
);
console.log("β
Content ready!");
// Option 2: Asynchronous processing (for large files)
const content = await client.ingestUri(
"https://example.com/very-large-video.mp4", // uri
// isSynchronous defaults to false
);
// Check status later
let isReady = false;
while (!isReady) {
const status = await client.isContentDone(content.ingestUri.id);
isReady = status.isContentDone?.result || false;
if (!isReady) {
console.log("β³ Still processing...");
await new Promise((resolve) => setTimeout(resolve, 2000));
}
}
console.log("β
Content ready!");
Control what content the agent can access during conversations:
// Example 1: Chat with specific documents only
const result = await client.promptAgent(
"What are the main points in these documents?",
undefined, // conversationId - will create new
{ id: specificationId },
undefined, // tools
undefined, // toolHandlers
undefined, // options
undefined, // mimeType
undefined, // data
{
// Only allow retrieval from specific content
contents: [{ id: "content-id-1" }, { id: "content-id-2" }],
},
);
// Example 2: Streaming with content filter
await client.streamAgent(
"Explain the technical details",
(event) => {
if (event.type === "message_update") {
process.stdout.write(event.message.message);
}
},
undefined, // conversationId
{ id: specificationId },
undefined, // tools
undefined, // toolHandlers
undefined, // options
undefined, // mimeType
undefined, // data
{
// Filter by collection
collections: [{ id: "technical-docs-collection" }],
},
);
Force specific content into the LLM context without retrieval:
// Example: Chat with a specific file always in context
const fileContent = await client.getContent("file-content-id");
await client.streamAgent(
"What patterns do you see in this code?",
(event) => {
if (event.type === "message_update") {
process.stdout.write(event.message.message);
}
},
undefined, // conversationId
{ id: specificationId },
undefined, // tools
undefined, // toolHandlers
undefined, // options
undefined, // mimeType
undefined, // data
undefined, // contentFilter
{
// Force this content into context
contents: [{ id: fileContent.content.id }],
},
);
Use both filters for precise control:
// Chat about specific code with documentation available
await client.promptAgent(
"How does this code implement the algorithm described in the docs?",
undefined,
{ id: specificationId },
undefined,
undefined,
undefined,
undefined,
undefined,
{
// Can retrieve from documentation
collections: [{ id: "algorithm-docs" }],
},
{
// Always include the specific code file
contents: [{ id: "implementation-file-id" }],
},
);
Workflows automatically process content when ingested:
import { Graphlit, Types } from "graphlit-client";
const client = new Graphlit(); // Uses env vars: GRAPHLIT_ORGANIZATION_ID, GRAPHLIT_ENVIRONMENT_ID, GRAPHLIT_JWT_SECRET
// Create specifications for AI models
const summarizationSpec = await client.createSpecification({
name: "Summarizer",
type: Types.SpecificationTypes.Summarization,
serviceType: Types.ModelServiceTypes.OpenAi,
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
},
});
// Create a workflow that summarizes all content
const workflow = await client.createWorkflow({
name: "Document Intelligence",
preparation: {
summarizations: [
{
type: Types.SummarizationTypes.Summary,
specification: { id: summarizationSpec.createSpecification.id },
},
],
},
});
// Set workflow as default for project
await client.updateProject({
workflow: { id: workflow.createWorkflow.id },
});
// Now all content will be automatically summarized
const content = await client.ingestUri(
"https://example.com/report.pdf", // uri
);
Specifications configure how AI models behave:
import { Graphlit, Types } from "graphlit-client";
// Create a conversational AI specification
const conversationSpec = await client.createSpecification({
name: "Customer Support AI",
type: Types.SpecificationTypes.Completion,
serviceType: Types.ModelServiceTypes.OpenAi,
systemPrompt: "You are a helpful customer support assistant.",
openAI: {
model: Types.OpenAiModels.Gpt4O_128K,
temperature: 0.7,
completionTokenLimit: 2000,
},
});
// Use the specification in conversations
await client.streamAgent(
"How do I reset my password?",
(event) => {
if (event.type === "message_update") {
console.log(event.message.message);
}
},
undefined,
{ id: conversationSpec.createSpecification.id },
);
const client = new Graphlit(organizationId?, environmentId?, jwtSecret?);
ingestUri(uri, name?, id?, isSynchronous?, ...)
- Ingest content from URLingestText(text, name?, textType?, ...)
- Ingest text content directlyqueryContents(filter?)
- Search and query contentgetContent(id)
- Get content by IDdeleteContent(id)
- Delete contentextractContents(prompt, tools, specification?, filter?)
- Extract data from contentsummarizeContents(summarizations, filter?)
- Summarize contentisContentDone(id)
- Check if content processing is complete
createConversation(input?)
- Create a new conversationstreamAgent(prompt, handler, ...)
- Stream AI responsespromptAgent(prompt, ...)
- Get AI response without streamingdeleteConversation(id)
- Delete conversation
createSpecification(input)
- Create AI model configurationquerySpecifications(filter?)
- List specificationsdeleteSpecification(id)
- Delete specification
createWorkflow(input)
- Create content processing workflowqueryWorkflows(filter?)
- List workflowsupdateProject(input)
- Update project settings
type AgentStreamEvent =
| { type: "conversation_started"; conversationId: string }
| { type: "message_update"; message: { message: string } }
| { type: "tool_update"; toolCall: any; status: string }
| { type: "conversation_completed"; message: { message: string } }
| { type: "error"; error: { message: string; recoverable: boolean } };
All examples in this README are tested and verified. See test/readme-simple.test.ts
for runnable versions of these examples.
To run the examples yourself:
# Clone the repository
git clone https://github.com/graphlit/graphlit-client-typescript.git
cd graphlit-client-typescript
# Install dependencies
npm install
# Set up your environment variables
cp .env.example .env
# Edit .env with your Graphlit credentials
# Run the examples
npm test test/readme-simple.test.ts
- π Documentation: https://docs.graphlit.dev/
- π¬ Discord Community: Join our Discord
- π Issues: GitHub Issues
- π§ Email: [email protected]
MIT License - see LICENSE file for details.