How to build an AI agent with LangChain.js and NestJS
3 July 2026 · SOM-OS
There are two ways to build an AI agent for businesses in Node.js. The first: paste the OpenAI API into an Express controller, without caching, queues or tools. It works in a demo and explodes in production when 200 concurrent requests arrive. The second: real architecture with LangChain.js on NestJS, BullMQ queues, Redis memory and typed tools. This tutorial covers the second.
If you are unsure why AI needs architecture before code, read AI without context is an expensive parrot and come back. Here, we assume you already know why.
The agent’s architecture
The workflow is this: a user sends a message to the NestJS endpoint. NestJS queues the job in BullMQ (Redis). A worker processes the queue, calls the LangChain agent with the available tools, persists the conversation in PostgreSQL and returns the response through WebSocket or polling. The LLM is stateless — context lives in your database, not the model’s memory.
Client (Nuxt/React)
│ POST /agents/:id/messages
▼
┌──────────────────────┐
│ NestJS Controller │ validation, auth, rate limit
└──────────┬───────────┘
│ enqueue(job)
▼
┌──────────────────────┐
│ BullMQ Queue (Redis) │ queues, retries, priority
└──────────┬───────────┘
│ process(job)
▼
┌──────────────────────┐
│ Agent Worker │ LangChain agent + tools
│ - chat history (PG) │
│ - tools (DB, API) │
│ - model (OpenAI/etc) │
└──────────┬───────────┘
│ ws.emit / db.save
▼
Client receives responseEvery piece has a reason. The queue prevents 200 simultaneous users from bringing down the service. The worker separates LLM latency from the HTTP request. Persisting history in PostgreSQL allows auditing every conversation and resuming contexts.
Step 1: install dependencies
npm i @nestjs/common @nestjs/core @nestjs/platform-fastify \
@langchain/core @langchain/openai @langchain/community \
langchain bullmq ioredis @prisma/clientWe use the Fastify adapter in NestJS for performance. If you come from Express, migration is straightforward; we covered why in NestJS vs Express.
Step 2: the agent module
The module groups the agent service, queue and worker. I register the worker as a provider so NestJS’s lifecycle manages it:
import { Module } from '@nestjs/common';
import { AgentService } from './agent.service';
import { AgentQueue } from './agent.queue';
import { AgentWorker } from './agent.worker';
@Module({
providers: [AgentService, AgentQueue, AgentWorker],
exports: [AgentService],
})
export class AgentModule {}Step 3: the controller
import { Body, Controller, Post, UseGuards } from '@nestjs/common';
import { AgentService } from './agent.service';
import { JwtGuard } from '../auth/jwt.guard';
class SendMessageDto {
conversationId: string;
content: string;
}
@Controller('agents')
@UseGuards(JwtGuard)
export class AgentsController {
constructor(private readonly agent: AgentService) {}
@Post('messages')
async send(@Body() dto: SendMessageDto) {
const jobId = await this.agent.enqueue(dto);
return { jobId, status: 'queued' };
}
}The controller does not call the LLM. It only queues. The response arrives asynchronously. This is the difference between a demo agent and a production agent: the user does not wait 30 seconds with the request open.
Step 4: the BullMQ queue
import { Injectable, OnModuleInit } from '@nestjs/common';
import { Queue } from 'bullmq';
import IORedis from 'ioredis';
@Injectable()
export class AgentQueue implements OnModuleInit {
private queue: Queue;
onModuleInit() {
const connection = new IORedis(process.env.REDIS_URL, {
maxRetriesPerRequest: null,
});
this.queue = new Queue('agent-jobs', { connection });
}
async add(data: SendMessageDto) {
return this.queue.add('run-agent', data, {
attempts: 3,
backoff: { type: 'exponential', delay: 2000 },
});
}
}Three retries with exponential backoff. If the LLM fails because of a rate limit, BullMQ retries without losing the message. Without a queue, a provider timeout costs you an entire conversation.
Step 5: the LangChain agent
Now the worker. The logic lives here: it retrieves conversation history, configures tools, runs the agent and saves the result.
import { Injectable, OnModuleInit, OnModuleDestroy } from '@nestjs/common';
import { Worker, Job } from 'bullmq';
import { ChatOpenAI } from '@langchain/openai';
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents';
import { ChatPromptTemplate, MessagesPlaceholder } from '@langchain/core/prompts';
import { searchCatalogTool } from './tools/search-catalog.tool';
import { PrismaService } from '../prisma/prisma.service';
@Injectable()
export class AgentWorker implements OnModuleInit, OnModuleDestroy {
private worker: Worker;
constructor(private readonly prisma: PrismaService) {}
onModuleInit() {
this.worker = new Worker(
'agent-jobs',
async (job: Job) => this.handle(job),
{ connection: { url: process.env.REDIS_URL } },
);
}
onModuleDestroy() { return this.worker.close(); }
private async handle(job: Job) {
const { conversationId, content } = job.data;
// 1. History from PostgreSQL
const history = await this.prisma.message.findMany({
where: { conversationId },
orderBy: { createdAt: 'asc' },
take: 20,
});
const model = new ChatOpenAI({
model: 'gpt-4o-mini',
temperature: 0.2,
});
// 2. Tools with access to your system
const tools = [searchCatalogTool(this.prisma)];
// 3. Prompt with brand instructions
const prompt = ChatPromptTemplate.fromMessages([
['system',
'You are a sales assistant for SOM-OS. ' +
'Reply in English and address the user directly. ' +
'Use the tools to search the catalogue before replying. ' +
'If you do not know, say so. Do not invent an answer.'],
new MessagesPlaceholder('history'),
['human', '{input}'],
new MessagesPlaceholder('agent_scratchpad'),
]);
const agent = await createToolCallingAgent({
llm: model,
tools,
prompt,
});
const executor = new AgentExecutor({ agent, tools });
// 4. Execute
const result = await executor.invoke({
input: content,
history: history.map((m) => [m.role, m.content]),
});
// 5. Persist
await this.prisma.message.createMany({
data: [
{ conversationId, role: 'human', content },
{ conversationId, role: 'ai', content: result.output },
],
});
return { conversationId, output: result.output };
}
}Five steps: history, model, tools, prompt, persistence. The agent has context from the last 20 messages, tools to access the real catalogue, brand instructions and saves every exchange for auditing.
Step 6: define a tool
A tool is a typed function the agent decides when to use. Here is one that searches for products in the catalogue:
import { tool } from '@langchain/core/tools';
import { z } from 'zod';
export const searchCatalogTool = (prisma) =>
tool(
async ({ query }) => {
const products = await prisma.product.findMany({
where: { name: { contains: query, mode: 'insensitive' } },
take: 5,
});
return JSON.stringify(products);
},
{
name: 'search_catalog',
description:
'Searches the catalogue for products by name. ' +
'Returns up to 5 results with id, name, price and stock.',
schema: z.object({
query: z.string().describe('search term'),
}),
},
);The Zod schema tells the model which arguments the tool accepts. LangChain converts it to function calling automatically. The agent decides whether to call it based on the user’s context.
Step 7: deployment
The minimum production deployment: NestJS in one container, Redis in another, managed PostgreSQL (Supabase, Neon or RDS). Environment variables: REDIS_URL, DATABASE_URL, OPENAI_API_KEY. A basic Dockerfile:
FROM node:22-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm prune --production
FROM node:22-alpine
WORKDIR /app
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
ENV NODE_ENV=production
EXPOSE 3000
CMD ["node", "dist/main.js"]Deploy to Fly.io, Railway or Coolify on your VPS. What matters: Redis and the worker may live in the same container or separately, but the queue is the synchronisation point. Do not put the worker on serverless — long jobs do not survive cold starts.
What the quick tutorial does not tell you
Three things we learned in production that no “build an agent in 10 minutes” tutorial mentions:
- LLM rate limits. OpenAI cuts you off at 500 RPM in tier 1. If your agent uses 2 tools per message, a single conversation consumes 3 requests. Without a queue and backoff, you fail on launch day.
- Context window. 20 messages take ~4,000 tokens. At 50, you exceed the inexpensive model’s limit. You need automatic summarisation or intelligent history truncation — not loading everything every time.
- Cost. An agent with gpt-4o-mini and 2 tools per message costs ~€0.002 per interaction. It looks small until you have 1,000 users with 10 daily conversations. That is €20/day in tokens alone. Measure before scaling.
A production AI agent is not an endpoint. It is a system: queue, persistence, tools, rate limits and controlled costs. Without that, you have an expensive parrot that crashes on launch day.
Why NestJS rather than Express here
You could do this in Express. But an agent with tools, queues, websockets and persistence needs modules, dependency injection and structure. Express lets you improvise — NestJS makes you organise. The full comparison is in NestJS vs Express, but the summary is: if your AI will grow beyond a demo, NestJS saves a refactor.
To see an agent with real context in production, look at Sando Capital: an agent qualifying financial leads with a loaded catalogue and fit rules. If your business does not yet have the connected data an agent needs, start with the Chaos Map before LangChain.