A retrieval-augmented assistant answers from your own documents, products and rules: text is chunked and embedded, the relevant chunks are retrieved for each question, and the model answers only from them, with citations. On .NET 10 the whole thing fits in the existing PostgreSQL database with pgvector. The model is the cheapest line item; construction, evaluation and maintenance are the real cost, and for restricted markets the first decision is where the data may go.

What an assistant does, and what it must not

A language model on its own has general knowledge and knows nothing about your company. Asked "how long is the warranty on product X", it either declines or invents a plausible number. The second is worse. An assistant worth its cost has three properties:

  • It answers from your data and shows the source of every answer.
  • When the answer is not in the data, it says so and hands over to a person.
  • It does something: books a slot, reports an order status, files a request. Otherwise a good FAQ page does the same job for less.

RAG, briefly and precisely

  1. Chunking. Documentation, product pages, policies and ticket history are split into chunks of 200 to 500 words, each carrying its heading and source. Chunk boundaries matter more than most teams expect; splitting a policy mid-clause produces confident half-answers.
  2. Embedding. Each chunk becomes a vector. For multilingual content, open-weight models such as bge-m3 perform well and keep the data on your infrastructure.
  3. Storage. Vectors live in PostgreSQL with the pgvector extension and an HNSW index. No second database; the same instance the application already uses, with the same backups and the same transactions.
  4. Retrieval. The question is embedded and the nearest chunks, usually 5 to 10, are found with a combination of vector similarity and keyword search. The combination is not optional: pure vector search loses product codes, SKUs and proper names.
  5. Generation. The chunks go to the model with an explicit instruction: answer only from this material, cite the source, say when it is absent.
  6. Tools, optionally. For "where is my order", the model calls a defined function that reads from your system with the caller's identity. It never has database access itself.

Three uses that pay for themselves, and two that do not

Worth it:

  1. Support over documentation and policies. Software with a long manual, or services with complex rules. Repetitive questions leave the support queue and every answer is sourced.
  2. Product selection. A catalogue of hundreds of technical products where the buyer cannot tell which fits. Three questions lead to three products with links. In Daroobin, an online pharmacy, this use carries a hard constraint: any medical statement comes from the official product label, never from the model's knowledge.
  3. Pre-sales and booking. A clinic or service business: first questions answered, the appointment written into the system. The effect is measurable in bookings made outside working hours.

Not worth it:

  • A corporate site with ten FAQs; one good page does the job and ranks in search.
  • "Because everyone has one." An assistant that does nothing is switched off after the first month while its monthly cost continues.

The architecture we standardised

This is the reference design for our custom software work, because it sits on the same stack as the applications themselves:

  • Entry: a widget on the site or a messaging channel, with rate limiting and session identification.
  • Retrieval layer: pgvector in the application's PostgreSQL; embeddings refreshed automatically when content changes, through the same outbox mechanism we use for payments (the transactional outbox explains why that pattern and not a cron job).
  • Model layer: swappable behind one interface. Cloud model or open-weight model on your own hardware; the choice must not change the architecture.
  • Guardrails: an allow-list of subjects, PII filtering on input and output, tools with least privilege, human confirmation for any irreversible action.
  • Logging and evaluation: every conversation is stored with the chunks that were retrieved, so a wrong answer can be traced to bad retrieval or to the model.
  • Handover: an always-visible path to a person, and automatic handover when the model is unsure, creating a ticket in the support system. In Idepoo CRM the handover is the ticket.

Constraints in restricted markets

We build for Iran and for European clients, and the constraints differ:

  • API access. Major cloud model providers do not officially serve some markets. Implementations there go through intermediaries: higher cost, dependence on a third party, and outage risk. If the assistant is part of a core business process, that risk belongs in the design.
  • Data residency. Every user message and every retrieved chunk reaches the model's server. For medical, financial or personal data that is often unacceptable, in Iran for practical reasons and in the EU under GDPR. The answer is an open-weight model on local hardware. Mid-size multilingual open models in 2026 are weaker than the large cloud models but adequate for answering from supplied text.
  • Hardware cost. A server with a single mid-range GPU runs a small-to-medium model and is rented monthly; at low volume it is cheaper than a cloud intermediary.
  • Latency. A round trip to a distant provider adds hundreds of milliseconds per answer. Streaming the response hides most of it.

The real cost model

Back-of-envelope figures for a shop with 1,000 conversations a month:

  • Roughly 8 messages per conversation; each message with retrieved chunks is about 2,000 input tokens and 300 output tokens. That is about 16 million input and 2.5 million output tokens a month.
  • At 2026 mid-tier cloud pricing that is a few tens of dollars a month; through an intermediary typically 1.5 to 3 times that. With a local model the server cost is fixed regardless of volume.
  • Construction for a RAG assistant with one operational tool (order status, say) runs 3 to 6 weeks in our projects, depending on the state of your documentation.
  • The hidden cost is content preparation. An assistant over messy documentation gives messy answers. One to two weeks usually go into organising the source material, and that is the best-spent time in the project.
  • Maintenance: content updates, a monthly review of handed-over conversations, and a model swap when a better one appears.

How we evaluate before release

  1. A golden set. Fifty real questions from support history, each with the correct answer and source. Every release runs the whole set and records the pass rate.
  2. Unanswerable questions. Twenty questions whose answer is not in the data. The assistant must decline; any fabricated answer is a critical defect.
  3. Dangerous questions. Requests for other people's data, medical or legal advice outside scope, instructions to ignore the rules. All must be refused.
  4. Sampled human review. Thirty real conversations read and scored every week. Our targets: above 90 percent on the golden set and zero fabrications on the unanswerable set.

Buy an off-the-shelf assistant or build one?

For question answering over a few pages, off-the-shelf is enough. If it must connect to your systems (orders, bookings, accounts) or the data must stay in your jurisdiction, build. Most serious projects are the second kind.

Does it replace the support team?

The repetitive part of their work, yes. The team does not shrink; it spends its time on real problems. An assistant without handover makes customers angry.

Which embedding and language models?

Retrieval quality, meaning the embedding model and text normalisation, matters more in practice than the language model. We start with a multilingual open embedding model and a swappable generation model, and change either when the golden set says so.

Summary

A RAG assistant is a software project, not a plugin: organised data, correct retrieval, a swappable model, guardrails and evaluation before release. The model's price is the smallest part. If you have a use that justifies it, talk to us about custom software; the first thing we build is the 50-question golden set.

Sources