When a client asks us for an AI assistant inside their website or app, the demo is never the hard part. A prototype that answers ten questions well can be built in a couple of days. What decides whether the client is still happy six months later is two boring things: what each answer costs, and what happens on the day the model provider has a bad afternoon.
This is how we plan for both before we write the first feature. It applies whether the assistant sits in a Laravel back office, a WordPress site, a React dashboard or a mobile app.
Start with a cost per answer, not a monthly guess
Most providers charge by tokens in and tokens out. Clients rarely think in tokens, so we translate it into something they can budget: cost per conversation, and conversations per month.
To get there we do a small exercise in the first week:
- Collect 30 to 50 real questions from the client's support inbox, chat logs or sales team. Not invented ones.
- Run them through the draft assistant and log input tokens, output tokens and the model used for each answer.
- Multiply by the client's realistic monthly volume, then by three, because usage always grows once people trust the thing.
The number that comes out is rarely the one the client expected. In our experience the surprise is usually on the input side. A support assistant that pastes five long document chunks into every request, plus the whole chat history, pays for that context on every single turn. Output is short; input is where the money goes.
Check the current price list of whichever provider you use before quoting, because prices and model names change often. We always put the date of the estimate in the project document so nobody treats it as permanent.
The cheap levers, in the order we pull them
Send less context. Retrieve fewer, better chunks. Three well-chosen passages usually beat eight loosely related ones, both on cost and on answer quality. If the assistant answers from client documents, this is mostly a retrieval-quality problem, not a model problem.
Trim the history. After a few turns, summarize older messages or drop them. Most client questions do not depend on what was said twenty messages ago.
Use a smaller model for easy work. Classifying an incoming message, extracting an order number, deciding whether a question is in scope: these do not need the largest model. We route those steps to a cheaper model and keep the stronger one for the final answer. The routing rule is plain code in the backend, not another model call.
Cache what repeats. Many providers discount repeated prompt prefixes, and your own backend can cache complete answers to identical questions such as opening hours, return policy or pricing tiers. A normal cache table in the client's existing database is enough. No new infrastructure.
Set hard limits. Max output length, max turns per session, and a daily spend ceiling per client account. A ceiling is not a sign of distrust. It is what stops a looping script or a curious visitor with a bot from running up an invoice overnight.
Reliability: assume the provider will fail sometimes
Every hosted model API has rate limits, slow periods and the occasional outage. The question is what the customer sees when that happens.
Our default design has four parts.
Timeouts that match the screen. A chat widget should give up and say something useful after a set number of seconds, not spin forever. On mobile, where the connection itself may drop, we set shorter timeouts and keep the typed message so the user does not lose it.
Retries with a limit. One or two retries with a short wait for temporary errors. Not endless retries, which turn a small outage into a pile-up of queued requests that hits the moment the provider recovers.
A fallback path. For some clients this is a second model or provider behind the same internal function. For many it is simpler: show the contact form, the phone number, or the FAQ search that already exists. An assistant that fails into a working human route is better than one that fails into an error message.
One internal interface. The application never calls the provider directly from ten places. There is one service class in Laravel, one module in the Node or Python service, one API route in the React app. Swapping providers or changing a model name then means editing one file, not hunting through the project.
Log enough to debug, not enough to leak
For every request we store: timestamp, model, token counts, latency, whether a fallback was used, and a reference to the conversation. That gives the client a real usage report and gives us something to look at when someone says the assistant "felt slow on Tuesday".
Storing full prompts and answers is more sensitive. If the assistant touches customer data, we agree with the client up front on what is kept, for how long, and who can read it. Often the answer is a short retention window and masking of emails and phone numbers. This is a conversation to have before launch, not after the first complaint.
Test with their questions, and keep testing
We keep the 30 to 50 questions from week one as a small regression set. When we change the prompt, the retrieval settings or the model, we run the set again and compare answers and cost side by side. It is not a research benchmark. It is a spreadsheet that tells us whether a change helped this client's real traffic or quietly made it worse.
When a provider retires a model or changes its behaviour, the same set tells us in an hour whether anything needs attention.
What to ask before you commission an AI feature
If you are a business owner thinking about adding an assistant to your site or app, these questions will tell you a lot about any vendor, including us:
- What is the expected cost per conversation, and how was it measured?
- What does the user see when the model provider is down?
- Is there a spending limit, and who gets alerted when it is reached?
- Can the provider or model be changed without rebuilding the feature?
- What conversation data is stored, and for how long?
Good answers are short and specific. Vague answers about "scalable AI" are a warning.
Where this fits in a Rayaxis project
We usually scope an assistant as a small phase attached to an existing build: a WordPress or Laravel site, a React front end, an Android or iOS app. The first deliverable is not the chat box. It is the cost estimate, the fallback behaviour and the question set. The interface comes after those are agreed, because by then the risky decisions are already made.
If you have an assistant idea and want a realistic cost and failure plan before committing budget, send us a few of your real customer questions through the contact page. We will tell you what we would build and roughly what it would cost to run.