LLM Integration with Your Existing and Legacy Systems
LLM integration connects a language model such as Claude, GPT or Azure OpenAI to the systems you already run (ERP, CRM, databases, document stores, older software) so AI works on your real data, with access control and logs.
In short
- Three common patterns: retrieval over your documents (RAG), an assistant or chat UI inside existing software, and model calls inside business workflows.
- Older systems without an API are reached through database views, file exports, middleware or, as a last resort, UI automation.
- Permissions follow the source system: the model only sees what the user asking is allowed to see.
- Model endpoints are chosen for data rules, including Azure OpenAI in Canadian regions where available.
- Price: $9,800 CAD + HST for a six-week AI Pilot Sprint that puts one integration into production.
Your situation
The data AI needs usually already exists; it sits in systems that were built long before language models. Typical requests:
- Staff search five systems for one answer. Policies in SharePoint, history in the CRM, numbers in the ERP.
- An older core system. A case management, ERP or line-of-business application with limited or no API.
- A software company adding AI. You deliver software to clients and need LLM features inside it, with tenant separation.
- A pilot that never reached production. The demo worked on sample data; the real systems were never connected.
Integration patterns
| Pattern | What it does | Typical use |
|---|---|---|
| Retrieval (RAG) | Indexes approved documents; the model answers with citations | Policy and knowledge search, support answers |
| Embedded assistant | Chat or side panel inside the existing application | Case summaries, draft replies, record lookup |
| Workflow step | Model call inside a process: classify, extract, draft | Intake, triage, document handling |
| Agent with tools | Model plans steps and calls your APIs | Multi-system tasks with approvals |
For agent designs, see AI agent development. For document-heavy flows, see intelligent document processing.
What the AI Pilot Sprint includes
One integration in production. One pattern, connected to your real systems.
Connection layer. APIs, database views, exports or middleware, with retries and error handling so the source system is protected.
Permission model. The model sees only what the requesting user may see.
Model endpoint setup. Provider and region chosen for your data rules; keys and accounts in your name.
Logs and evaluation. Every request and response logged; a test set from your data run before launch.
Runbook and handover. Your IT team learns to operate, monitor and change it.
Security checklist for LLM integration
- Service accounts with the least access needed, read-only where possible.
- Permission checks at retrieval time, so answers only draw on documents the user may open.
- Field masking for data the model does not need, such as account numbers or health card numbers.
- Rate limits, retries and timeouts that protect the source system from load.
- Full logs of requests, retrieved documents and responses, kept for the period your policy sets.
How long it takes
Six weeks. In our experience the first two weeks go to access, security review and data mapping, especially for older systems. The schedule depends on your change-management process, vendor support for the older system and data quality.
What it costs
The AI Pilot Sprint is $9,800 CAD + HST for one production integration. Model and hosting costs are billed to you directly. If you first need to choose between providers, see AI vendor selection; all packages are on the pricing page.
Why Remolda for LLM integration
- Source systems stay stable. Read-only access first, rate limits and retries by default.
- Permissions enforced. Retrieval follows the rights in the source system.
- Provider-neutral. Azure OpenAI, OpenAI, Anthropic or a combination, tested on your task.
- Canadian privacy rules in the design. Data flows documented for PIPEDA and Law 25.
- You own it. Code, configuration and logs in your repository and tenant.
How the work runs
Integration spans the Strategy and Implement steps of the Remolda Cycle (Audit → Strategy → Implement → Empower → Evolve).
- Architecture call. Systems, data, security rules, target pattern.
- Access and mapping. Service accounts, data map, test data.
- Build. Connection layer, retrieval or workflow, permission checks.
- Evaluation. Test set run, security review, fixes.
- Production and review. Live use, logs reviewed, go / adjust / stop.
Frequently asked questions
What is LLM integration?
LLM integration is the engineering work that lets a large language model read from and act on your business systems: connecting APIs or databases, retrieving the right documents, enforcing permissions, logging every call and handling failures so the existing system keeps working.
What are the main LLM integration patterns for enterprise software?
Retrieval-augmented generation (the model answers from your indexed documents), an assistant embedded in the existing user interface, model calls inside workflows (classify, extract, draft), and agents that use your APIs as tools. Projects often combine two of them.
Can you integrate AI with a system that has no API?
Usually yes: through read-only database views, scheduled exports, an integration layer (middleware) or, when nothing else exists, UI automation. We choose the least fragile route and document it.
Which LLM provider should we use?
It depends on data residency, existing contracts and task quality. Azure OpenAI can process prompts inside Canada for some models (standard deployments in Canada East). The OpenAI API can store data at rest in Canada for eligible customers, with processing elsewhere; Anthropic processes in the US or globally, and through Bedrock from Canada Central keeps data at rest in Canada. We test candidate models on your task before committing.
How is sensitive or financial data protected?
Least-privilege service accounts, permission checks at retrieval time, masking of fields the model does not need, business tiers that do not train on your data by default, regional endpoints where required, and logs of every request.
How much does an LLM integration project cost?
Remolda's six-week AI Pilot Sprint is $9,800 CAD + HST for one production integration. Larger programs across several systems are scoped after the pilot. Model usage is billed by the provider.
How long does it take?
Six weeks for the first integration. In our experience access approvals, security review and the state of the older system's data set the pace more than the model work.
Sources
- Microsoft Learn — Foundry models sold by Azure: region availability
- OpenAI — Business data privacy, security and compliance
- Claude Platform docs — Data residency
- OpenAI API docs — Your data
- Anthropic Privacy Center — Is my data used for model training?
Facts checked:
Related services
Approach phases
Related insights
LLM Integration into Existing CRM and ERP Software in Canada: Architecture, Data Residency and Cost
Mitacs AI Advantage: Ottawa Commits $162M to 10,000 AI Work Placements
AI for Canadian Municipalities: Where It Works in 2026
Talk to an AI transformation consultant
A 30-minute call: you describe the situation, we tell you what to do first and what it would cost.
Book a 30-min call30 minutes. English or French.