Customer Conversations in Financial Services

We have had a chatbot since 2017, long before Generative AI for Financial Services existed as a category. It handled eleven intents, failed outside them, and customers learned to type the word agent immediately.

Replacing it with Generative AI for Financial Services capability was not the hard part. Deciding what it is allowed to do was.

The failure mode changed completely

Our old system failed visibly.

Ask something outside its eleven intents and it said it did not understand. Customers were frustrated and nobody was misled.

The replacement answers anything, fluently, including questions it should not answer. That is a different problem and it needs different controls.

The history of conversational AI explains the shift. Systems from ELIZA in the mid 1960s through the intent based era of the 2010s could only follow paths somebody had defined. Models built on the transformer architecture published in 2017 removed that boundary, and with it the honest failure that came from it.

Separate answering from acting

The distinction that structures everything.

Answering a question is reversible. A wrong answer can be corrected, and a customer who acts on it has usually not acted irreversibly.

Acting on an account is not reversible in the same way. Moving money, changing a limit, cancelling a product, or updating a record creates something that has to be undone.

Our Generative AI for Financial Services system answers. It does not act. Every account action is a handover to a person, and that boundary is enforced in the integration rather than in instructions.

Ground everything and show the source

Fluency is not accuracy.

Every answer comes from our published material, with a link to the page it drew from. Where nothing relevant exists, it says so and offers a person.

Customers click those sources more than we expected. Showing the working increased trust measurably, and it means a wrong answer is discoverable rather than invisible.

Make the handover good

The part most implementations neglect.

When a customer asks for a person, they get one, immediately, with the conversation history attached so they do not repeat themselves.

Our old system made leaving it difficult, which is why customers typed agent first. Making the exit easy reduced how often people used it, because they stopped feeling trapped.

Decide what it will never discuss

A written list, enforced technically.

Complaints, financial difficulty, bereavement, suspected fraud, and anything suggesting vulnerability go straight to a person. The system does not attempt an answer and does not ask qualifying questions first.

That list was written with our vulnerable customers team. It is the control I would be least willing to give up.

Measure refusal as a success

Our scorecard counts four things.

Questions answered without a person. Questions correctly handed over. How often a customer returns within a day with the same question. And how often the system answered something on the never list.

That last number is zero and is checked every month. Containment rate alone would reward a system that answers everything regardless.

Connect it to the records carefully

Personalised answers need reach, and reach needs controls.

Our Enterprise Application Integration Services work provides the customer view, with permissions inherited rather than bypassed and every retrieval logged. The system can see what the authenticated customer could see and nothing more.

Before authentication it answers only from public material. That separation is absolute and it is where most of the design effort went.

Write the tone down, then test it

Fluency without direction produces a voice nobody chose.

Ours is defined in writing. Plain language, no financial jargon without explanation, no false cheerfulness, and never apologetic about applying a rule correctly.

We test against it. A sample of conversations is reviewed each week against the written tone, and prompts are adjusted where the output drifts.

Generative AI for Financial Services deployments that leave tone to the model will get whatever the model defaults to, which is usually several registers at once.

Keep a person accountable for it

Not a committee.

One named person owns the system, reviews sampled conversations weekly, receives the scorecard, and can disable it within minutes.

That weekly sampling has found things no metric reported. A phrasing that was technically correct and read as dismissive. A journey where the handover happened one step too late.

What we required before launch

●        Answering only, with account actions handed to a person.

●        Grounded responses with sources shown.

●        An immediate handover with history attached.

●        A written list of topics it never attempts.

●        Public material only before authentication.

●        Logged retrieval with inherited permissions.

●        A named owner sampling conversations weekly.

Common questions

Does it need a separate knowledge base?

No, and it needs a current one. Ours answers from published material, kept current by the team that writes it rather than by a copy nobody updates.

What containment rate is realistic?

Ours handles under half, deliberately. Pushing it higher would mean answering things that should reach a person.

Should it ever act on an account?

Not in our design. Generative AI for Financial Services can prepare an action for a person to approve, and the approval is the control.

Does this need its own governance?

It does. Generative AI for Financial Services in a customer channel carries conduct obligations that an internal drafting tool does not.

How do you handle vulnerability?

A written list of topics that route straight to a person, agreed with the team that specialises in it, enforced technically.

How is the customer view supplied?

Through our Enterprise Application Integration Services layer, with permissions inherited and every retrieval logged. Before authentication it sees only public material.

Was the old chatbot worse?

It was more limited and more honest about it. The new one is better and needs considerably more governance to stay that way.

What changed for customers

Fewer people type agent immediately, and the ones who need a person reach one faster.

The capability improvement was the easy part. What took the time was deciding what it would refuse to do, and building those refusals into the integration rather than into a prompt that a clever question could talk around.