Could Your Historical Data Be Hurting Your AI?

When organisations start introducing AI into customer support and service management, there’s an understandable temptation to give it as much information as possible.

After all, more data should mean better AI. Right?

Not necessarily.

The quality and relevance of the information available to AI can be just as important as the volume. If years of historical support tickets contain outdated products, old pricing, superseded processes or policies that no longer apply, that history can become noise rather than useful context.

That creates an important consideration when moving to a new service management or support platform: should everything you migrate automatically become part of the information your AI uses?

---

TL;DR

Historical support data remains important for compliance, reporting, trend analysis, and giving agents context on past customer interactions.

However, historical data and AI-ready data are not the same.

When migrating, separate the info AI needs from records your organization must keep. Recent, relevant, well-maintained data offers a better AI foundation than all past tickets.

---

Why more data doesn't always mean better AI

Imagine looking at a support ticket from five years ago.

The product mentioned in it might have been retired. The customer could be on a pricing model that no longer exists. The support team might have followed an entirely different workflow, while the resolution could refer to a policy that has since been replaced.

The ticket remains a perfectly valid historical record.

But is it something you want informing an AI-generated answer to a customer today?

Help Desk Migration recently explored this problem in its AI-First Help Desk Migration Guide. Its recommendation is to think carefully about the data used to establish your AI baseline before bringing across the complete historical archive.

That distinction between historical data and AI-ready data is useful well beyond help desk migrations.

What makes support data useful for AI?

There isn't one universal formula because organisations, platforms and AI implementations differ. However, there are several useful questions to ask when assessing your data.

Is it recent?

Recent tickets are more likely to reflect your current products, services, policies and ways of working. Help Desk Migration suggests that, for many organisations, the most recent 12–18 months can provide a useful starting point, with an even shorter window potentially appropriate following a significant product or pricing change.

That shouldn't be treated as a hard rule. The right period depends on how quickly your organisation changes.

Was the issue actually resolved?

A closed or successfully resolved interaction tells a much more complete story than a conversation that stopped halfway through.

If you're selecting examples from which AI should derive useful patterns, completed interactions can therefore be considerably more valuable than unresolved tickets.

Was it a good resolution?

Where reliable customer satisfaction data is available, it can provide another useful quality signal.

After all, you probably don't want AI learning equally from your best customer interactions and from the cases where something went badly wrong.

Is the underlying knowledge still accurate?

This is particularly important.

Your knowledge base, service documentation and other maintained sources should reflect how your organisation operates today. Before thinking about how much historical data AI can access, it is worth asking whether the current knowledge you're giving it is accurate, complete and well maintained.

Don't delete the history

None of this means that old tickets have suddenly become useless.

Quite the opposite.

Historical service records can remain important for:

• compliance and auditing;

• management and operational reporting;

• identifying long-term trends;

• understanding previous customer interactions; and

• giving agents additional context when investigating an issue.

The important point is that retaining information isn't the same as using all of it to inform AI.

This is where a more deliberate migration strategy becomes valuable.

Consider an AI-ready migration first

One approach described by Help Desk Migration is to split the migration into two stages.

The first stage establishes a cleaner, more relevant dataset for AI. Depending on the organisation, this could prioritise current knowledge base content, recently resolved tickets and other records that meet agreed quality criteria.

Once that information has been migrated, the AI implementation can be tested and validated.

The second stage brings across the remaining historical records required for compliance, reporting and agent context.

This approach allows organisations to preserve their history without assuming that every historical record should carry equal weight in an AI-enabled environment.

It's also a useful reminder of a broader migration principle: a migration doesn't have to be a straightforward copy-and-paste exercise.

Migration is an opportunity to improve

Moving to a new platform provides a rare opportunity to look critically at how information has accumulated over time.

Before migrating, it is worth asking:

• Which information do we genuinely need?

• Which records need to remain accessible for compliance or reporting?

• How much historical context do our users need?

• Which information accurately represents how we work today?

• Are our categories, tags and custom fields still meaningful?

• Is our knowledge base current?

• What information should our AI actually use?

These questions can be particularly important when moving between service management platforms or consolidating several systems into one.

At BDQ, migration projects typically begin by understanding the desired outcomes and pain points rather than simply recreating the old system in a new platform. Discovery, prototyping, testing and training can all help ensure the destination system supports how the organisation wants to work going forward.

That principle has proved valuable in complex migrations too. For example, BDQ helped the Rainforest Alliance migrate a large, mission-critical Confluence environment to Atlassian Cloud using discovery, trial migrations, user acceptance testing and training to reduce risk and disruption.

→ Explore BDQ's case studies

“BDQ told us they could do it the way we asked but also recommended alternate options when they were available. They provided guidance and direction, which was nice. After all, they are the experts, I trust them completely.”

• Cintia Rivera | Senior Knowledge Management & Standardization Officer, Rainforest Alliance

Test before you trust

A curated dataset is only useful if it actually produces better results.

That's why validation should be built into an AI migration rather than treated as something to do after go-live.

Help Desk Migration recommends testing AI performance using tickets excluded from the initial AI-ready dataset. The idea is straightforward: give the system real examples it hasn't already seen and assess how accurately it understands and responds to them.

The precise measures will depend on the AI platform you're implementing, but you might assess areas such as:

• intent or request classification;

• accuracy of suggested responses;

• relevance of knowledge articles;

• routing accuracy; and

• whether generated answers remain grounded in approved information.

If results aren't good enough, investigate the data before simply adding more of it.

Sometimes the answer to an AI quality problem isn't more training information. It is better information.

Data quality should be part of your AI strategy

AI projects can quickly become conversations about models, features and automation. But the less glamorous subject of data quality can have just as much influence on the eventual outcome.

If an AI system is working with obsolete processes, inconsistent categorisation and outdated knowledge, moving that information into a shiny new platform doesn't make it current.

A migration therefore provides an ideal point to ask what information deserves to make the journey, how it should be structured and how it will be used once it arrives.

The objective isn't necessarily to migrate less.

It's to migrate more deliberately.

---

Planning a service management migration?

If you're considering a migration to a new work or service management platform, it's worth thinking beyond simply moving records from A to B.

BDQ provides consultancy, implementation, migration and training services, helping organisations understand their requirements, design the destination environment, test the solution and support user adoption.

Talk to BDQ about your migration