Skip to content

Data migration and data quality

Moving the data is the easy part.
Knowing it is right is harder.

A migration succeeds when the new system opens with customers, suppliers, products, stock, balances and transactions people can actually trust. We treat migration as a controlled business process, not a final technical task before go-live.

See how we approach it

We would rather find a data problem before go-live than explain it afterwards.

Sound familiar?

A new system cannot fix data nobody trusts.

Migration problems rarely start with an error message. They appear when somebody asks a simple question and nobody can answer it confidently.

  • Nobody agrees which record is right

    The same customer, supplier or product exists in several systems with different information.

  • Years of history have become part of the scope

    Nobody uses most of it, but nobody wants to be responsible for leaving it behind.

  • The old system contains workarounds nobody understands

    Codes, fields and spreadsheets acquired meaning over years without anyone writing it down.

  • Reconciliation happens at the end

    The data loads, then finance or operations discovers the totals do not agree.

How we approach it

Do not start by moving everything.

Migration decides whether customers are correct, suppliers can be paid, stock can be trusted and finance can explain the opening position. That makes it a business process with a technical step in the middle, rather than the other way round.

  1. Understand: What exists, who owns it and whether it is still needed.
  2. Clean: Resolve duplicates and invalid values before they become new-system problems.
  3. Map: Agree what the old data means in the new model.
  4. Move: Load it through a repeatable and controlled process.
  5. Reconcile: Prove that source and destination agree.
  6. Prove: Let the business confirm the data supports the process it has to run.

What should actually migrate

Not everything deserves to come with you.

Nobody wants to be the person who decides ten years of history can be left behind. It is still a decision, and it sets the size, cost and risk of the migration more than anything else does.

  • Migrate

    Needed to operate in the new platform.

  • Archive

    Needs to stay accessible, but not as live transactional data.

  • Rebuild

    Can be recreated more reliably from another source than carried across.

  • Leave behind

    No continuing business, legal or operational value.

Retention, regulatory and contractual obligations still need to be considered before information is archived or removed. That is a decision for your own advisers, not for a migration plan.

Not all data carries the same risk.

Master data and the opening position fail in different ways, so they should not be planned as if they were the same job.

CDM

Migration capability built from doing this repeatedly.

After working through migrations across different generations of ERP, it stopped making sense to rebuild the mechanics from scratch every time. CDM is our reusable migration capability, built around Microsoft data and automation technologies, to make extraction, transformation, validation and repeatable loading more controlled.

  • Extract

    Business users select the data and legal entities in scope, and it lands in SQL Server staging away from the live source.

  • Transform

    Legacy structures are converted into the shape the target platform needs, with mapping held in one controlled place.

  • Validate

    Records that fail the agreed rules are identified in a test environment rather than in production.

  • Repeat

    Azure Data Factory pipelines run the cycle again as cleansing progresses, so go-live is not the first real attempt.

CDM makes the technical migration controlled and repeatable. It does not decide what your data means, and it does not read an undocumented legacy field for you.

The source system is rarely tidy.

Some migrations start from a recent cloud application. Others begin with an ERP that has been modified for more than a decade, most often Dynamics AX 2009 or AX 2012 on the way to Finance & Supply Chain. The approach has to cope with both.

  1. 01

    Legacy ERP, finance and operational data

  2. 02

    Profile, clean and map

  3. 03

    CDM and Microsoft data services

  4. 04

    Dynamics 365, Business Central or CRM

Prove it before cutover

A successful import is not proof of a successful migration.

A hundred thousand records loading does not prove the right records loaded, the values are correct, balances agree, relationships survived, users can operate or reports reconcile. Go-live is a poor time to discover the stock balance does not add up.

SourceMigrationTarget
  • Count it

    Did the right number of the right records arrive?

  • Value it

    Do the balances and quantities carry the correct values?

  • Reconcile it

    Can the difference between source and target be explained?

  • Test it

    Can users run their own scenarios on the migrated data?

  • Sign it off

    Do the people who own the data accept the position?

Data quality is whether the information is good enough for the process that depends on it.

  • Customer data

    Can we invoice the customer and reach the right people?

  • Supplier data

    Can purchasing order, and can finance pay correctly?

  • Product data

    Can sales, planning and the warehouse actually use it?

  • Inventory

    Does the quantity and the value reconcile?

  • Financial data

    Can finance explain the opening position?

Go-live should not be the first time the migration works.

Rehearsing the migration before cutover is how a team learns how long it takes, what fails, which cleansing is still outstanding, what depends on what and whether reconciliation holds. By the real run, it should be the most familiar thing on the plan rather than the most exciting.

A file loading successfully does not mean finance will agree with it. That is why business validation, not technical completion, closes the migration.

Who decides what is right

IT can move the data. The business has to say whether it is right.

We profile, extract, transform, map, validate, load and reconcile. What the data means, and whether it is correct, stays with the people who work with it every day.

  • Finance

    Balances, receivables, payables and the opening position.

  • Sales

    Customer records, contacts and the commercial terms attached.

  • Purchasing

    Suppliers, payment details and purchasing terms.

  • Operations

    Products, stock, locations and the detail people work to.

  • HR

    Employee information, where it forms part of the scope.

The same applies after go-live. If nobody owns data quality once the project team has gone, the new system eventually inherits the old problem. Explore Optimise.

Migration and data work in practice

What we can currently show, and what is still being written up.

Selected migration stories are being prepared for publication. Until they are, this is the verified capability behind the migration work rather than a case-study library.

Legacy Dynamics estates

Migrations from Dynamics AX 2009 and AX 2012 into Dynamics 365 Finance & Supply Chain, including the legal-entity model, layered customisation and the reporting dependencies that came with them.

Controlled, repeatable loading

CDM stages selected data and legal entities in SQL Server, applies mapping and transformation, then loads through Azure Data Factory into a Dynamics test environment for validation, as many times as the cleansing needs.

Data work alongside delivery

Documented customer challenges on this site cover master-data control, integration and reporting work. They are related, but we would rather point you at them for what they actually show.

See customer stories

Selected customer examples. More customer stories and delivery evidence are being added.

Timing

Migration is not the last phase of the project.

Data exposes process differences, missing ownership, legacy customisation, duplicate structures and reporting dependencies. All of those are cheaper to find during design than during UAT.

  1. Discover

    Data profiling starts

  2. Design

    Mapping agreed

  3. Build

    Migration developed

  4. Test

    Migration rehearsals repeated

  5. UAT

    Business validates

  6. Cutover

    Final controlled migration

CDM: move and prepare data for change

Migration moves the required business position into the new platform, once, and then the job is finished.

IDM: keep systems connected afterwards

Integration keeps information moving between systems that both remain in use, every day, with someone responsible when it fails.

Explore Integration & Connected Systems

If a migration is already under way and nobody can say which extract built the current environment, the problem is control rather than data. Explore Recovery.

Check your migration position

Six questions before migration scope is agreed.

A routing aid rather than a quote. Your answers come with you into the conversation, so nothing has to be repeated.

Question 1 of 6

0 of 6 answered

1. Why is the data moving?

Select one option.

Common questions

Questions leaders ask about data migration.

Start with the data

What would you be worried about migrating tomorrow?

It might be customer records, stock, opening balances, years of history or a legacy field nobody can explain any more. Start there and we can work out what needs to move, what needs cleaning and how it will be proven.

See customer stories