Kemlat Header
Kemlat
  • Home
  • About
  • Services
    Business & TransformationA clear view of the problem and a plan you can deliver.8 Project & DeliveryGovernance that connects plans to results.3 ERP & CRMConnected processes and data in systems that fit your work.2 Digital ProductsWebsites, apps and platforms built around your needs.8 AI & AutomationPractical solutions for tasks that should take less time.8 Quality EngineeringMore confidence in every release.3 Cloud & SecurityA dependable foundation for your digital work.5 Managed ServicesStability, support and improvement after launch.3 Brand & CreativeAn identity and experience that tell your story clearly.4 Digital GrowthBetter visibility and decisions based on performance.5 Games & ImmersiveInteractive experiences for learning and for brands.4 Training & EnablementSkills your team can put to work.3
    All services56

    The KEMLAT Method

    KEMLAT
    Build your projectPick the services you need and get an approximate cost range in a minute.
  • Work
  • The Method
  • Insights
  • Contact
ARGet in touch
  • Home
  • About
  • Business & Transformation Project & Delivery ERP & CRM Digital Products AI & Automation Quality Engineering Cloud & Security Managed Services Brand & Creative Digital Growth Games & Immersive Training & EnablementAll services Build your project
  • Work
  • The Method
  • Insights
  • Contact
Get in touch
العربية [email protected] +962 79 122 3241
Check the Main Menu location in Apppearance->Menus->Display Location.
Cleaning Historical Data Before a System Migration - Kemlat Insights
Kemlat2026-10-21T16:51:29+03:00
All insightsاقرأ بالعربية
Enterprise Systems

Cleaning Historical Data Before a System Migration

Moving fifteen years of records into a new system just in case is a frequent reason migrations run late. Decide what to bring, clean it properly and archive the rest.

16 November 20265 min read
Share
Cleaning Historical Data Before a System Migration - Kemlat Insights

The old system holds fifteen years of transactions, thousands of customers who have not bought anything in years, and products nobody remembers stocking. The natural instinct is to move everything into the new system “just in case”. That instinct is usually wrong. Acting on it is a frequent reason migrations run late and new systems start life full of old problems.

Decide how much history you really need

Data cleaning before migration starts with a simpler question: what is worth migrating at all? This is a business decision, not a technical one, and it should be made by finance and operations together with the sponsor. Every record you move must be mapped, cleaned, loaded and checked. Records you leave behind can be archived safely and still consulted when needed.

A common approach separates data into three groups:

  • Master data in active use: customers, suppliers, products, employees and accounts that are still relevant. These must be migrated and cleaned carefully.
  • Open transactions and balances: unpaid invoices, open orders, stock on hand, fixed assets and opening balances. The new system needs these to operate from day one.
  • Closed history: past invoices, orders and journals. These are often migrated as monthly totals, or only for a limited recent period, with the full detail kept in an accessible archive.
Data typeTypical approachPoints to check
Active customers and suppliersMigrate after cleaning and deduplicationDefine “active”, for example a transaction in the last two years
ProductsMigrate active and stocked items onlyItems with stock or open orders must be included
Open receivables and payablesMigrate each open invoice and bill individuallyMust reconcile to the general ledger at cut-off
General ledger historyOpening balances plus monthly totals for comparison periodsConfirm what your auditors and finance team need
Detailed past transactionsArchive in a searchable, read-only formRetention requirements and who needs access

Tax and legal retention rules apply to the archive regardless of what you migrate, so confirm the required retention periods with your auditor or tax adviser before switching anything off.

The problems you will find

Historical data in long-running systems tends to carry the same kinds of issues, and it helps to know what to look for:

  1. Duplicates. The same customer entered several times with slightly different names, phone numbers or spellings in Arabic and English.
  2. Inactive and test records. Customers who closed years ago, discontinued products, and records created for training or testing.
  3. Inconsistent coding. Categories, cities or units of measure that changed over the years, so the same thing appears under several codes.
  4. Free text used for structured data. Credit terms, contact names or delivery instructions typed into notes rather than proper fields.
  5. Balances that do not reconcile. Customer and supplier account totals that differ from the general ledger because of old manual adjustments.
  6. Missing mandatory data. Tax numbers, addresses or product classifications that the new system requires but the old one did not.

None of these problems is a sign that anyone did a bad job. They are the normal residue of years of real work in a system that allowed shortcuts. The point is to deal with them deliberately, before go-live, rather than discovering them one by one afterwards.

A scenario: a pharmacy chain replacing its system

Consider a pharmacy chain with branches across Jordan that has used the same system for over a decade. The product file holds tens of thousands of items, many of them discontinued medicines and duplicate entries created by different branches. The customer file includes insurance companies, corporate accounts and many walk-in customers recorded inconsistently.

The team agrees early that only items stocked or sold in the last two years will migrate, with the rest archived. Duplicate products are matched using barcodes and supplier codes, with pharmacists reviewing uncertain matches. Corporate and insurance customers are cleaned and verified one by one, because their contracts and balances matter. Walk-in customers are not migrated individually.

Opening stock is loaded from a physical count close to go-live rather than from the old system’s balances, which were known to be unreliable. History stays available in a read-only copy of the old database for reference and audit, with access limited to the people who need it.

A practical cleaning routine

  1. Run a data health check. Run counts, find blanks, spot duplicates and list unusual values before deciding anything.
  2. Agree cleaning rules with the business. What counts as active, how duplicates are merged and which record wins. Finance and operations should own these rules, not IT (information technology) alone.
  3. Clean in the source where possible. Fixing records in the old system keeps operations consistent until cut-off and makes repeated test migrations easier.
  4. Automate the repeatable fixes. Standardising formats, mapping old codes to new ones and flagging duplicates can be scripted so they run the same way on every trial load.
  5. Assign people to the judgement calls. Uncertain duplicate matches and unclear balances need someone who knows the business to decide.
  6. Run trial migrations and reconcile. Load into a test environment, compare counts and totals with the source, and repeat until the results are stable.
  7. Freeze and sign off. Shortly before cut-off, stop creating new master records in the old system where you can, run the final load, and have finance sign off the reconciled balances.
Rule of thumb: start cleaning months before go-live, not weeks. Data cleaning almost always takes longer than planned, and it is the one task that cannot be rushed at the end without consequences.

Starting the new system with clean foundations

A new system loaded with old problems quickly loses the trust of its users. Being selective about history and cleaning the master data you keep gives the new platform a fair start. Data architecture and migration are a central part of our Enterprise Systems service, and the repeatable cleaning steps are often automated so every trial load behaves the same way. The time spent deciding what to leave behind is usually some of the best invested in the whole migration.

Found this useful?Share it with your team or someone who needs it.
  • WhatsApp
  • LinkedIn
  • X
  • Facebook
  • Telegram
  • Email
Related service

Enterprise Systems

We design and build systems, and migrate data into them, based on real operational needs, not generic templates.

Also related: Intelligent Automation

Explore our services

Related insights

Mobile Apps for Field Teams: When They Are Worth Building - Kemlat Insights
Enterprise Systems

Mobile Apps for Field Teams: When They Are Worth Building

Paper job sheets sent by WhatsApp make the case for an app feel obvious. Sometimes a custom app is right. Often a simpler option fits the…

15 Nov 2026 · 5 min read
Budgeting for an ERP Project: The Costs People Forget - Kemlat Insights
Enterprise Systems

Budgeting for an ERP Project: The Costs People Forget

ERP budgets often overrun because of costs that were real from the start but never made it into the budget. These are the lines to add…

14 Nov 2026 · 5 min read
Reporting Architecture: One Data Model, Many Reports - Kemlat Insights
Enterprise Systems

Reporting Architecture: One Data Model, Many Reports

When the sales report and the finance report disagree, meetings turn into arguments about numbers. A shared data model means every report uses the same definitions,…

13 Nov 2026 · 5 min read

Kemlat Footer
Kemlat
Kemlat

We help organisations transform complex operations into clearer ways of working, practical solutions, and measurable outcomes.

Quick Links

  • Home
  • About Kemlat
  • Our Services
  • Selected Work
  • The Method
  • Contact Us

Services

  • Business & Transformation
  • ERP & CRM
  • Digital Products
  • AI & Automation
  • Cloud & Security
  • Digital Growth
  • All services
  • Build your project

Get in Touch

Email [email protected]
Location Almadina Almunawwara St., Amman, Jordan
WhatsApp & Phone +962 79 122 3241

© 2026 Kemlat. All rights reserved.

Privacy Policy Terms of Use
Kemlat Method

Request a Service

Tell us what you need and we will get back to you within one business day.

By sending this form you agree to our privacy policy.

Request received

Thank you. Our team will review your request and contact you within one business day.