Data Migration
Data migration & cleansing (or Extract, Transform, & Load)
One of the most difficult, time consuming and least popular tasks to be tackled when moving to a new system is the migration of data from the ‘old’ source system to the ‘new’ target system. The visibility of the data in the source system may be poor and the old adage “garbage in, garbage out” is a risk that poses long term disruption and cost.
During a migration there are a number of activities to be achieved, these include:-
- cleansing data,
- reporting data changes to business process owners, auditing revisions, reconciling control totals,
- adding new fields, and changing the data structure,
- adding new validations, and translating values,
- joining data from multiple sources etc.
The source database is frequently incapable of recording these data changes whilst maintaining the ‘live’ environment. Frequently the ubiquitous workhorse, MS Excel, is enlisted for this work. Unfortunately, despite Excel’s flexibility to manipulate data it has enormous shortcomings for this task: these include limited control over data changes & no audit trail, limited access to the data, and poor reporting and work flow to advise and assess the impact of data changes.
WhiteSky Software has considerable experience in managing and migrating large volumes of data and has applied this to building a flexible Cloud based RAD configuration tool that is ideal for data migration. (See Case Study) Using WhiteSky Migrate:
- data can be imported from the source system
- fields can be configured to cleanse and validate the source data
- data changes can be validated by the cleanse team
- work processes can advise the business of the impact of proposed revisions and data conflicts can be resolved.
- as the cleansing process progresses data can be exported and re-imported from the source to keep the ‘live’ environment current
- data can be enriched for the target system
- target system validations can be set and data repaired
- data can be exported to the target system as data is updated and modified, these exports can be created with a ‘single-click’ command.
WhiteSky Migrate can be accessed from any WEB enabled device allowing groups of people to complete different tasks. As a secure Cloud based system using MS Azure data services WhiteSky can scale its server processing and storage capacity to expedite all data transfers and storage requirements.
