Data migration
What’s a data migration for ERP? In a nutshell: it is a project to create or to copy data in a target ERP system based on existing data or based on business processes to be executed in the target system. Such a project can be triggered by many different events. For example: To deploy a new ERP, so migrating data from a legacy solution to the new ERP solution; to import the data from an acquisition; to support a systems consolidation…
It is usually presented as a well-organized process, but experience might have already taught you that it often resembles chaos and that it can easily derail the project entirely. So how do we avoid the pitfalls, mitigate the risks and finish on time for the go-live?
What data do we need to create or to transfer? Do not merely think products, customers, purchase orders and the likes. We need to think which products, which customers, which orders.
It means that we need to define a logic to decide what data is worth something for business and meeting the objectives of the project. A basic example: what is the definition of an active customer? Of an active product? If we need to carry over some unpaid invoices, we also need rules to decide if we take them all, no matter how old, if we will write some off.
Migrating data is usually not as simple as cutting and pasting rows in a spreadsheet. It must be executed in a way that will support the project and that ensures that the migration itself can be validated. Therefore, the various activities described below must be carefully organized.
We must consider the different phases of the project, the clarity we need to have about the data required, the flexibility – or level of unknown – we can be comfortable with, the amount of coordination required between the teams, etc.
By now, we know why we need to do a data migration, we have an idea of what we need to do and when we need to do it. We can look into how to do it. Most often, cleaning up the data is the first activity that comes to mind.
The requirements depend on the objectives of the project and the scope of data: are completing a merger? Or are we upgrading the software to a newer version? In the first case, we may have to revisit the customer database, identify duplicates, fix addresses and contact details. With the latter, we might get away with a copy-paste!
We also need to prepare for formal and informal expectations. For example, some people will demand or just expect that the data be cleansed even if it is not required for the project and often without giving the resources needed to support this activity.
Depending on the project’s objectives, this step may be trivial – an upgrade of a standard software – or awfully complex – a multi-billion dollars merger of 2 global corporations. It is also the step most easily assigned to IT. Make no mistake that ETL is a business concern.
When does it make sense to extract data, not just to manage the impact on the systems, but from a business point of view, to extract the right information at that moment?
When is it possible to load data, to avoid impacting ongoing business activities or to avoid corrupting important reports?
And the transformation is the activity that formalizes all the business rules and the business logic in a way that the new data is ready to support the company’s operations.
To ensure it is indeed fit for use, the quality of the data must be managed during the entire process. But how do we know that the data is of high quality or not?
We already identified some requirements to execute the cleansing. We also defined business rules to execute the transformation. Following the scope and objectives of the project, we will define the remaining rules and conditions, in increasing levels of detail, from the need to register a customer account, to the rules for delivering goods on time to its warehouses.
An indispensable component of any project activity. The data migration must be managed like any project, with a plan for controlling scope, time and budget, a process to manage risks, to manage resources, etc.
We need a solid communication plan. Not only to share the progress of the work, but also to explain and remind what is being done, why, how, when and by whom.
It will help all participants with following the process and it will also help control, if not prevent, informal expectations.
Our work cannot be complete if we do not prepare for the future, and that means to ensure that there are processes in place to manage the data after the project. Maybe they are unchanged, maybe they do not exist yet!
Ideally, these processes must strike a balance between protecting the quality of the data, providing the data on time and where it is needed, being cost efficient, and dealing with the legacy of the company.
Finally, but only lastly, the tooling needs to be studied. The selection depends on what is already available in the company, what is the scope and objectives of the project, what is the volume of data to be migrated, what the complexity of the data, etc.
There is a wide selection of software available on the market today, with amazing functionality, criss-crossing many business cases. SAP, Oracle, Microsoft, Talend, SAS, Informatica, IBM, Tibco are all familiar names and they provide state-of-the-art solutions. And there are many more, providing more specialized tools that should not be ignored if we seek the best solution for the company: a solution that gets the right job done correctly at the best price…
We got the job done for many happy clients
 |
 |
 |
 |
