Data migration

Data migration process deals with databases: to put it simply, data from one database is transferred to another one.

You might have already heard about Extract, Transform, Load (ETL) concept which is applied here:

1. First, data is taken from one source or several sources;
2. Second, data is processed to a proper format and/or structure;
3. Third, data is put to a desired database.

However, this process is not that trivial as it might seem at first sight. Why? You might encounter certain difficulties while doing data migration:

• One of the most common obstacles is that data in different databases can be presented in different formats;
• In some cases, not only data, but also logic needs to be transported;
• Moreover, there can be sets of text files that need to be moved;
• Today, as the data volumes increase, companies need to deal with big data migration and transport huge data sets from one storage to another, which requires certain experience.

This is where a professional service provider comes to rescue your project!

How is data migration is actually done?

Sometimes it can be done manually. Alternatively, a special script can be written to automate this process. This script can be tuned so that the migration process can be done on a regular basis, not just single-time.

As for automatic data transfer, most software solutions providers utilize two methods to achieve this. Let’s look through them.

1. Migration can be done on a database level. In other words, a database gets a command to take certain data from a certain source and create a certain table with data in a certain place. To achieve this, a script with migration rules is written.
2. Another way to implement migration is to write a special application that will interact with two (or more) databases. This way an application will take data from one source, transform it to a proper form, and insert into the desired destination.

Which way to choose depends on many factors. These are: a project type, tasks to be solved, available specialists, etc. Here at ISS Art we know the best ways to transfer data without any loss.

ISS Art experience

Data migration is useful when we work with one type of data, and need another type as an output. For instance, a Client may need to generate reports in another form. In case there is one database, and having another one is preferable, conversion tools can be used. Spark – an engine for data stream analysis – is one of them.

We rendered data migration services to our Customers while working on complex projects. When building a social network for music lovers, we implemented big data migration. In particular, we transported data from PostgreSQL to Hadoop, converted it to a proper format with Spark, and inserted back to PostgreSQL.

Does your project deal with data of different types and structures? Is your desired output in another format? Don’t worry, we know what to do. ISS Art is happy to be your consultant and your service provider. Get in touch with us to learn more.