Data mapping

Data mapping is the process that maps data elements between different data models. Data mapping is needed as a first step for various tasks of information integration:

  • Data transformation or data mediation between a data source and a data target. For example, data mapping could be used to exchange purchasing and billing information between different companies. The data of a company, for example, according to standardized ANSI ASC X12 mapped messages.
  • Identification of data relationships as part of the analysis of data places of origin
  • Detect hidden, sensitive data such as parts of the social security number to IDs of anonymous data
  • Aggregation of different databases into a single database
  • Detection of redundant information for the consolidation or elimination

Standards

ANSI ASC X12 or EDIFACT are generic standards that enable companies of different industries, exchange data.

Techniques

Data mapping can be implemented algorithmically in different ways. These include the implementation using procedural code, the use of XSLT transformations or by means of graphical mapping tools automatically create the executable transformation programs.

Graphical tools allow the user lines between the fields of a data structure to fields of different data structure to pull. The tools can automatically recognize the relations of the fields due to their names and ranges of values ​​on a button. The defined relationships, these programs automatically generate SQL, XSLT or program code, for example, in Java (programming language ) or C . Such tools are usually part of ETL tools.

Semantic mapping is similar to the automatic detection of the above-mentioned relationship graphical tools with the addition that a metadata directory is used to identify synonyms. For example, if a data source places of residence, the other data source lists whereabouts, semantic mapping recognizes that these describe the same thing when residence and whereabouts are listed in the metadata directory as a synonym. However, semantic mapping detects only exact synonyms, and example, no transformations between place of residence and zip code.

Furthermore, there are also libraries that support the mapping data in memory. Dozer and are examples.

Data-driven mapping is the latest approach. By means of parallel evaluation of the data values ​​of two data sources tried Data-driven mapping automitisch to identify complex mappings between the two data sources on the basis of heuristics and statistics. This approach recognizes things such as pieces of data, data merges or arithmetic relations. Similarly, this approach recognizes exceptions that do not meet the identified mapping logics.

Extract, transform, load
219546
de