Blogs

Interfacing Virtual MDM through DataStage

Jul 25, 2019 6:46:27 AM / by Kanimozhi


Introduction

MDM Connector stage is a key to open the door of IBM Virtual MDM. Yes, we can manipulate the data in MDM (MDM refers to IBM Virtual MDM in this post) using the MDM Connector stage which was introduced in IBM DataStage v11.3.

We know that loading data into MDM is not an easy task since it involves many tables and the relationship among the tables should be maintained properly, otherwise will end up dealing with junk not with the data. MDM Connector stage makes this task simpler by allowing us to configure everything in the single configuration window.

This blog post details on how the basic operations (read/write) on data can be performed using the Connector stage in v11.5.

DataStage and MDM Server Connectivity

To connect to the MDM server, the following should be specified in the stage properties


Figure 1: Specifying Stage Properties

  • Host name – IP Address/Host Name of the MDM Server
  • Port number – http Port number of the MDM Server. This can be found in the app.properties file of MDM
Writing Data into MDM

The Interaction mode should be specified as Member write as shown in the following figure if we want to write anything into MDM

Figure 2: Write Data to MDM

Model identifier specifies the data model of the MDM which should be imported into IIS using the IBM Metadata Asset Manager. The steps to import IBM Infosphere MDM assets into an IBM Infosphere Information Server is given in the tutorial https://developer.ibm.com/recipes/tutorials/importing-ibm-infosphere-mdm-assets-into-ibm-infosphere-information-server-repository/

The Member attributes will be listed in the Configuration Window based on this Model identifier.

Source

Following is the source data which we would like to load into MDM:

Member write Configuration

Required Member type and attributes can be selected and the column wise mapping can be performed by clicking the next button which is at the bottom of the configuration window

Figure 3: Member write Configuration

Target

Tables corresponding to the selected Member attributes can be queried to check the data. Rejections can also be captured by adding the output link from MDM Connector stage and configuring the properties required for the Reject link as shown below.

Reading Data from MDM

The Interaction mode should be specified as Member read as shown in the following figure if we want to read anything from MDM

Figure 4: Data Read from MDM

Source

To read the records from MDM we have to provide one of the following values as an input:

  • MEMIDNUM and SRCCODE
  • ENTRECNO
  • MEMRECNO

The input data is MEMIDNUM and SRCCODE

Figure 5: Input Data

Member read Configuration

Required Member type and attributes can be selected and the column wise mapping can be performed by clicking the next button which is present in the configuration window

Figure 6: Member read Configuration

The output link can be chosen and the required columns can be selected across the member attributes. We have two output links – one for Name data and another for Address data

Target

Data can be written to the file or table or it can be processed further. Data in the address output link is shown as below:

Issues faced and the solution
Version Compatibility

DataStage job was failing and the log has below warning message

‘DicStore could not be created; Possibly due to the userId and/or password not being valid.
Underlying IOException: java.io.IOException: ErrCode.EVERSION’

By default, the version of the MDM API library (madapi.jar) provided with IIS is 11.3. This should be replaced with the correct version of a jar which is present in MDM

Location of madapi.jar in MDM server -

<WAS_INSTALL_LOC>/profiles/<NODE>/installedApps/<CELL>/MDM-native-*.ear/native.war/lib/

Location of madapi.jar in IIS server -

/opt/IBM/InformationServer/Server/DSComponents/bin

Multiple Rows returned for single MEMIDNUM

The DataStage job was returning multiple records if more than one table was being queried for a single Entity ID in v11.5.

The issue was resolved after applying the “fix pack2”.

About the Author

Kanimozhi is a Senior Consultant at Mastech InfoTrellis and has good knowledge on IBM DataStage. She has over 4 years of experience in implementing EDI solutions.

Topics: Blog, Access Virtual MDM usingDataStage, DataStage and MDM Connectivity, DataStage versions, MDM Connector Stage, Member read configuration, Member write configuration, Version compatibility

Kanimozhi

Written by Kanimozhi

Subscribe to Email Updates

Lists by Topic

see all

Posts by Topic

See all

Recent Posts