uLinga’s New Data Transformation Capabilities – Including ISO8583 Support

uLinga for Kafka, widely recognised as the best solution for integrating NonStop applications with Kafka, and uLinga Gateway, the newest product in the uLinga suite, have recently been enhanced to add powerful data transformation capabilities.  This article gives an introductory overview to these capabilities, and then takes a deep dive into one possible use case – transforming the venerable ISO8583 standard into JSON.

uLinga Gateway – Introduction

uLinga Gateway, recently added to the uLinga suite, provides a wide range of network protocol handling and message manipulation services that can be used in a number of applications and use cases.  uLinga Gateway can handle any combination of TCP/IP, TLS, HTTP, Pathway, Guardian IPC and more, allowing you to connect almost any NonStop application with any endpoint via simple configuration.

Now, with Data Transformation, uLinga product capabilities are extended to allow easily configured data transformation and message manipulation.   uLinga’s Data Transformation can handle any combination of binary and JSON messages for any input or output messages, with additional message formats in development.

Use Case – ISO8583 Data Transformation

The primary focus of this article is a very common use case in NonStop circles, involving messages using the ISO8583 standard.  ISO8583 has been in very wide usage in the NonStop financial transaction processing space for decades, courtesy of its broad adoption between financial institutions, switches and card brands such as Visa and Mastercard.  While ISO8583 is undoubtedly powerful and very capable in this space, it is not a simple protocol, and developing new applications to support this standard can be a challenge.

uLinga’s Data Transformation now supports ISO8583 messages, and can transform them into a format more easily supported by external entities, such as JSON, allowing REST endpoints to access traditional transaction processing mechanisms.

ISO8583 messages have a number of aspects that can take some time to learn and understand, including bitmaps and variable length fields.  A standard ISO8583 message looks something like this:

The Message Type Indicator (MTI) indicates what type of message this is, and bitmaps are used to indicated which fields are present in the message.  The fields themselves can be fixed format, or variable with a two- or three-digit length (LLVAR and LLLVAR). The entire message can be in ASCII or in binary/binary-coded decimal.  There is a good primer on ISO8583 message formats available in Wikipedia.

An example use case would be where a NonStop application processes incoming ISO8583 requests to allow access to the financial transaction processing services it provides (for instance, balance enquiries).  Traditional clients would access this service using ISO8583 requests sent directly into the NonStop, as depicted below.

Providing this balance enquiry transaction as a REST service would make it much more easily accessible for a variety of new applications, such as web applications, mobile phone apps and so on.

uLinga Gateway can facilitate this capability, by accepting incoming JSON messages and converting them on the fly to ISO8583 to present to the financial transaction processing application.  The NonStop application requires no changes in this scenario, and continues to process the ISO8583 requests as it always has done.

The JSON request does not need bitmaps, and variable length fields are handled simply through the JSON formatting – no need for LLVAR and LLLVAR fields.  uLinga Gateway adds these elements as part of building its ISO8583 request.

A similar transformation takes place with the response, allowing the client to once again deal with simple JSON.

uLinga Gateway’s Data Transformation GUI (built into the WebCon UI) allows you to easily build your data transformation, mapping fields from input to output via drag and drop.

uLinga Gateway Data Transformation can also transform other types of binary messages into JSON and even convert between formats – binary to binary or JSON to JSON, for instance.  When combined with uLinga Gateway’s protocol transformation capabilities, you have an extremely powerful tool that can address virtually any application connectivity requirement.

uLinga for Kafka and Data Transformation

uLinga for Kafka, which is now in use at a number of HPE NonStop installations worldwide, has a broad range of options for integrating NonStop applications and data with Kafka.  Some of the key features of uLinga for Kafka include:

• FILEREADER to automatically monitor and read new records as they are written to entry-sequenced Enscribe files, and stream (produce) the data read straight to Kafka.
• AUDITREADER to automatically monitor and read new records as they are written to a TMF audit trail, and stream (produce) that data straight to Kafka.
• IPCSERVER support, allowing applications to directly send data to uLinga for Kafka, and from there be streamed (produced) straight to Kafka.

• FILEWRITER support, allowing for data to be streamed (consumed) from a Kafka topic and written to an Enscribe entry-sequenced file.
• IPCCLIENT support, allowing for data to be streamed (consumed) from a Kafka topic and sent to a NonStop application process.

With the addition of the new Data Transformation feature, this allows you to configure uLinga for Kafka to (for instance) read records from an entry-sequenced file, and convert those records into JSON before streaming them to Kafka, allowing consumers to much more easily understand the streamed data.

Do you need data transformation in your environment?  Chances are that uLinga’s new Data Transformation feature can help.  Get in touch with us at [email protected] to arrange a demo so you can see for yourself.