NonStop Integration – LogWatch and uLinga
Updated: Mar 26
Modernization via Integration
One of the best ways to modernize NonStop is to integrate its applications with other software, platforms and cloud services. Integration with Kafka will open NonStop to interoperate with many different platforms and applications. uLinga for Kafka from Infrasoft provides the perfect mechanism to integrate NonStop applications with Kafka.
uLinga for Kafka
uLinga for Kafka is an Infrasoft product that enables NonStop applications to publish data or events to Kafka topics easily. uLinga is a natively compiled Guardian process pair and supports the Kafka communications protocols directly over TCP/IP. uLinga communicates directly with the Kafka cluster, ensuring data is streamed as quickly and reliably as possible.
uLinga for Kafka provides multiple ways for applications to publish data into Kafka topics, including:
Enscribe file monitor – New records in a monitored file are automatically streamed to specific Kafka topics
Pathsend and IPC support – Via a Pathsend or a WRITE/READ operation to uLinga, a NonStop process can invoke data to be published into a Kafka topic.
LogWatch from TIC Software, an Infrasoft partner, makes it even easier for NonStop applications to benefit from uLinga with its unique features like filtering and reformatting.
LogWatch
LogWatch is a TIC product that provides real time detection of errors and raise alerts to facilitate quick resolution. LogWatch is a Guardian program that monitors system log files (EMS and VHS), application logs, OSS files and third-party product logs. When a configured error pattern is detected, LogWatch takes preconfigured actions to forward the data to a variety of targets including email, text, another application or another system. Using LogWatch with uLinga provides an easy way to publish NonStop data and events to Kafka topics.
LogWatch can be used to extend the interface to uLinga for Kafka easily. LogWatch extends uLinga File Interface by monitoring EMS logs and Guardian files with additional capabilities, and leverages the uLinga Pathsend interface to publish the data to Kafka topics
Here are some of the LogWatch features which add value to the interface with uLinga.
Instead of sending all the records to Kafka, one may want to only send selective records. For example, those with errors or warnings, or ones with certain data fields such as certain RESPONSE CODE, etc. LogWatch has a MATCHING PATTERN feature which allows one to easily specify which records are of interest. This can be as simple as a pattern data match or as sophisticated as using REGEX.
Matching Patterns can be one of the following:
A pattern to match, e.g “*” Matches anything “*warning” Matches anything with trailing “warning” in it “*error*” Matches any occurrence of “error” in it
A regular expression which can specify any combination of pattern matching
While Kafka can work with any type of data, best practice dictates that published data should be in a structured format, preferably with meta data. Instead of just sending the raw data to Kafka, LogWatch can transfer it into a more structured meaning format by configuring a reformatting rule.
Original Raw Data (9001) 2021/03/03 17:02:24.75 $DMR 00390 ERROR AWAITIO to SMTP-GATEWAY failed ERROR (9001)
Reformatting Rule Format String=”Node=\K2K,Date=%P1%,Time=%P2%,Pname=%C24.6.T%,Error=%C30.5.T%,Text=%C43.100.T%”
Output is Name/Value Pair Node=\K2K, Date=2011/03/03, Time=17:02:24.75, Pname=$DMR,Error=00390, Text=AWAITIO to SMTP-GATEWAY failed ERROR (9001)
In fact, LogWatch’s reformatting rule can be used to convert the raw data into a JSON payload.
LogWatch can monitor a single input file or EMS for different match pattern. It can be configured to take a different ACTION based on the matched pattern. One simple example is that if EMS Log contains the key word “BASE24”, LogWatch can send that to uLinga for Kafka producer that handles the “BASE24” topic, while a record with the keyword “ACH” will be routed to a different topic accordingly.
Summary
Kafka is a growing platform in the enterprise, and it opens up opportunity for NonStop to integrate easily with rest of the enterprise and cloud services. uLinga from Infrasoft makes it easy to interface to Kafka with its available interfaces. LogWatch from TIC extends those interfaces and serves a perfect companion product to uLinga to further facilitate publish NonStop data to Kafka.
Read More Blogs:
Feedback please
Do you find this tutorial blog helpful? Let us know what you think, and how we can make it even better. Don’t forget, you can subscribe to our blogs to get automatic email notification when a new blog is available.
Phil Ly is the president of TIC Software, a leading provider of software and services for the NonStop community focused on NonStop application modernization. Under Phil's guidance, TIC Software has a proven track record of helping clients seamlessly integrate NonStop with next-generation technologies. A recognized technical leader, Phil is passionate about empowering organizations to unlock the full potential and longevity of their NonStop platform through innovative solutions.