top of page
  • Writer's picturePhil Ly

How to Aggregate NonStop Logs to Kafka in One Simple Step


Log Aggregation chart

Apache Kafka is a powerful streaming platform that has become a popular choice for log aggregation due to its ability to handle large volumes of data with low latency and high throughput. It provides a reliable and scalable infrastructure for collecting, storing, and processing log events from various sources, enabling centralized log management and real-time monitoring. Kafka's strengths in distributed data streaming, durability, and fault tolerance make it well-suited for handling the demanding requirements of log aggregation in modern distributed systems.

uLinga for Kafka dn Infrasoft logos

You can take advantage of this unique feature and aggregate all your NonStop logs to Kafka by using uLinga for Kafka from our partner Infrasoft. uLinga runs as a natively compiled Guardian (or OSS) process pair, and supports the Kafka communications protocols directly over TCP/IP. It enables NonStop applications to integrate easily with Kafka Clusters. By using uLinga, you can publish NonStop logs to Kafka in any one of these three simple methods below.


Method 1: uLinga File Reader Support

uLinga for Kafka performs an “end-of-file chase” whereby new records are read from the entry-sequenced file as they are written by the application, and immediately streamed to the Kafka cluster. So just configure uLinga to monitor one of your log files, and new records will automatically be published to Kafka.


uLinga File Reader Support chart

Method 2: uLinga Pathsend and Guardian IPC Support

Applications can also explicitly send data via the Pathsend and IPC interfaces provided by uLinga for Kafka. A Pathsend client, such as a Pathway Requestor or Pathway Server, simply sends the relevant data via a Pathsend request to uLinga. Just send your application logs to uLinga process to be published.

uLinga Pathsend and Guardian IPC Support chart

Method 3: Leverage LogWatch

Our TIC LogWatch product monitors application log files to provide real time detection of errors and raise alerts to facilitate quick resolution. It is a TIC product that watches log files for new entries and examines the content for a configured error pattern. When an anomaly is detected, LogWatch raises alert to notify team members via email, text, VHS or file records. By using LogWatch and uLinga, you can automate sending all your different NonStop logs easily.


LogWatch, uLinga chart

Conclusion

By using uLinga for Kafka, you can easily aggregate all your NonStop logs to Kafka in any one of these three simple methods. This will enable you to take advantage of Kafka's powerful features for centralized log management and real-time monitoring.


Visit our website to learn more about uLinga for Kafka and LogWatch.


Read More Blogs:

NonStop Integration – LogWatch and uLinga

Kafka and HPE NonStop - A Perfect Combo

 
Info logo

Feedback please

Do you find this tutorial blog helpful? Let us know what you think, and how we can make it even better. Don’t forget, you can subscribe to our blogs to get automatic email notification when a new blog is available.

 
Phil Ly Profle image

Phil Ly is the president and founder of TIC Software, a New York-based company specializing in software and services that integrate NonStop with the latest technologies, including Web Services, .NET and Java. Prior to founding TIC in 1983, Phil worked for Tandem Computer in technical support and software development.

Categories

Archive

bottom of page