How to Aggregate NonStop Logs to Kafka in One Simple Step

Apache Kafka is a powerful streaming platform that has become a popular choice for log aggregation due to its ability to handle large volumes of data with low latency and high throughput. It provides a reliable and scalable infrastructure for collecting, storing, and processing log events from various sources, enabling centralized log management and real-time monitoring. Kafka's strengths in distributed data streaming, durability, and fault tolerance make it well-suited for handling the demanding requirements of log aggregation in modern distributed systems.

You can take advantage of this unique feature and aggregate all your NonStop logs to Kafka by using uLinga for Kafka from our partner Infrasoft. uLinga runs as a natively compiled Guardian (or OSS) process pair, and supports the Kafka communications protocols directly over TCP/IP. It enables NonStop applications to integrate easily with Kafka Clusters. By using uLinga, you can publish NonStop logs to Kafka in any one of these three simple methods below.

Method 1: uLinga File Reader Support

uLinga for Kafka performs an “end-of-file chase” whereby new records are read from the entry-sequenced file as they are written by the application, and immediately streamed to the Kafka cluster. So just configure uLinga to monitor one of your log files, and new records will automatically be published to Kafka.

Method 2: uLinga Pathsend and Guardian IPC Support

Applications can also explicitly send data via the Pathsend and IPC interfaces provided by uLinga for Kafka. A Pathsend client, such as a Pathway Requestor or Pathway Server, simply sends the relevant data via a Pathsend request to uLinga. Just send your application logs to uLinga process to be published.

Method 3: Leverage LogWatch

Our TIC LogWatch product monitors application log files to provide real time detection of errors and raise alerts to facilitate quick resolution. It is a TIC product that watches log files for new entries and examines the content for a configured error pattern. When an anomaly is detected, LogWatch raises alert to notify team members via email, text, VHS or file records. By using LogWatch and uLinga, you can automate sending all your different NonStop logs easily.

Conclusion

By using uLinga for Kafka, you can easily aggregate all your NonStop logs to Kafka in any one of these three simple methods. This will enable you to take advantage of Kafka's powerful features for centralized log management and real-time monitoring.

Visit our website to learn more about uLinga for Kafka and LogWatch.

Read More Blogs:

NonStop Integration – LogWatch and uLinga

Kafka and HPE NonStop - A Perfect Combo

---

Feedback please

Do you find this tutorial blog helpful? Let us know what you think, and how we can make it even better. Don’t forget, you can subscribe to our blogs to get automatic email notification when a new blog is available.

---

Phil Ly is the president of TIC Software, a leading provider of software and services for the NonStop community focused on NonStop application modernization. Under Phil's guidance, TIC Software has a proven track record of helping clients seamlessly integrate NonStop with next-generation technologies. A recognized technical leader, Phil is passionate about empowering organizations to unlock the full potential and longevity of their NonStop platform through innovative solutions.