uLinga For Kafka. A Different Approach
Modernization Through Integration
Apache Kafka is an open-source distributed events streaming platform used by many corporations for data integration, streaming analytics, log analysis and other mission critical applications in a high performance and fault-tolerant environment.
Now, with uLinga for Kafka from Infrasoft, a NonStop application can publish NonStop events and data to Kafka clusters to be picked up by any consuming subscribers on any platforms.
uLinga for Kafka is a NonStop based solution and requires no additional software or hardware components.
uLinga for Kafka allows your NonStop applications and data sources to be completely integrated with Kafka. uLinga takes a unique approach to Kafka integration: it runs as a natively compiled Guardian process pair, and supports the Kafka communications protocols directly over TCP/IP. This removes the need for Java libraries or intermediate databases, providing the best possible performance on NonStop. It also allows uLinga for Kafka to directly communicate with the Kafka cluster, getting streamed data across as quickly and reliably as possible.
​
uLinga for Kafka supports a wide range of access points to enable NonStop users to stream data to and from Kafka. These include Inter-Process Communication (IPC), Pathsend, HTTP/REST and a number of file-oriented approaches. IPC and Pathsend interfaces allow NonStop applications to open uLinga for Kafka and send data via Guardian IPC, or via a Pathsend message. uLinga for Kafka also supports TMF audit trails, allowing for any TMF-protected files to be monitored and changes to those files sent to Kafka. The FILEREADER/FILEWRITER option allows entry-sequenced Enscribe files to be a source (and destination) for Kafka data. Finally, the uLinga Enscribe Intercept Library captures any changes to other types of Enscribe files, including relative and key-sequenced.
Some of the features recently added to uLinga for Kafka include:
​
-
Kafka Consumer support, allowing for data to be streamed from Kafka and sent to a NonStop application or file. (Add Connection link if you think that is helpful).
-
A new Enscribe Intercept Library, meaning that changes to all types of Enscribe files can now be captured, and forwarded to Kafka, without application changes.
-
TMF Audit Trail support, meaning that any changes to TMF-protected Enscribe files and SQL/MP tables can be streamed to Kafka, without application changes.
-
Userexit Support, allowing for custom process and data manipulation, at any point in the uLinga for Kafka processing path, for both Kafka Producer and Kafka Consumer functionality. (Add Connection link if you think that is helpful)
​
Kafka provides these functionalities and benefits to its users:
​
-
Publish and subscribe to data.
-
Store data in the order they were generated efficiently.
-
Real-time / On-the-fly processing of data.
-
Built-in fault tolerance and high performance
-
Kafka Consumer support: Allowing data to be streamed from Kafka and sent to a NonStop application or file. Read article.
-
Enscribe Intercept Library: Changes to all types of Enscribe files can now be captured, and forwarded to Kafka, without application changes.
-
TMF Audit Trail support: Any changes to TMF-protected Enscribe files and SQL/MP tables can be streamed to Kafka, without application changes.
-
User Exit support: Allowing for custom process and data manipulation, at any point in the uLinga for Kafka processing path, for both Kafka Producer and Kafka Consumer functionality. Read article.
-
OAUTH2.0/JWT support
​
Sample NonStop Use Cases:
​
-
Publish NonStop application logs or system events to Kafka to be real time analysis by off platform tools such as Splunk
-
Push NonStop data changes to other database platforms such as SQL Server or Oracle
-
Asynchronous messaging interface to applications on other platforms such as Windows or Linux
-
Publish NonStop applications or data to stay in sync with Cloud Services
Three Easy Ways to Publish to Kafka
-
Publish an event to Kafka can be done simply by a Pathsend or WRITE using uLinga IPCSERVER.
-
Capture data changes to disk and push data to Kafka easily using uLinga FILEREADER.
-
Route web requests to Kafka easily using uLinga’s HTTP Interface.
Online Configuration
Web browser interface utilising HTML 5 and Web Sockets, with no client software to install.
Online Management
Interactive command line interface, with support for obey/script files.
Tracing
Built in tracing facility & standard logging support.
Unparalleled Reliability
Support for Kafka Exactly Once Semantics (EOS) along with traditional NonStop fault-tolerance ensures that every transaction is sent to Kafka.
Connections
Native Kafka protocol support within uLinga for Kafka allows for highest performance and reliability
Standard APIs
Support for standard APIs and file system: IPC and Pathsend, HTTP, and Enscribe files allow for a range of access methods.
Articles
Blog: How to Aggregate NonStop Logs to Kafka in One Simple Step
Apache Kafka is a powerful streaming platform that has become a popular choice for log aggregation due to its ability to handle large volumes of data with low latency and high throughput. It provides a reliable and scalable infrastructure for collecting, storing, and processing log events from various sources, enabling centralized log management and real-time monitoring.
Blog: Kafka and HPE NonStop – A Perfect Combo!
One of the best ways to modernize NonStop is to integrate its applications with other software, platforms and cloud services. This interoperability is extremely important as it will extend the value of the NonStop applications, and underscore the flexibility of the platform in the enterprise.
Blog: NonStop Integration – LogWatch and uLinga
Integration with Kafka will open NonStop to interoperate with many different platforms and applications. uLinga for Kafka from Infrasoft provides the perfect mechanism to integrate NonStop applications with Kafka.