Background

During my internship at eBay, I was tasked with creating a consumer application that could effectively detect breaks while processing incoming events in an event-driven distributed architecture. The goal was to manage the processing of 120M daily events and track their lifecycles within a complex distributed system by mapping all possible event paths. This was crucial for ensuring accurate payment processing and maintaining customer satisfaction.

Problem

The main challenge we faced was developing an application that could handle the high volume of daily events while accurately tracking their lifecycles. We needed to ensure that the application could effectively detect any breaks in the process and quickly resolve them to prevent any delays or errors in payment processing. Additionally, we had to create a solution that would work seamlessly within the existing distributed architecture.

Solution

To address these challenges, we developed a consumer application using Java and Apache Kafka as our messaging system. We used Kafka Streams API to process incoming events and track their lifecycles within the distributed system. We also implemented a custom monitoring system that allowed us to quickly identify any breaks in the process and resolve them before they caused any issues with payment processing.

Impact

Improved accuracy of payment processing by X%
Reduced downtime due to breaks in event processing by X%
Increase in customer satisfaction due to faster payment processing times

Team

Amy Nguyen - Software Engineer
- Oversaw development of consumer application using Java and Apache Kafka
Daniel Lee - Data Analyst
- Analyzed data from monitoring system to identify potential issues and improve overall performance of the application
Jessica Kim - Project Manager
- Oversaw project timeline, budget, and communication with stakeholders throughout development process
Kevin Patel - Quality Assurance Engineer
- Developed testing plan for consumer application and monitored performance during testing phase

Java Kafka App Detects Breaks in Event-Driven System