Job Description:
We are seeking a highly skilled Kafka Architect with expertise in real-time streaming architectures and GCP Pub/Sub. The ideal candidate will have deep knowledge of Kafka, Kafka Streams, Kafka Connect, Spark Streaming, Schema Registry, Flink, and Beam, along with hands-on experience in GCP architecture and Pub/Sub.
Key Responsibilities:
* Design and architect real-time data streaming solutions using Kafka and GCP Pub/Sub.
* Develop and optimize Kafka-based streaming pipelines using Kafka Streams, Kafka Connect, Spark Streaming, Flink, and Beam.
* Implement schema management using Schema Registry to ensure data consistency.
* Collaborate with cloud architects and engineering teams to deploy and scale streaming applications on Google Cloud Platform (GCP).
* Ensure high availability, fault tolerance, and security of Kafka-based architectures.
* Optimize event-driven architectures for low-latency, high-throughput applications.
* Define best practices and governance for managing Kafka clusters and streaming pipelines.
Must-Have Skills:
* Strong architectural knowledge of Kafka, Kafka Streams, Kafka Connect, and Schema Registry.
* Experience with real-time data processing frameworks like Spark Streaming, Flink, and Beam.
* Deep expertise in GCP architecture with hands-on experience in GCP Pub/Sub.
* Solid understanding of event-driven architectures and distributed data processing.
* Strong problem-solving skills and the ability to troubleshoot streaming pipeline issues.
#J-18808-Ljbffr