Kafka Platform Engineer

Job Description Administer, support, and maintain Apache Kafka and Confluent Platform environments across enterprise production and non-production landscapes. Build, configure, deploy, and manage Kafka infrastructure including brokers, topics, partitions, replication, consumer groups, Kafka Connect, and Schema Registry. Provide L3 production support for mission-critical event streaming platforms and participate in 24×7 operational support and incident management activities.

Troubleshoot and resolve Kafka-related issues including consumer lag, broker failures, replication issues, throughput bottlenecks, topic misconfigurations, and connectivity problems. Support platform automation, provisioning, and operational excellence initiatives using Jenkins, Git, Python, Shell scripting, Ansible, and CI/CD pipelines. Perform infrastructure lifecycle management including installation, upgrades, patching, configuration management, capacity planning, and performance tuning.

Support cloud and containerized Kafka deployments using Docker, Kubernetes, OpenShift, AWS MSK, ECS, and Linux environments. Implement proactive monitoring and observability using Prometheus, Grafana, AppDynamics, Elastic Stack, Splunk, and enterprise monitoring tools. Manage platform security including TLS/SSL encryption, Kafka ACLs, RBAC, authentication mechanisms (LDAP, Kerberos, OAuth), and certificate lifecycle management.

Collaborate with application, infrastructure, network, database, and cybersecurity teams to enhance platform resiliency, scalability, compliance, and event-driven architecture adoption.

Preferred Qualifications Bachelor’s degree in computer science, Information Technology, Engineering, or related field. 5–10+ years of Middleware / Messaging Administration experience. Strong understanding of ITIL Incident, Problem, Change, and Release Management.

Experience in financial services or other highly regulated environments. Excellent troubleshooting, communication, stakeholder management, and documentation skills. Proven experience in automation, production stability, and operational transformation initiatives.

Areas of responsibility Gathering of requirements-Participate in gathering requirements for specific modules and ensure clear, structured documentation. Work across multiple business processes to capture linkages, dependencies, and required changes. Solution Design Facilitate creation of application solution designs for a specific module under the guidance of the Project Manager, ensuring alignment with business requirements and technical standards.

Coding and Configuration Review the developed codes, resolve technical queries of the team for the assigned module and create consistent replication patterns, to ensure that the developed code aligns with the project standards.Collaborate with project managers through all phases of software development life cycle. Perform root-cause analysis to identify and troubleshoot recurring technical issues. Modify software codes to resolve errors, adapt to new hardware and software, enhance performance, and upgrade interfaces.

Implementation Collaborate with technical teams to validate configurations, facilitate integration of new applications, and address dependencies during deployment. Prepare user training documents and related frameworks to ensure smooth adoption new applications and minimize transition risks.