At TechBiz Global , we are providing recruitment service to our TOP clients from our portfolio. We are currently seeking an Data Engineer to join one of our clients ' teams. If you're looking for an exciting opportunity to grow in a innovative environment, this could be the perfect fit for you.
Requirements
Key Responsibilities:
- Design, develop, and maintain data ingestion pipelines using Kafka Connect and
Debezium for real-time and batch data integration.
- Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud
Storage (GCS), and BigQuery.
- Implement best practices for data modeling, schema evolution, and efficient partitioning
in the Bronze Layer.
- Ensure reliability, scalability, and monitoring of Kafka Connect clusters and connectors.
- Collaborate with cross-functional teams to understand source systems and downstream
data requirements.
- Optimize data ingestion processes for performance and cost efficiency.
- Contribute to automation and deployment scripts using Python and cloud-native tools.
- Stay updated with emerging data lake technologies such as Apache Hudi or Apache
Iceberg.
Required Skills and Qualifications:
- 5+ years of hands-on experience as a Data Engineer or similar role.
- Strong experience with Apache Kafka and Kafka Connect (sink and source
connectors).
- Experience with Debezium for change data capture (CDC) from RDBMS.
