Job Description
Key Responsibilities
- Develop and maintain infrastructure automation, tools, and applications for cloud-based data platforms
- Support and optimize AWS services including EMR, Glue, and S3 for ingestion and analytics workloads
- Manage and monitor operational health, performance, and upgrade cycles of data platforms
- Work with data lake architectures, supporting ingestion (Zone 2 – Curated) and derived datasets (Zone 3 – Derived)
- Implement and manage CI/CD pipelines and infrastructure as code
- Perform incident management including triage, troubleshooting, and resolution
- Support data operations such as reruns, recovery, and schema-level changes in controlled environments
- Collaborate with cross-functional teams including cloud operations, delivery teams, and platform partners
- Drive cost optimization and performance improvements using metrics and data insights
- Contribute to next-generation platform transitions (e.g., EMR to Snowflake/Iceberg pipelines)
- Identify, escalate, and remediate cloud risks and vulnerabilities
Required Skills & Qualifications
Core Technical Skills
- Strong programming experience in Python (primary), Bash/Shell scripting, and SQL
- Hands-on experience with AWS services: EMR, EC2, Glue, Lambda, S3, SQS, SNS, CloudFormation
- Experience with Data Lake architectures and related technologies
- Knowledge of Snowflake and Apache Iceberg
- Understanding of Glue Catalog and Hive Metastore
- Familiarity with Data Mesh architecture
- Strong Linux system administration skills
DevOps & Automation
- Experience with CI/CD pipelines
- Infrastructure automation and scripting expertise
Operational & Support Skills
- Production support experience: incident management, diagnostics, escalation
- Experience with monitoring and alerting tools
- Understanding of data governance, security, and access controls (IAM)
Additional Competencies
- Strong problem-solving and debugging skills
- Excellent communication and stakeholder management abilities
- Ability to work independently and collaboratively
- Experience mentoring team members and handling escalations
- Proactive mindset with focus on continuous improvement and innovation