Lead Data Engineer (Remote, Global)

Job summary

The role involves architecting and evolving a comprehensive data platform to support advanced analytics and AI-driven automation. The Lead Data Engineer will be responsible for shaping the data ecosystem and ensuring it meets enterprise standards for data architecture and governance.

Qualifications

• 10–15+ years of experience in building and evolving enterprise-scale distributed data platforms.
• Proven ability to influence architectural decisions and promote collaboration across teams.
• Experience with cloud platforms, preferably Azure, but other clouds are acceptable.
• Familiarity with batch and streaming technologies such as Spark, Kafka, and Flink.
• Knowledge of semantic modeling, embeddings, and knowledge graphs.
• Strong understanding of data governance and observability frameworks.
• Proficiency in programming languages such as Python, Java, or Scala, along with SQL optimization skills.
• Excellent communication skills for executive-level engagement and stakeholder alignment.

Responsibilities

• Design and enhance the end-to-end data platform architecture, integrating various data types and products.
• Establish data and architecture standards, ensuring compliance with privacy and governance requirements.
• Define patterns for AI/ML enablement and real-time data features.
• Lead and mentor cross-functional teams to align on architectural roadmaps.

Skills

• Advanced knowledge of data governance, including quality frameworks and compliance.
• Experience with Microsoft Fabric, ADLS, and Git-based CI/CD processes.
• Familiarity with RBAC/ABAC and data masking techniques.

Education

• Relevant degree in Computer Science, Engineering, or a related field is preferred but not explicitly required.

Tools

• Experience with Microsoft Fabric, Azure services, and data processing frameworks like Spark and Kafka.