- Company Name
- TheFork
- Job Title
- Senior Data Engineer
- Job Description
-
Job Title: Senior Data Engineer
Role Summary:
Lead the design, implementation, and continuous improvement of scalable, production‑grade data pipelines and a modern self‑service data platform. Drive the transition from a centralized BI model to a federated, domain‑oriented ecosystem, ensuring governance, quality, and performance standards across the organization.
Expactations:
- Deliver and maintain high‑availability data pipelines using best software engineering practices.
- Guide architectural decisions and governance frameworks for trusted data assets.
- Collaborate with DataOps, architects, BI, engineering, and data science teams to align platform capabilities with business needs.
- Champion cost‑aware design and continuous optimization of cloud data workloads.
Key Responsibilities:
- Design & implement scalable ELT pipelines in Snowflake/Databricks, employing medallion architecture and safe schema evolution.
- Lead Airflow DAG development, orchestration pattern evolution, and failure‑resilience strategies.
- Drive technical POCs for new tools, architectures, and performance enhancements.
- Establish data contracts, quality controls, and observability across pipelines.
- Embed lineage, metadata, and monitoring mechanisms into every workflow.
- Mentor and document platform standards, ensuring cross‑team adoption and developer experience.
- Work with DataOps on infrastructure, CI/CD, Terraform, and AWS services (S3, IAM, networking).
- Implement security‑by‑design principles in the cloud data environment.
- Optimize query performance and compute cost in cloud data warehouses.
Required Skills:
- ≥6 years of data engineering experience with production‑grade platforms.
- Deep hands‑on expertise in Snowflake, Databricks, or equivalent modern cloud lakehouses.
- Advanced Airflow, Airbyte, or similar ingestion framework skills.
- Strong analytical modeling (medallion), schema evolution, and incremental ELT design.
- Proficiency in MPP systems, distributed data processing, and query optimization.
- Experience with AWS data services, Infrastructure as Code (Terraform), and CI/CD for data workflows.
- Data governance, quality frameworks, contracts, and observability practices.
- Leadership and ability to drive technical standards across multiple stakeholder teams.
Required Education & Certifications:
- Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent practical experience).
- Prefer certifications in Snowflake, Databricks, AWS, Airflow, or Terraform, though not mandatory.