- Company Name
- Oliver James
- Job Title
- Senior Azure Data Engineer (Databricks)
- Job Description
-
**Job Title:** Senior Azure Data Engineer (Databricks)
**Role Summary:**
Lead the design, implementation, and optimization of enterprise‑grade data pipelines and analytics platforms using Azure services and Databricks. Drive data architecture strategy, ensure data quality, security, and performance, and mentor junior staff while collaborating with cross‑functional analytics teams.
**Expectations:**
- Own end‑to‑end data solutions from ingestion to serving
- Continuously improve pipeline reliability, scalability, and maintainability
- Champion CI/CD best practices for data workflows
- Champion data governance, security, and quality standards
**Key Responsibilities:**
- Design, build, and refactor Azure data pipelines (Data Lake, Data Factory, Synapse, Functions)
- Develop Spark/Databricks workloads (Delta Lake notebooks, jobs, and pipelines) in Python and/or Scala
- Architect scalable, high‑performance data processing solutions for production workloads
- Implement CI/CD pipelines using Git, Azure DevOps, or equivalent tools
- Collaborate with Data Scientists, Analysts, and Architects to translate business requirements into technical solutions
- Mentor junior engineers and provide technical guidance
- Maintain and enhance data platform documentation
**Required Skills:**
- 5+ years as a Data Engineer
- Deep Azure expertise: Data Lake, Data Factory, Synapse, Function Apps, and related services
- Hands‑on experience with Azure Databricks (Spark, Delta Lake, notebooks, job scheduling)
- Proficient in Python and/or Scala for data processing
- Strong understanding of CI/CD, Git workflows, orchestration, and best practices
- Experience delivering production‑grade data solutions with monitoring and troubleshooting
- Excellent problem‑solving, communication, and collaboration abilities
**Required Education & Certifications:**
- Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent practical experience)
- Optional: Azure Data Engineer Associate (DP-200/DP-201) or Databricks certification preferred but not mandatory.