- Company Name
- Dew Software
- Job Title
- Data Engineering Lead
- Job Description
-
**Job Title:** Data Engineering Lead
**Role Summary:**
Lead end‑to‑end design, implementation, and optimization of scalable data pipelines, data warehouses, and data platform solutions. Provide technical guidance and mentorship to cross‑functional engineering teams while ensuring data quality, governance, and performance standards are met.
**Expectations:**
- 8–10 years of data engineering experience, with proven leadership in designing large‑scale pipelines.
- Mastery of Python, PySpark, SQL, and cloud data warehouse platforms (Snowflake, Redshift, BigQuery, or Synapse).
- Extensive hands‑on experience with ETL and data integration tools (Informatica, Talend, Azure Data Factory, DataStage).
- Demonstrated ability to implement CI/CD, version control, monitoring, and best practices for data engineering.
- Strong knowledge of data quality frameworks, governance, security, and privacy.
**Key Responsibilities:**
- Architect and implement batch and real‑time ETL workflows on cloud platforms.
- Develop and optimize data models, schemas, and warehouse solutions in Snowflake or equivalent.
- Design efficient PySpark/Python transformations for structured and semi‑structured data.
- Integrate data from on‑prem, cloud, and API sources using ETL tools.
- Define and enforce data quality checks, validation frameworks, and exception handling.
- Optimize storage, partitioning, and query performance to meet SLAs.
- Establish and promote best practices for CI/CD, version control (Git, Jenkins), monitoring, and documentation.
- Conduct code reviews, provide technical leadership, and mentor junior engineers.
- Ensure compliance with data governance, security, and privacy requirements.
**Required Skills:**
- Advanced Python, PySpark, and SQL.
- Snowflake (or Redshift, BigQuery, Synapse) data warehousing expertise.
- Proficiency with ETL tools: Informatica, Talend, Azure Data Factory, DataStage.
- Cloud experience: AWS, Azure, or GCP.
- Data quality, validation, and governance frameworks.
- Data modeling, performance tuning, and query optimization.
- CI/CD & version control (Git, Jenkins).
- Strong analytical, communication, and leadership abilities.
**Required Education & Certifications:**
- Bachelor’s or Master’s degree in Computer Science, Information Systems, or related discipline.
- Certifications in Snowflake, AWS/GCP/Azure Data Engineering, or relevant ETL tools are a plus.