- Company Name
- Exalto Consulting
- Job Title
- Senior DataOps Engineer (Active SC Clearance)
- Job Description
-
**Job Title**
Senior DataOps Engineer (Active SC Clearance)
**Role Summary**
Lead the design, implementation, and maintenance of enterprise data pipelines and infrastructure. Drive automation, data quality, and monitoring across multi‑cloud environments while ensuring compliance with security and governance standards.
**Expectations**
- Operate with active UK SC Clearance and UK residency.
- Deliver scalable, reliable data solutions that support business analytics and AI initiatives.
- Collaborate cross‑functionally with data scientists, engineers, and business stakeholders.
- Adhere to best practices for data security, privacy, and governance.
- Provide technical mentorship to junior team members.
**Key Responsibilities**
- Architect and develop end‑to‑end data pipelines using orchestration tools (Airflow, Dagster).
- Build and maintain containerized services (Docker, Kubernetes) for data processing workloads.
- Implement data quality frameworks and automated monitoring (Grafana, Prometheus).
- Manage data ingestion from varied sources (APIs, streaming, batch) into cloud storage (S3, GCS, Azure Blob).
- Optimize performance and cost of data workflows.
- Ensure compliance with security policies, including encryption, access controls, and audit logging.
- Troubleshoot and resolve pipeline failures and performance bottlenecks.
- Document architecture, processes, and best‑practice guides.
**Required Skills**
- Strong programming in Python, SQL, and Bash.
- Experience with workflow orchestration (Airflow, Prefect, Dagster).
- Proficiency in containerization (Docker, Kubernetes).
- Familiarity with cloud data services (AWS Glue, GCP BigQuery, Azure Data Factory).
- Knowledge of data lake and data warehouse concepts (Snowflake, Redshift, BigQuery).
- Expertise in monitoring and logging (Prometheus, Grafana, ELK).
- Solid understanding of security, compliance, and data governance frameworks.
- Excellent problem‑solving and communication skills.
**Required Education & Certifications**
- Bachelor’s degree in Computer Science, Engineering, or related field (or equivalent experience).
- Current UK SC Clearance (Active).
- Optional certifications: AWS Certified Data Analytics – Specialty, GCP Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate.