Skills

Communication Data Governance MongoDB PostgreSQL CI/CD Kubernetes Monitoring Change Management Quality Management Architecture Architecture Strategy Data Architecture Spark

Job Specifications

Description:

Tasks:

Develop and update a data architecture strategy that adapts to evolving needs and accommodates both Business Intelligence and AI workloads.
Design and implement architectures for the cloud that are vendor agnostic.
Design a modern scalable data platform to replace a large legacy data system in a phased approach.
Align architectural decisions with data governance policies and the department’s vision on cloudification.
Establish and enforce data management policies and processes, including data quality, security and platform health monitoring.
Ensure regulatory compliance and adherence to audit requirements.
Provide guidance and mentorship to data analysts and data engineers.
Facilitate change management by guiding colleagues and users through the migration process.
Document and maintain data architecture and data assets in detail.
Assistance with deployment, configuration and testing of the system.
Participation in meetings with other project teams.

Skills Required:

Experience in migrating legacy data systems to a modern cloud-based, open-source data platform solution (preferably Data Lakehouse).
Excellent knowledge of designing scalable and flexible modern cloud-based and open-source data architectures.
Experience with AI-powered assistants like Amazon Q for innovative data solutions design.
Strong understanding of Kubernetes.
Knowledge of database systems, both relational (PostgreSQL, Oracle) and non-relational (Elasticsearch, MongoDB).
Experience with ETL/ELT processes and related data ingestion and transformation tools (like Spark, dbt, Trino).
Proficiency in data pipeline orchestration tools (like Airflow, Dagster, Luigi).
Knowledge of data governance frameworks and tools (like DataHub, Open Metadata), data quality management, data security, access control, and regulatory compliance.
Proficiency with system-to-system integration via RESTful APIs.
Experience with DevSecOps practices and tools related to data pipelines, including CI/CD for data infrastructure.
Excellent communication skills to articulate data architecture concepts to technical and non-technical stakeholders. Good knowledge of modeling tools.

About the Company

We focus on delivering high-performance and premium services to diverse organizations. ThoughtLabs® is a team of software engineers, programmers, technology experts and technical recruiters who work together to provide our clients with outstanding services. What we do? • IT Consultation • IT Staffing • Web Development • Mobile App Development • IT Consultation In this high-tech and digitalized world, you may be stuck in certain things. You need to analyze your current business practices, approach and standings in tec... Know more