Back to search
We are seeking an experienced Data Engineer to design, build, and optimize our complex data pipelines and big data infrastructure. The ideal candidate will have deep expertise in architecting scalable data solutions, integrating machine learning workflows, and optimizing data processing systems. This role requires both technical excellence and the ability to influence architectural decisions that align with business objectives.
Key Responsibilities- Design and implement robust, scalable data pipelines for processing large volumes of structured and unstructured data
- Architect end-to-end data solutions that support machine learning model training and deployment
- Develop and maintain data infrastructure that ensures data quality, reliability, and accessibility
- Optimize existing data workflows for performance, cost-efficiency, and maintainability
- Contribute to strategic architectural decisions and technical roadmaps
- Implement data governance and security best practices across the data ecosystem
- Mentor junior engineers and promote best practices in data engineering
- 5+ years of experience in data engineering roles with progressively increasing responsibility
- Proven experience designing and implementing complex data pipelines at scale
- Strong knowledge of distributed computing frameworks (Spark, Hadoop ecosystem)
- Experience with cloud-based data platforms (AWS, Azure, GCP)
- Proficiency in data orchestration tools (Airflow, Prefect, Dagster, or similar)
- Solid programming skills in Python, Scala, or Java
- Experience integrating ML workflows into production data systems
- Strong understanding of data modeling, ETL processes, and database design
- Demonstrated ability to architect solutions for big data challenges
- Experience with real-time data processing (Kafka, Kinesis, Flink)
- Knowledge of containerization and infrastructure-as-code (Docker, Kubernetes, Terraform)
- Familiarity with MLOps practices and tools (MLflow, Kubeflow, etc.)
- Experience with data governance frameworks and data cataloging
- Understanding of graph databases and unstructured data processing
- Knowledge of advanced analytics techniques and statistical methods
- Experience with data mesh or data fabric architectural patterns
- Bachelor's degree in Computer Science, Data Science, or related field (Master's preferred)
- Relevant certifications in cloud platforms or data technologies
Please apply within for further details or call on
Alex Reeder
Harvey Nash Finance & Banking
