Data Engineer

Mayfair IT Consultancy · London, United Kingdom
Location
London, United Kingdom
Type
Full-time
Seniority
Senior
Salary
Not stated
Posted
18 Nov 2026

Verified open 19 Nov 2026 · Source: Company careers page

About the role

OverviewWe are seeking experienced Data Engineers to join a major UK Government data programme, building and improving large-scale data solutions.You will design, develop and maintain reliable ETL/ELT pipelines, working with Python, PySpark, Cloudera Data Platform and Google Cloud Platform (GCP).This is an opportunity to solve complex data challenges, improve pipeline performance and contribute to modern data engineering practices.Key ResponsibilitiesDesign, build and maintain scalable ETL/ELT data pipelines.Extract, transform and load data into strategic platforms.Profile data, identify quality issues and support data validation.Develop reusable Python packages and engineering components.Optimise queries, data processing and pipeline performance.Improve automation and data engineering processes.Work with technical teams and business stakeholders.Apply engineering standards, governance and security requirements.Evaluate new tools and technologies and share best practice.Essential Skills & ExperienceProven experience as a Data Engineer or in a similar role.Strong Python programming skills.Hands-on ETL/ELT pipeline development.Experience with PySpark and large-scale data processing.Cloudera Data Platform or comparable enterprise data platforms.GCP or equivalent cloud data environments.Knowledge of SQL, data profiling and data quality.Experience optimising pipelines and query performance.Understanding of data governance, security and engineering standards.Strong problem-solving and stakeholder collaboration skills.Technical Skills & ToolsCore TechnologiesPythonPySparkETL / ELT pipeline developmentCloudera Data PlatformGoogle Cloud Platform (GCP)Additional Platforms & CapabilitiesSQL and query optimisationLarge-scale data processingData profiling and quality managementData integration and transformationReusable software packagesCloud data platformsPipeline monitoring and automationCI/CD and version controlData governance and securityExperience across every listed technology is not essential. Strong transferable data engineering experience is welcomed.SFIA Level ExpectationsIndependently designs and delivers data engineering solutions.Applies sound technical judgement to complex challenges.Builds scalable, reusable and maintainable pipelines.Identifies and resolves data quality and performance issues.Recommends improvements to tools and processes.Works effectively with technical teams and stakeholders.Shares knowledge and promotes engineering best practice.DesirableUK Government or public-sector experience.Experience with large-scale or complex datasets.Strong Cloudera Data Platform experience.GCP data engineering experience.Development of reusable Python packages.Experience improving enterprise data pipelines.Knowledge of data governance and compliance.Contribution to engineering standards or communities.Clearance RequirementSC Security Clearance is required for this assignment.Candidates must be able to confirm their current clearance status and meet the required screening and onboarding requirements.

Similar jobs in London at other employers

More jobs at Mayfair IT Consultancy

Browse similar jobs