Data Engineering Lead

Closing date

21st August 2026

What is the purpose of this job?

As the Data Engineering Lead, you will provide senior technical leadership and oversight for oversee Rail Delivery Group’s current AWS-based data platform, ensuring it is robust, scalable and able to support RDG and wider rail-industry needs. You will act as RDG’s principal expert in data engineering, guiding major technical decisions and enabling teams across RDG to deliver high-quality, reliable and efficient analysis and insight.

You will work closely with the Head of Data, Analysis and Insights and the Head of Architecture to ensure the data platform evolves appropriately, supporting increasing volume, complexity and integration demands across the organisation and its partners helping to establish the RDG Data warehouse as a central part of GBR’s data vision. The role also calls for strong leadership skills to mentor and support data engineers at all levels, influencing how data engineering evolves within GBR.

The technologies in use include Redshift, EC2, Glue, Lambda, Kinesis Firehose, Athena, and S3. Proficiency in Python, SQL, and Bash is essential. You’ll encounter a wide range of source systems, from legacy technology patterns to modern solutions, working with both fixed-width files and contemporary REST APIs, as well as managing batch and streaming data workflows. To further develop RDG’s DataOps practice, you should be comfortable with both the technical tooling and process aspects of DevOps.

What can I expect to do in this job? 

This isn’t an exhaustive list, but things you can expect to be involved with include:

• Provide authoritative technical guidance on data engineering design and tools across RDG.
• Lead the development, operation and continuous improvement of RDG’s data platform, ensuring high availability, resilience and performance.
• Oversee the implementation of complex data pipelines, integrations and transformations across batch and streaming workloads.
• Make key decisions on data models, storage approaches, tooling and engineering techniques to ensure efficient and sustainable solutions.
• Lead, mentor and develop a team of data engineers, setting clear expectations around engineering quality and operational excellence.
• Support the establishment of consistent engineering practices, including code standards, testing and DataOps approaches.
• Provide guidance to engineers across other teams, acting as the organisation’s primary point of technical escalation for complex data engineering challenges.
• Work with Architecture, Information Security, Quality Assurance and other technical teams to ensure the data platform integrates effectively with wider systems, complies with necessary controls and promoting good data practice
• Collaborate with internal and external partners (including AWS, DfT, DfTO, GBRX and TOCs) to support complex integrations, data flows and platform interactions.
• Engage with business stakeholders to understand technical requirements and translate them into well-engineered solutions.
• Work with internal and external stakeholders to manage and prioritise the engineering workload and delivery.
• Support the Head of Data, Analysis and Insights with shaping the future role of data across RDG and the wider rail industry.
• Ensure appropriate controls, monitoring and operational processes are in place for all data engineering activities.
• Maintain and enhance engineering standards across data quality, testing, documentation and application of best practice.
• Work with Architecture/AWS team on data projects and other appropriate AWS initiatives.

Who will my key contacts be?

The role requires extensive collaboration with both internal and external teams. Some of those are listed below.

Internally to RDG:

• Data, Analysis and Insights
• Architecture
• Information Security
• Quality Assurance

Externally:

• AWS
• DfTO
• GBRX
• TOCs
• DfT
• Network Rail and the Customer & Revenue Growth team

Requirements

As Data Engineering Lead, you will bring proven experience working with extremely large datasets (including tables with over 85 billion records), writing efficient SQL, and building both batch and streaming pipelines in Python. You’ll have hands-on AWS expertise in a data context within the last three years, and a strong grasp of modern data storage and processing options, such as data lakes, warehouses, relational and key-value stores, and distributed frameworks like Spark. Experience with CI/CD and DataOps is essential, ideally including the development of pipelines and tooling.

You will champion best-practice data warehouse and data lake design, driving automation and high standards in code quality. You’ll lead by example, curious about data and eager to uncover insights

whilst unafraid to challenge and transform data where needed. Above all, they will be a team player who champions continuous improvement and personal development.

Benefits

Why Work for RDG?

We offer a highly competitive package, including:

• 75% discount on rail travel (for work and leisure), plus international rail discounts
• 30 Days annual leave (plus buy/sell options and additional leave for key life events)
• Two Personal Development Days are granted to employees annually
• Interest-free Season Ticket Loan for commuting costs
• Enhanced family leave – 30 weeks full pay for maternity, adoption, surrogacy, and shared parental leave
• Pension scheme – up to 8.64% employer contribution
• Subsidised Private Medical Insurance (Vitality PPP) and Health Cash Plan (Healthshield)
• Access to our flexible benefits portal - a central hub for accessing retail discounts, shopping, dining, travel, and entertainment perks
• Flexible working arrangements available