Job Description

Senior Data Engineer - Elastic Stack / Apache NiFi

📍 Worcester (Onsite)

💰 £70-£75 per hour | Outside IR35

📅 12-Month Rolling Contract

⏰ Compressed Working Available - Work 3 Long Days, Paid for 5

We are seeking an experienced Senior Data Engineer to join a high-profile programme delivering secure, scalable data platforms within a highly regulated environment. This is an exciting opportunity to work on complex data engineering challenges using cutting-edge technologies, with the flexibility of compressed working patterns.

The Opportunity

As a Senior Data Engineer, you will play a key role in designing, building, and maintaining robust data pipelines and platforms. You will work closely with architects, engineers, and security teams to deliver high-performance data solutions that meet stringent security and governance requirements.

This role is fully onsite in Worcester, with the option to work a compressed schedule (e.g. three extended working days while being compensated for a full five-day week).

Key Responsibilities

Design, develop, and maintain secure, scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi

Build and manage data ingestion, transformation, and integration processes

Ensure data quality, integrity, security, and compliance with governance standards

Collaborate with data architects and security teams within highly secure environments

Monitor and manage large-scale, real-time data flows, optimising performance and reliability

Develop robust data models to support analytics and reporting capabilities

Create dashboards and visualisations using Kibana to drive operational insights

Troubleshoot, debug, and tune data pipelines and platform performance

Implement high availability, backup, and disaster recovery strategies

Produce clear technical documentation, including architecture, workflows, and security protocols

Essential Skills & Experience

Strong experience with the Elastic Stack (Elasticsearch, Logstash, Kibana)

Proven expertise with Apache NiFi

Extensive experience designing and building enterprise-scale data pipelines

Strong understanding of secure data architectures and data governance

Experience operating within regulated or security-conscious environments

Ability to design and deliver end-to-end data engineering solutions

Excellent troubleshooting and performance optimisation skills

Desirable Experience

Experience working within government, defence, or other highly regulated sectors

Knowledge of Apache Kafka, Spark, or Hadoop

Experience with Docker and Kubernetes

Familiarity with monitoring and observability tools such as Prometheus, Grafana, or ELK

Understanding of machine learning algorithms and their implementation

Experience with Infrastructure as Code tools such as Terraform and Ansible

What's on Offer?

✅ £70-£75 per hour

✅ Outside IR35

✅ 12-month rolling contract

✅ Compressed working available (3 long days, paid for 5)

✅ Opportunity to work on complex, mission-critical data platforms

✅ Long-term contract stability within a high-profile programme

If you're an experienced Data Engineer with strong Elastic Stack and Apache NiFi expertise and are looking for a long-term, high-impact contract opportunity, we'd like to hear from you