The Data Engineering team is looking for a software engineer who’s interested in designing and building libraries, frameworks and services for working with big data on a cloud platform.
The Data Engineering team delivers the building blocks that are used to build data pipelines on the Edison big data platform on AWS. This is a multi-tenancy platform that is currently driving the digital acceleration and AI advancement. It is deployed across five countries within the group, with Belgium operating on the largest scale.
Our product portfolio consists of data services and frameworks that helps data engineers and data scientists build data pipelines according to industry best practices. We support thousands of data pipelines built with these frameworks.
A summary of what we are building and maintaining:
A data processing library that provides the access to the data lake using different data processing technologies
A capability to move data off the platform or across AWS accounts
A plugin for data pipelines to make them GDPR compliant
Retention and cleanup capabilities that can be used to clean up the data lake
Data lineage for observability on how data flows through the platform
Expose metadata to the metadata-driven capabilities on the Edison big data platform
Best practices for big data processing and storage
We are looking for someone to design, implement and maintain new features for these existing products, as well as to contribute to the creation and onboarding of new products. Finally, collaboration with other Edison teams to align and integrate the Data Engineering products with the rest of the Edison product offering is essential.
You also need to be able to communicate with both technical and non-technical stakeholders and understand how the products we create and maintain deliver value to and impact the organization.
What’s in it for you?
You’ll get the opportunity to get hands-on experience with new technologies (including supporting AI workloads) on the cloud in a strong team. You’ll have a big impact in the big data domain, at the digital heart of the organization, and be able to work on high-leverage products in a fast-paced agile environment.
Responsibilities
Design and implement complex requirements for large scale software frameworks
Create AI code tools agents/skills/plugins/policies
Help resolve and coordinate cross-team functional and technical dependencies
Give input on framework design and architecture
Help maintain and support the complete set of our products
Requirements
Strong proficiency in (Python) programming, design patterns and best practices
Strong technical analysis and design skills
Knowledge of and experience with AI tools like Claude code or GitHub Copilot
Good communication skills
Value high quality software and releases
Ability to find the balance between tactical solutions and the strategical vision
Work well in team environment, as well as strong cross-team collaboration
Debugging skills
Ownership and decision-making
Mentoring
Experience level: medior-senior preferred (5-7 years)
Tech stack
Python
Cloud: AWS
Data processing frameworks (Spark, Pandas, Polars)
Claude code and GitHub Copilot
CICD (Jenkins, Bitbucket, GitHub, Concourse)
Scheduling: Airflow
