Data Engineer (m/w/d) Fokus ETL | Azure | Databricks | Kubernetes | docker

Leistungsbeschreibung

In this project,the Customer is developing and operating a machine learning tool for their network operators that can be used to detect faults in underground cables, local network stations, and gas pipelines at an early stage. This means that equipment can be replaced before it fails. This solution helps to ensure high network availability, thereby avoiding high costs for downtime and repairs.

Anforderungen

• Data Engineering tasks related to projects, e.g. Predictive Maintenance Solutions
• Help in supporting and designing business-critical data engineering use cases, from the business problem to delivery and operation. 
• Programming of data ingestion pipelines from various sources, e.g Graph Database or Data Lakehouse.
• Writing of production feature engineering code in Python and PySpark on a Databricks Tech-Stack.
• Build and maintain data pipelines for static, mixed, and time-series data.
• Design, implementation and maintenance of data infrastructure for ML algorithms in Azure Databricks.
• Responsible for the design and implementation of the CI/CD pipelines
• Data modeling and architecture (schemes, sources, optimizations)

• Good team player and communicator, but great at independent work. Solution oriented. Analytical thinking
• Experience in machine learning engineering, exploratory data analysis, and software development and writing of ETL-Pipelines. Optimally, candidate should have a degree in mathematics, physics, computer sciences or in a related field
• Experience in Python programming is mandatory, especially with PySpark, whereas XGBoost, Seaborn, Matplotlib and dbutils (Databricks) are nice-to-have
• Expertise in Git, Gitlab, and CI/CD are beneficial (including Azure CLI and Azure-Cloud specific APIs)
• Experience in working with Azure, Databricks, Kubernetes and Docker
• Candidate should be familiar or inclined to working in an agile environment. Prior experience with predictive maintenance tasks is a plus

Über den Auftraggeber

Start: ASAP

Ende: 31.12.2025; + Option auf Verlängerung

Auslastung: 16 h/week

Vertragsart: Contracting

Standort: Remote und ggf. Hamburg