Senior Data Engineer

Developers/Programmers (Information & Communication Technology)

We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic data team. As the most senior data engineer, you will play a pivotal role in supporting the Data Lead to build and maintain the infrastructure that enables the creation and distribution of insights and analytics throughout the business. Your responsibilities will include implementing an end-to-end analytics and data platform (Databricks/Microsoft Fabric) as well as integration with Microsoft Dynamics 365 (specifically the Customer Insights Module i.e. their CDP). You will also be responsible for modelling data across multiple sources and understanding business concepts to deliver effective data solutions. This role would suit someone with an ex-consulting background or someone interested in marrying their deep technical depth and breadth with a strong sense of commerciality & business acumen.

Key Responsibilities:

• Modernise the technological infrastructure by implementing an end-to-end analytics and data platform (Databricks/Microsoft Fabric) as well as integration with critical business applications (Microsoft Dynamics 365).
• Develop and maintain ETL processes that ingest data from various sources such as SQL Server, Salesforce, and other source systems into Azure Fabric's OneLake or Databricks.
• Create and optimise data models, data pipelines, and data processing workflows using Azure Fabric/Databricks.
• Develop and maintain documentation for data pipelines, models, and processes to ensure clarity and maintainability.
• Implement and manage CI/CD pipelines for data engineering tasks to enhance deployment efficiency and reliability.
• Support and maintain data warehouse and data lake solutions, ensuring scalability and performance.
• Provide database administration, ensuring data integrity, performance tuning, governance, quality, and security.
• Assess the viability of, and potentially execute on, migrating existing SQL Server databases into the modernised analytics and data platform.
• Collaborate with data scientists and analysts to automate the distribution of insights and predictions through Power BI reports and integration with Dynamics 365.
• Act as a technical mentor to junior team members, fostering a collaborative and innovative team environment.
• Communicate effectively with various business units to understand their data needs and deliver appropriate solutions.
• Stay updated with emerging technologies and industry trends, particularly in AI/ML, to integrate innovative solutions into the data architecture.

Technical Requirements:

• Extensive hands-on experience in ETL development, report development, database design, development, and administration.
• Strong expertise in data modelling and data engineering with Microsoft Fabric or Databricks, including data pipeline development, data integration, database management, data processing, and optimisation.
• Proficiency in programming languages such as SQL, Spark SQL, Python, PySpark, Spark (Scala), Power Query M, C#, JAVA.
• Extensive experience with CI/CD tools and practices for data engineering (Azure DevOps).
• Experience with containerisation/orchestration technologies.
• Proficient in implementing data governance frameworks and relevant software (MS Purview).
• Understanding of machine learning concepts and experience working with data scientists to operationalise ML models.
• Ability to model data across multiple sources and understand business concepts to deliver effective data solutions.
• Strong communication skills to interact with various business units and understand their data needs.

 Preferred Qualifications:

• Advanced degree in Computer Science, Data Science, Information Technology, or a related field.
• Certifications in Azure or Databricks.
• Experience in the financial services or technology sectors.
• Familiarity with other cloud platforms and tools (e.g., AWS, GCP).

Join us in transforming data into actionable insights and driving business success with cutting-edge technology and innovative data solutions

At NobleOak, you’ll be more than just an employee, you’ll be a NobleOaker

NobleOakers are a close-knit group of motivated and passionate people. We work as one team, support each other and do the right thing by our customers, with nobility at the core of our culture. You will feel truly valued here at NobleOak. We provide regular recognition, the opportunity to develop, and a fun atmosphere where we celebrate our wins together.

One Team, One Purpose

We are group of people with a diverse range of skills and backgrounds, working collectively as one united team. We pride ourselves on our collaborative culture where everyone is willing to lend a helping hand, regardless of their role. NobleOakers from all areas of the business are committed to putting our customers first at all times.

Stay Connected

We take every opportunity to stay connected with each department, hear from one another and maintain focus on our objectives. Our regular company-wide "huddles" are just one of our initiatives where we get together to share results, provide business updates and enjoy ourselves.

We show appreciation to our teammates who demonstrate NobleOak values through initiatives such as our Tough Nut Awards (Employee of the Month). Whilst we are busy striving to achieve results, we always take the time to celebrate our wins through regular drinks, lunches, and social events.

From the moment you join NobleOak, there is an abundance of training and support provided to set you up for success. We will enhance your personal and professional growth through tailored development plans, our NobleOak Academy of Learning & Development and external training opportunities.

At NobleOak, you’ll be more than just an employee, you’ll be a NobleOaker

NobleOakers are a close-knit group of motivated and passionate people. We work as one team, support each other and do the right thing by our customers, with nobility at the core of our culture. You will feel truly valued here at NobleOak. We provide regular recognition, the opportunity to develop, and a fun atmosphere where we celebrate our wins together.

One Team, One Purpose

We are group of people with a diverse range of skills and backgrounds, working collectively as one united team. We pride ourselves on our collaborative culture where everyone is willing to lend a helping hand, regardless of their role. NobleOakers from all areas of the business are committed to putting our customers first at all times.

Stay Connected

We take every opportunity to stay connected with each department, hear from one another and maintain focus on our objectives. Our regular company-wide "huddles" are just one of our initiatives where we get together to share results, provide business updates and enjoy ourselves.

We show appreciation to our teammates who demonstrate NobleOak values through initiatives such as our Tough Nut Awards (Employee of the Month). Whilst we are busy striving to achieve results, we always take the time to celebrate our wins through regular drinks, lunches, and social events.

From the moment you join NobleOak, there is an abundance of training and support provided to set you up for success. We will enhance your personal and professional growth through tailored development plans, our NobleOak Academy of Learning & Development and external training opportunities.

Don’t provide your bank or credit card details when applying for jobs.