Lead Data Engineer

Vor 3 Tagen


Sydney, Österreich Talenza Vollzeit

Other (Information & Communication Technology)

Lead Data Engineer (DevOps)

Sydney, Australia

$220,000 Including Super + Bonus

Overview:

As a Lead Data Engineer reporting to our Engineering Manager, you will be responsible for building, implementing, and enhancing enterprise-scale data platforms. Working in a DevOps environment, you will have end-to-end accountability for designing, developing, deploying, and supporting your data assets. Additionally, you will create templates, implementation methods, standards, and frameworks.

You will engage with various stakeholders and become an expert in your domain while supporting the broader data engineering and Site Reliability Engineering (SRE) community. Your role will include continuously evaluating and recommending improvements to infrastructure and processes, leading incident response efforts, and participating in post-mortem analysis.

Key Requirements

  • Mentor and guide a team on best practices for data engineering and site reliability (SRE).
  • Utilize Big Data and Hadoop technologies with a focus on Spark, Hive, Presto, and big data storage formats such as Parquet, ORC, Avro.
  • Manage CI/CD tools like Bamboo/Ansible.
  • Operate within a cloud-based environment, preferably AWS or GCP.
  • Demonstrate strong skills in Python, SQL, and shell scripting.
  • Work with ETL tools such as Talend, Informatica, Datastage, and understand programming languages like Scala.
  • Apply broad knowledge of technical solutions, design patterns, and coding for complex applications in a distributed computing environment.



If you feel you possess the relevant skills to the above, please respond with most updated CV

Report this job advert

Don’t provide your bank or credit card details when applying for jobs.

#J-18808-Ljbffr
  • Lead Data Engineer

    vor 1 Woche


    Sydney, Österreich BaptistCare NSW & ACT Vollzeit

    Fulltime permanent position in Norwest | Hybrid office environment Newly created role supporting the Data & Analytics Strategy Design, implement & optimise data platform with an industry leading NFP About the role: Reporting to the Data and Analytics (D&A) Manager, the Lead Data Engineer will design and implement data warehouse and foundational data assets...

  • Lead Data Engineer

    Vor 7 Tagen


    Sydney, Österreich ING Bank N.V. Vollzeit

    Great new opportunity for a “Lead Data Engineer – Cloud” to join our Data tribe at ING. Reporting to the Program Director, this role will be working with large volumes of data, both structured and unstructured, and implementing data solutions on the Azure platform. You’ll be responsible for design and build of complex data pipelines and extending...

  • Lead Data Engineer

    vor 3 Wochen


    Sydney, Österreich Macquarie Bank Limited Vollzeit

    Join our Banking and Financial Services data engineering team to help build the best technology company in the financial services industry. Help us build the next evolution of our Data and Analytics Platforms. Joining our team means you’ll get to use innovative approaches to data analytics and play a lead role in shaping and driving this forward. At...

  • Senior Data Engineer

    vor 2 Monaten


    Sydney, Österreich TMGM Vollzeit

    Join our team of talented data engineers and take on the exciting challenge of designing, constructing, installing, testing, and maintaining highly scalable data management systems. Not only will you ensure our systems meet business requirements and adhere to industry practices, but you will also have the opportunity to integrate cutting-edge data management...

  • Senior Data Engineer

    vor 4 Wochen


    Sydney, Österreich TMGM Vollzeit

    Join our team of talented data engineers and take on the exciting challenge of designing, constructing, installing, testing, and maintaining highly scalable data management systems. Not only will you ensure our systems meet business requirements and adhere to industry practices, but you will also have the opportunity to integrate cutting-edge data management...

  • Lead Data Engineer

    Vor 7 Tagen


    Sydney, Österreich Macquarie Bank Limited Vollzeit

    Join our Commodities and Global Markets, Central Technology team as we delivery an exciting strategic program of work that aligns our strategy, risk obligations and platforms to will help Macquarie businesses meet customer needs. At Macquarie, our advantage is bringing together diverse people and empowering them to shape all kinds of possibilities. We are...


  • Sydney, Österreich Towards AI, Inc. Vollzeit

    Mechanical Commissioning Engineer Lead - Data CenterCAI Published 28 Apr 2024 Share this job Sydney, Australia Full Time Role Highlights S3 Project Management Startup Data Testing QA TroubleShooting Operations Google Analytics Tools, Libraries and FrameworksDescription The Mechanical Commissioning Engineer Lead at CAI is tasked with...

  • Senior Data Engineer

    vor 3 Wochen


    Sydney, Österreich ClearCompany Vollzeit

    Business facing role for an accomplished Senior Data Engineer within the Financial Services industry About our client: Our client is an instantly recognisable household name within the Financial Services industry. In recent years they have led the way in the creation of wealth management products and services for their customers. Due to this expansion in...

  • Python Data Engineer

    Vor 4 Tagen


    Sydney, Österreich eFinancialCareers Ltd. Vollzeit

    We are currently partnering with a forward-thinking tech business seeking a highly skilled Data Engineer to join their data engineering team and play a pivotal role in shaping their data-driven future. This is an incredible opportunity to work alongside experienced data professionals and a highly-regarded senior data leader with a wealth of industry...

  • Data Engineer

    vor 4 Wochen


    Sydney, Österreich Experis Vollzeit

    Database Development & Administration (Information & Communication Technology) Our government client are looking for several data engineers to join their team on a contract basis.Please note only Australian Citizens will be considered as security clearance is required.Role Responsibilities and DutiesDesign, construct, and optimize data lakes, CI/CD, and...

  • Data Engineer

    vor 4 Wochen


    Sydney, Österreich Experis ManpowerGroup Sp. z o.o. Vollzeit

    Our government client are looking for several data engineers to join their team on a contract basis.Please note only Australian Citizens will be considered as security clearance is required.Role Responsibilities and DutiesDesign, construct, and optimize data lakes, CI/CD, and Synapse pipelines within the Azure stack.Develop scalable architecture practices...


  • Sydney, Österreich Culture Amp Vollzeit

    How you can help make a better world of workCulture Amp is on a mission to embed analytics and insights across our product suite. We have recently acquired the people analytics platform Orgnostic, and are looking to build out our data platform to further unlock the value of our data. To help us deliver on this mission, we are seeking a Principal Data...

  • Lead Data Engineer

    vor 1 Monat


    Sydney, Österreich Thoughtworks Inc. Vollzeit

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • Lead Data Engineer

    vor 4 Wochen


    Sydney, Österreich Thoughtworks Inc. Vollzeit

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • Data Engineer

    vor 2 Wochen


    Sydney, Österreich Reo Group Vollzeit

    Developers/Programmers (Information & Communication Technology) Location: Sydney CBDSalary Package: $120,000 to $150,000 + Super + BonusRole Highlights: Become a pivotal member of a top-tier energy firm, driving critical data solutions that shape the future of energy trading and analytics. This role offers a unique blend of technical challenge and...

  • Data Engineer

    vor 3 Wochen


    Sydney, Österreich ClearCompany Vollzeit

    Data Engineer Sydney $175,000 Inc Super + Bonus As a Data Engineer within our dynamic Engineering team, you will play a crucial role in developing cutting-edge applications to support our Data Transformation initiatives. This is a permanent position offering the opportunity to work on innovative projects within a supportive team environment. Key...

  • Lead Data Engineer

    vor 1 Monat


    Sydney, Österreich Referrals Only Vollzeit

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • Lead Data Engineer

    vor 4 Wochen


    Sydney, Österreich Referrals Only Vollzeit

    Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems....

  • Senior Data Engineer

    vor 1 Monat


    Sydney, Österreich ClearCompany Vollzeit

    Senior Data Engineer Sydney, Australia 6 Months (Intially) $970pd Inc Super We are seeking a highly skilled and experienced Azure Senior Data Engineer to join a leading insurance business based in Sydney. As an Azure Senior Data Engineer, you will play a critical role in designing and building ETL processes for their data warehouse, utilizing your...

  • Senior Data Engineer

    vor 4 Wochen


    Sydney, Österreich ClearCompany Vollzeit

    Senior Data Engineer Sydney, Australia 6 Months (Intially) $970pd Inc Super We are seeking a highly skilled and experienced Azure Senior Data Engineer to join a leading insurance business based in Sydney. As an Azure Senior Data Engineer, you will play a critical role in designing and building ETL processes for their data warehouse, utilizing your...