Efinancialcareers Ltd. | Staff Data Engineer

Vor 3 Tagen

Sydney, Österreich Tideri Jobbörse Vollzeit

Staff Data Engineer - Flink Commonwealth Bank of Australia Sydney, Australia
Do work that matters:
We're building tomorrow's bank today.
This means we need creative and diverse engineers to help us redefine what customers expect from a bank.
Envisioning new technologies that are still waiting to be invented and reimagining products that support our customers and help build Australia's future economy.
We are a multi-disciplinary team of passionate end-to-end data & analytics delivery professionals that enables data delivery democratisation, uplift data maturity and literacy across the Group.
We provide data delivery thought leadership and create leading examples of delivering data safely and swiftly for the Group, using latest data & AI strategies, cutting edge solutions, latest technology tooling, simplified delivery patterns & new ways of working.
This role will focus on our Groups real-time stream processing services with the key focus on Apache Flink.
Key Responsibilities:
Design, develop, and optimize Flink data streams to implement event buffering strategies based on fixed time intervals, sliding windows, session windows, and custom triggers.
Develop and maintain robust Flink pipelines for processing and aggregating data from various real-time data sources, including Kafka, Amazon MSK , etc.
Implement and manage stateful stream processing with checkpointing, fault tolerance, and exactly-once semantics.
Create custom Flink Process Functions with precise time-based buffering logic and timers for controlling event triggers and processing.
Collaborate with data engineers and devops to ensure seamless integration of Flink applications with downstream data storage and processing systems (e.g., S3, databases, data lakes).
Monitor and fine-tune Flink job performance using built-in metrics, profiling tools, and customized metrics and alerts to ensure minimal event lag, low latency, and high throughput.
Identify and resolve bottlenecks, data loss issues, and challenges related to buffering, windowing, and high-volume stream processing.
Implement and configure event retention policies on streaming sources such as Apache Kafka and ensure end-to-end data consistency and accuracy.
Work with cross-functional teams to architect scalable solutions for streaming data processing across multiple business units.
Stay up-to-date with the latest trends in stream processing, distributed data processing, and Apache Flink technologies to bring innovative solutions to the organization.

We're interested in hearing from people who:
Have experience building and managing stateful stream processing pipelines, including checkpointing and state backends.
Bring a strong understanding of distributed systems, data buffering, and data pipeline architectures.
Provide excellent problem-solving skills with a focus on performance optimization and reliability.
Have strong collaboration and communication skills for working in cross-functional teams.
Have the ability to document technical designs, processes, and troubleshooting guides.

Tech Skills:
Expert-level knowledge of Apache Flink and its windowing mechanisms (Tumbling, Sliding, Session windows, Global windows, etc.).
Hands-on experience with data streaming platforms like Apache Kafka, Amazon MSK , and related technologies.
Strong proficiency in Python for Flink application development.
Experience with monitoring, logging, and alerting in distributed data processing environments (e.g., Grafana, Prometheus, CloudWatch).

Preferred Qualifications:
Certification with AWS Data Engineering and AWS Solution Architect Professional (e.g., AWS Kinesis Data Analytics, AWS Lambda).
Experience with NoSQL or SQL databases used in conjunction with Flink.
Familiarity with Infrastructure-as-Code (IaC) tools like Terraform, CloudFormation, or CDK.

Advertising End Date: 01/12/2024#J-18808-Ljbffr

  • Staff Data Engineer Design | Sydney, Au

    Vor 2 Tagen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Data Engineer DesignCommonwealth Bank of Australia Sydney, AustraliaStaff Data Engineer DesignCommonwealth Bank of Australia Sydney, AustraliaYou are an experienced solution designer with strong experience in Data modelling, Big Data platforms.We are one of the critically performing Data Engineering teams in CBATogether we can lead strategic data...

  • Staff Data Engineer

    Vor 3 Tagen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Data Engineer - Accelerated Data DeliveryCommonwealth Bank of Australia Sydney, AustraliaStaff Data Engineer - Accelerated Data DeliveryYou are a passionate data engineer & keen learner with extensive multi-disciplinary background in end-to-end data & analytics delivery value chain activities and modern data engineering...

  • Staff Engineer | Sydney, Au

    vor 4 Wochen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Engineer Commonwealth Bank of Australia Sydney, Australia Do work that mattersTechnology is responsible for the world leading application of technology and operations across every aspect of CommBank, from innovative product platforms for our customers to essential tools within our business.We also use technology to drive efficient and timely...

  • Staff Platform Engineer | Sydney, Au

    vor 3 Wochen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Platform Engineer Commonwealth Bank of Australia Sydney, Australia Do work that matters:As Australia's leading technology bank, we process millions of transactions every day.Around 40% of all payments made by Australians go through our systems.Our scale means unparalleled opportunity to accelerate digital innovation and positively impact...

  • Staff Data Engineer

    Vor 3 Tagen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Data Engineer - FlinkCommonwealth Bank of Australia Sydney, AustraliaDo work that matters:We're building tomorrow's bank today.This means we need creative and diverse engineers to help us redefine what customers expect from a bank.Envisioning new technologies that are still waiting to be invented and reimagining products that support our customers and...

  • Data Engineer | Sydney, Au

    vor 1 Woche

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Data Engineer Westpac Banking Corporation Sydney, Australia Data Engineer Westpac Banking Corporation Sydney, Australia Job DescriptionData Engineer BI Projects and ChangeMax Term until October 2025Barangaroo Head Office LocationHybrid work from office and homeWestpac Group is made up of Australia's most recognised financial services brands, including...

  • Staff Platform Engineer

    vor 4 Wochen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Platform Engineer (MLOps Platforms) Commonwealth Bank of Australia Sydney, Australia You are highly experienced in building customer focussed solutionsWe are a team of big thinkers, who love to push boundaries and create new solutionTogether we will build tomorrow's bank today, using world-leading technology and innovation Staff Platform Engineer...

  • Efinancialcareers Ltd. | Acquisition Senior Data Engineer | Sydney

    Vor 3 Tagen

    Sydney, Österreich Tideri Jobbörse Vollzeit

    Acquisition Senior Data EngineerJob No: 673414Sydney - Inner Suburbs & CBD, Brisbane - Inner Suburbs & CBD, Melbourne - Inner Suburbs & CBD, VariousPermanent Full TimeITPay Band 5Flexible working locations of Melbourne, Brisbane or Sydney Hybrid Working Environment + Flexi Days Drive data innovation and lead engineering initiatives As an Acquisition Senior...

  • Efinancialcareers Ltd. | Data Engineer | Sydney, Au | Sydney

    vor 1 Tag

    Sydney, Österreich Tideri Jobbörse Vollzeit

    Data Engineer Westpac Banking Corporation Sydney, Australia Data Engineer Westpac Banking Corporation Sydney, Australia Job DescriptionData Engineer BI Projects and ChangeMax Term until October 2025 Barangaroo Head Office Location Hybrid work from office and home Westpac Group is made up of Australia's most recognised financial services brands, including...

  • Efinancialcareers Ltd. | Data Engineer | Sydney

    Vor 2 Tagen

    Sydney, Österreich Tideri Jobbörse Vollzeit

    Data Engineer Westpac Banking Corporation Sydney, Australia Job DescriptionData Engineer BI Projects and ChangeMax Term until October 2025 Barangaroo Head Office Location Hybrid work from office and home Westpac Group is made up of Australia's most recognised financial services brands, including Westpac, St George, Bank of Melbourne, BankSA, BT Financial...

  • Staff Network Security Engineer | Sydney, Au

    vor 1 Monat

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Network Security Engineer Commonwealth Bank of Australia Sydney, Australia You are a problem solver with a strong background in Network securityWe're looking to set the standard for a world class, self-service, and secure public cloud platformTogether we can build state-of-the-art solutions to drive seamless experiences for millions of customers Staff...

  • Staff Quality Engineer | Sydney, Au

    vor 1 Woche

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Quality Engineer Commonwealth Bank of Australia Sydney, Australia Staff Quality Engineer Commonwealth Bank of Australia Sydney, Australia You are an expert in Quality Engineering, designing & building test automation strategiesWe are a team of big thinkers who are engineering the future of banking.Together, we will build tomorrow's bank today, using...

  • Staff Engineer Chapter Lead Commsec Software Engineering | Sydney, Au

    vor 4 Wochen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Engineer Chapter Lead CommSec Software Engineering Staff Engineer Chapter Lead CommSec Software Engineering Commonwealth Bank of Australia Sydney, Australia Staff Engineer Chapter Lead CommSec Software Engineering Staff Software Engineer Chapter Lead - CommSec You are an empowering leader, passionate, driven, flexible and collaborativeWe are a bunch...

  • Staff Site Reliability Engineer | Sydney, Au

    vor 1 Monat

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Site Reliability Engineer Commonwealth Bank of Australia Sydney, Australia You are passionate about SRE and systems engineering We are undergoing one of Australia's largest digital transformations Together we can reimagine banking for millions of customers You are passionate about SRE and systems engineering We are undergoing one of Australia's largest...

  • Staff Site Reliability Engineer | Sydney, Au

    vor 1 Monat

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Site Reliability Engineer Commonwealth Bank of Australia Sydney, Australia You are passionate about SRE and systems engineeringWe are undergoing one of Australia's largest digital transformationsTogether we can reimagine banking for millions of customers You are passionate about SRE and systems engineeringWe are undergoing one of Australia's largest...

  • Vice President

    vor 2 Wochen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Vice President - Sydney, Australia - eFinancialCareers Ltd.DescriptionJoin BlackRock's Multi-Asset Strategies & Solutions team as Vice President, StrategistBlackRock's MASS group is seeking a highly skilled and motivated Vice President, Strategist to join our Sydney, Australia team.Our team of over 350 multi-asset investment professionals manage various...

  • Staff Software Engineer- Generative Ai | Sydney, Au

    vor 1 Tag

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Software Engineer- Generative AICommonwealth Bank of Australia Sydney, AustraliaStaff Software Engineer- Generative AICommonwealth Bank of Australia Sydney, AustraliaYou are a technology savvy and passionate about Generative AI, agentic computing and the challenge of building safe and reliable systems on cutting edge technology.We are leading the world...

  • Staff Software Engineer, Unloan,X15Ventures | Sydney, Au

    vor 1 Monat

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Staff Software Engineer, Unloan,x15ventures Staff Software Engineer, Unloan,x15ventures Let's introduce ourselves.Unloan is a new digital lending venture, backed by CommBank and built in x15Ventures.We are a diverse team of product managers, designers, engineers, risk managers and lenders.And we're working together to deliver Australia's most-loved home...

  • Data Engineer

    Vor 3 Tagen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Data EngineerWestpac Banking Corporation Sydney, AustraliaJob DescriptionData Engineer BI Projects and ChangeMax Term until October 2025Barangaroo Head Office LocationHybrid work from office and homeWestpac Group is made up of Australia's most recognised financial services brands, including Westpac, St George, Bank of Melbourne, BankSA, BT Financial Group...

  • Data Engineer

    Vor 2 Tagen

    Sydney, Österreich Efinancialcareers Ltd. Vollzeit

    Data EngineerWestpac Banking Corporation Sydney, AustraliaJob Description Data Engineer BI Projects and Change Max Term until October 2025Barangaroo Head Office LocationHybrid work from office and homeWestpac Group is made up of Australia's most recognised financial services brands, including Westpac, St George, Bank of Melbourne, BankSA, BT Financial Group...