Staff Data Engineer
vor 1 Woche
This is a Staff Data Engineer - Flink role with one of the leading companies in AU right now 🔥 -- Commonwealth Bank -- with an amazing team. They are continuing to grow rapidly. This is the chance to join right as the 🚀 takes off.
More About the Role at Commonwealth Bank 🧐🧐
Do work that matters: We're building tomorrow’s bank today. This means we need creative and diverse engineers to help us redefine what customers expect from a bank. Envisioning new technologies that are still waiting to be invented and reimagining products that support our customers and help build Australia’s future economy. We are a multi-disciplinary team of passionate end-to-end data & analytics delivery professionals that enables data delivery democratisation/ autonomy, uplift data maturity and literacy across the Group. We provide data delivery thought leadership/ guidance and create leading examples of delivering data safely and swiftly for the Group, using latest data & AI strategies, cutting edge solutions, latest technology tooling, simplified delivery patterns & new ways of working. This role will focus on our Groups real-time stream processing services with the key focus on Apache Flink. See yourself in our team. CommBank Data is the emerging Cloud data platform solution consisting of AWS native services and capabilities that ingests data from multiple source systems and executes business use cases for AI, advanced analytics (machine learning) at scale, across the comprehensive dataset and facilitates data discovery by advanced analytics users. This role will see you become part of the Capability Streaming Ingestion chapter, a part of the wider Data Engineeering and Data Platform crew. The chapter is accountable and responsible for the Groups real time data streaming platform services and or solutions. Key Responsibilities: - Design, develop, and optimize Flink data streams to implement event buffering strategies based on fixed time intervals, sliding windows, session windows, and custom triggers. - Develop and maintain robust Flink pipelines for processing and aggregating data from various real-time data sources, including Kafka, Amazon MSK, etc. - Implement and manage stateful stream processing with checkpointing, fault tolerance, and exactly-once semantics. - Create custom Flink Process Functions with precise time-based buffering logic and timers for controlling event triggers and processing. - Collaborate with data engineers and devops to ensure seamless integration of Flink applications with downstream data storage and processing systems (e.g., S3, databases, data lakes). - Monitor and fine-tune Flink job performance using built-in metrics, profiling tools, and customized metrics and alerts to ensure minimal event lag, low latency, and high throughput. - Identify and resolve bottlenecks, data loss issues, and challenges related to buffering, windowing, and high-volume stream processing. - Implement and configure event retention policies on streaming sources such as Apache Kafka and ensure end-to-end data consistency and accuracy. - Work with cross-functional teams to architect scalable solutions for streaming data processing across multiple business units. - Stay up-to-date with the latest trends in stream processing, distributed data processing, and Apache Flink technologies to bring innovative solutions to the organization. We’re interested in hearing from people who: - Have experience building and managing stateful stream processing pipelines, including checkpointing and state backends. - Bring a strong understanding of distributed systems, data buffering, and data pipeline architectures. - Provide excellent problem-solving skills with a focus on performance optimization and reliability. - Have strong collaboration and communication skills for working in cross-functional teams. - Have the Ability to document technical designs, processes, and troubleshooting guides. Tech Skills: - Expert-level knowledge of Apache Flink and its windowing mechanisms (Tumbling, Sliding, Session windows, Global windows, etc.). - Hands-on experience with data streaming platforms like Apache Kafka, Amazon MSK, and related technologies. - Strong proficiency in Python for Flink application development. - Experience with monitoring, logging, and alerting in distributed data processing environments (e.g., Grafana, Prometheus, CloudWatch). Preferred Qualifications: - Certification with AWS Data Engineering and AWS Solution Architect Professional (e.g., AWS Kinesis Data Analytics, AWS Lambda). - Experience with NoSQL or SQL databases used in conjunction with Flink. - Familiarity with Infrastructure-as-Code (IaC) tools like Terraform, CloudFormation, or CDK.
🚨🚨 If you don’t think you're a perfect fit, you should still sign up to Hatch and create a profile, we'll match you to other roles that suit your profile.
Hatch exists to level the playing field for people as they discover a career that’s right for them. We model this in our hiring process for our partners like Commonwealth Bank.
✅ Applying here is the first step in the hiring process for this role at Commonwealth Bank.
We do not discriminate on the basis of gender identity, sexual orientation, cultural identity, disability, age, or any other non-merit factors. To put it simply, Hatch is for everyone.
-
Data Engineer
vor 1 Monat
Melbourne, Österreich Data Privacy VollzeitContract role initially until 30th June 2022Geelong based organisation - hybrid work modelGenerous rateThis is an opportunity for a Senior Data Engineer to join an organisation undertaking a business transformation, that is not only improving the way they operate, improving customer experience, but contains data and analytics as one of its pillars.The role...
-
Staff Data Engineer
vor 1 Woche
Melbourne, Österreich Blinq Technologies VollzeitWHAT IS BLINQ?The first interaction two people have is the bedrock of all strong business relationships.If you can make that experience special, you can start to build a great second interaction, and so on.Blinq is the tool to help people do that.We're building a platform that allows you to share a snapshot of who you are with anyone, anywhere via digital...
-
Staff Data Engineer Design
vor 2 Wochen
Melbourne, Österreich Commonwealth Bank VollzeitStaff Engineer - Data Solution Design You are an experienced solution designer with strong experience in Data modelling and Big Data platforms.We are one of the critically performing Data Engineering teams in CBA.Together we can lead strategic data warehouse applications and largely complex operational data store!Do work that matters: As an experienced...
-
Gis Specialist Data Engineer
vor 2 Monaten
Melbourne, Österreich Ecf Data, Llc VollzeitCapgemini is a global strategic and technological expert collective, operating in over 50 countries with more than 350,000 professionals.They partner with renowned clients to transform businesses using cloud, data, AI, and digital engineering.The company has significantly expanded in Australia and New Zealand, with over 3,500 team members.Join their rapidly...
-
Senior Azure Data Engineer
vor 2 Monaten
Melbourne, Österreich Ntt Data Business Solutions VollzeitNTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...
-
Data & Analytics Consultant
vor 3 Wochen
Melbourne, Österreich Ntt Data VollzeitConsultants (Information & Communication Technology) Full time Data & Analytics Consultant is to transform data into critical information and knowledge that can be used to make sound business decisions and is also responsible for the establishment of system applications and/or tools that the company uses to record, organize, store, secure, and access...
-
Data & Analytics Consultant
vor 3 Wochen
Melbourne, Österreich Ntt Data VollzeitConsultants (Information & Communication Technology)Full timeData & Analytics Consultant is to transform data into critical information and knowledge that can be used to make sound business decisions and is also responsible for the establishment of system applications and/or tools that the company uses to record, organize, store, secure, and access data....
-
Staff Software Engineer, Data Platform
vor 1 Monat
Melbourne, Österreich Airtree Ventures Pty VollzeitThe RoleAs Staff Software Engineer for Linktree's Data Platform team, you'll play a crucial role in building a robust, scalable data platform that drives innovative experiences in our core product.Your contributions will unlock new opportunities for our tens of millions of users and our billions of visitors around the world, helping us achieve Linktree's...
-
Senior Data Engineer
vor 4 Wochen
Melbourne, Österreich Opus Recruitment Solutions VollzeitEngineering - Software (Information & Communication Technology) Job Title: Senior Data Engineer - Remote (Australia)Company Overview: Our client is a leading retail media technology provider experiencing rapid growth. Their platform helps retailers monetize digital media effectively.Role Summary: Join their dynamic team as a Staff Data Engineer. You'll be...
-
Data Engineer
vor 1 Monat
Melbourne, Österreich Sitepoint Pty VollzeitWorking on behalf of a Local Authority, Aatom Recruitment has a new opportunity for a Data Engineer. This is a permanent vacancy - a cover letter must be submitted along with candidate CV. The Council is on a journey to implement a modern enterprise data and analytics cloud platform and are seeking data engineers to contribute to this project.Essential...
-
Senior Data Engineer
vor 2 Wochen
Melbourne, Österreich At VollzeitEngineering - Software (Information & Communication Technology)Job Title: Senior Data Engineer - Remote (Australia)Company Overview: Our client is a leading retail media technology provider experiencing rapid growth.Their platform helps retailers monetize digital media effectively.Role Summary: Join their dynamic team as a Staff Data Engineer.You'll be a key...
-
Data Engineer
vor 1 Monat
Melbourne, Österreich Aatom Recruitment VollzeitWorking on behalf of a Local Authority, Aatom Recruitment has a new opportunity for a Data Engineer.This is a permanent vacancy – a cover letter must be submitted along with candidate CV.The Council is on a journey to implement a modern enterprise data and analytics cloud platform and are seeking data engineers to contribute to this project.Essential...
-
Staff Backend Engineer, Data Insights Platform
vor 2 Monaten
Melbourne, Österreich Airwallex VollzeitAbout AirwallexAirwallex is the only unified payments and financial platform for global businesses.Powered by our unique combination of proprietary infrastructure and software, we empower over 100,000 businesses worldwide – including Brex, Rippling, Navan, Qantas, SHEIN and many more – with fully integrated solutions to manage everything from business...
-
Staff Backend Engineer, Data Insights Platform
vor 2 Monaten
Melbourne, Österreich Tideri Jobbörse VollzeitAbout Airwallex Airwallex is the only unified payments and financial platform for global businesses.Powered by our unique combination of proprietary infrastructure and software, we empower over 100,000 businesses worldwide – including Brex, Rippling, Navan, Qantas, SHEIN and many more – with fully integrated solutions to manage everything from business...
-
Senior Data Engineer
vor 4 Wochen
Melbourne, Österreich Peoplebank VollzeitArchitects (Information & Communication Technology) LeadersIT is proud to partner with a trusted Australian organisation in their recruitment of a Senior Data Engineer to join their dynamic Technology team. This role will be instrumental in shaping the future of their data and analytics environment, ensuring that their systems and processes scale to meet the...
-
Data Engineer
vor 4 Wochen
Melbourne, Österreich Experis Manpowergroup Sp. Z O.O. VollzeitAre you a skilled Data Engineer with expertise in data modelling and a strong background in Microsoft Azure?Do you have a passion for working with large datasets and geospatial data?If so, we want to hear from you!A dynamic organisation is seeking a Data Engineer to join their team and help improve the performance of a critical database used for airspace...
-
Senior Data Engineer
Vor 6 Tagen
Melbourne, Österreich Tridant Pty Ltd VollzeitEngineering - Software (Information & Communication Technology)Full timeWe are seeking an experienced Data Engineer or Senior Data Engineer to design, implement, and maintain data solutions using Microsoft Fabric and related technologies. The ideal candidate will bridge the gap between business needs and technical solutions, working within an Agile framework...
-
Senior Data Engineer
Vor 5 Tagen
Melbourne, Österreich Tridant Pty Ltd VollzeitEngineering - Software (Information & Communication Technology) Full time We are seeking an experienced Data Engineer or Senior Data Engineer to design, implement, and maintain data solutions using Microsoft Fabric and related technologies.The ideal candidate will bridge the gap between business needs and technical solutions, working within an Agile...
-
Data Engineer
vor 2 Monaten
Melbourne, Österreich Ultradynamix VollzeitRole: Data Engineer Client: Federal Government Contract: 12 Months Extension: 12 Months(2) Clearance: baseline Location: ACT or Offsite Requirements The CIO Division provides a range of enabling services and operations delivery to the department and to Australian businesses and is seeking an experienced and motivated data engineer to assist with the delivery...
-
Data Engineer
vor 3 Wochen
Melbourne, Österreich Onyx Gaming Pty Ltd VollzeitEngineering - Software (Information & Communication Technology) Full time Onyx Gaming are a trusted operational advisor and a leading Electronic Gaming Machine (EGM) systems company.Our commitment to excellence has made us a force in the gaming industry, supporting hospitality venues across Australia.Due to continued growth we are seeking a highly skilled...
