Lead Data Engineer
Vor 3 Tagen
Time Zone: Australia based (Sydney preferred, alternatively Melbourne)Role Description & Requirements:Lead technical architecture and solution design for large scale data platform solutionsInteract with customer's data engineering team to provide opinionated guidance on data and analytics platform architecture, best practices, solution design and developmentIndependently develop solutions, i.e., perform hands-on coding for the designed solution, handover developed solution to customer's technical teamPerform pair programming with customer's data engineersProject manage small to mid-sized projects, managing scope, priorities, deliverables, risks/issues, and timelines for successful outcomesCollaborate with cross-disciplinary project team, play the role of workstream or overall project tech lead depending on size and scale of project/programMinimum QualificationsIn-depth knowledge and experience of GCP data and analytics technologiesDesigning data platform architecture and design for:(a) migrating open source or other public cloud based data platforms to GCP cloud native technologies (not just lift and shift migration but re-engineering to native services)(b) Designing green field data platforms on cloudDefine solution architecture and detailed design, and perform hands-on implementation for:(a) Data pipelines for batch and event driven ingestion and processing of data from a variety of sources such as on-prem files, on-prem databases, APIs, etc.
(b) Data pipelines for real time data ingestion and processing.Automatic transpilation of legacy code (HIVE, Teradata, python logic etc.)
to BQ SQLCI/CD pipelines for data workloads using Cloud Build, Artifact Registry, TerraformData governance solutioning using GCP governance tooling (Dataplex, Data Catalog)Tools and languages experience requiredMust have:Programming knowledge and willingness to be hands-on - Python, JavaGood to Have:Experience with dbt or Dataform, Terraform in context of data pipelinesExperience with open source ecosystem and distributions such as Hadoop, Spark, Cloudera/Hortonworks and frameworks and tech such as SPARK, Oozie, Kafka, HBASEUnderstanding and experience with NoSQL databases such as HBASE, MongoDBKnowledge of cloud databases such as Spanner, BigTable, Cloud SQL, DB migrationsCertification: Good to have GCP Data engineeringSeniority levelMid-Senior levelEmployment typeContractIndustriesBusiness Consulting and Services
#J-18808-Ljbffr
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Pyramid Global Technologies VollzeitJob Description for Lead Data Engineer in Sydney.Experience: 10+ years in Software Engineering, Cloud Computing, Databricks, and Big Data.Key Responsibilities:Data EngineeringData WarehousingSolution DesignData AnalysisVisualizationData ModelingProgramming Languages: Python, Scala, SQLCloud Technologies: Azure and DatabricksBig Data Technologies: Cloudera...
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Clearcompany VollzeitSydney CBD with some WFH12 + 12 + 12 month contractNV1 Security Clearance preferredOur client, a large and well-funded Federal Government entity focused on agriculture, is seeking a Lead Data Engineer to join their team in March.Responsibilities: Design and develop scalable data pipelines for ingesting, transforming, and storing structured and unstructured...
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Clearcompany VollzeitHybrid work - Any state in Australia - Sydney, CBD, NSW12 + 12 + 12 months - Baseline clearance requiredData Engineering, ELT, Coding, Data Modelling, DevOpsIntegrated cargo clearance system for biosecurity risk managementOur client, a biosecurity focussed Federal Government department in Melbourne CBD, requires an experienced Lead Data Engineer for a 12 +...
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Clearcompany VollzeitHybrid work - Any state in Australia - Sydney, CBD, NSW12 + 12 + 12 months - Baseline clearance requiredData Engineering, ELT, Coding, Data Modelling, DevOpsIntegrated cargo clearance system for biosecurity risk managementOur client, a biosecurity focussed Federal Government department in Melbourne CBD, requires an experienced Lead Data Engineer for a 12 +...
-
Lead Data Engineer
vor 3 Wochen
Sydney, Österreich Clearcompany VollzeitLead Data Engineer Sydney, Australia $220,000 (including Super) + Bonus About the Role: Join a high-performing Data Team at a leading financial services institution to lead regulatory initiatives and drive innovation in data engineering, utilizing AI and automation to enhance platform efficiency.Key Responsibilities: Lead the development and transformation...
-
Lead Data Engineer
vor 2 Monaten
Sydney, Österreich West Recruitment Pty Ltd VollzeitCall: West Sector: Technology Job Summary Join this certified 'Great Place to Work' business.Spearhead a Data Transformation initiative.Western Sydney Location - Defined WFH Policy.Job Description About Your New Employer:West Technology is proud to be partnering with an Australian owned group that has grown to become a genuine market leader across multiple...
-
Lead Data Engineer
vor 3 Monaten
Sydney, Österreich Baptistcare Nsw & Act VollzeitFulltime permanent position in Norwest | Hybrid office environment Newly created role supporting the Data & Analytics Strategy Design, implement & optimise data platform with an industry leading NFP About the role: Reporting to the Data and Analytics (D&A) Manager, the Lead Data Engineer will design and implement data warehouse and foundational data assets...
-
Clearcompany | Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Tideri Jobbörse VollzeitHybrid work - Any state in Australia - Sydney, CBD, NSW 12 + 12 + 12 months - Baseline clearance required Data Engineering, ELT, Coding, Data Modelling, DevOps Integrated cargo clearance system for biosecurity risk management Our client, a biosecurity focussed Federal Government department in Melbourne CBD, requires an experienced Lead Data Engineer for a 12...
-
Lead Data Engineer
vor 3 Monaten
Sydney, Österreich Baptistcare Nsw & Act VollzeitFulltime permanent position in Norwest | Hybrid office environmentNewly created role supporting the Data & Analytics StrategyDesign, implement & optimise data platform with an industry leading NFPAbout the role:Reporting to the Data and Analytics (D&A) Manager, the Lead Data Engineer will design and implement data warehouse and foundational data assets in...
-
Lead Engineer
vor 3 Monaten
Sydney, Österreich Experis Manpowergroup Sp. Z O.O. VollzeitExperis are proud to be representing a prestigious financial services organisation who are recruiting for an experienced Lead Engineer - Big Data to join their team on a contract basis and lead the design, development, and delivery of future-ready data solutions.As a technical lead, you will work closely with cross-functional teams to provide cutting-edge...
-
Pyramid Global Technologies | Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Tideri Jobbörse VollzeitJob Description for Lead Data Engineer in Sydney .Experience: 10+ years in Software Engineering, Cloud Computing, Databricks, and Big Data.Key Responsibilities: Data Engineering Data Warehousing Solution Design Data Analysis Visualization Data Modeling Programming Languages: Python, Scala, SQL Cloud Technologies: Azure and Databricks Big Data Technologies:...
-
Data Insights Lead
vor 3 Monaten
Sydney, Österreich Data Privacy VollzeitThe role focuses on leading data and analytics requirements within the People & Culture function at NRMA.It involves managing monthly data reporting, seeking efficiency improvements, and providing data insights that drive operational excellence.The position requires collaboration with various business partners to understand data requirements and deliver...
-
Lead Engineer
vor 1 Woche
Sydney, Österreich Resmed Inc VollzeitGlobal Technology Solutions (GTS) at ResMed is a division dedicated to creating innovative, scalable, and secure platforms and services for patients, providers, and people across ResMed. The primary goal of GTS is to accelerate well-being and growth by transforming the core, enabling patient, people, and partner outcomes, and building future-ready...
-
Lead Engineer
vor 1 Woche
Sydney, Österreich Resmed Inc VollzeitGlobal Technology Solutions (GTS) at ResMed is a division dedicated to creating innovative, scalable, and secure platforms and services for patients, providers, and people across ResMed. The primary goal of GTS is to accelerate well-being and growth by transforming the core, enabling patient, people, and partner outcomes, and building future-ready...
-
Lead Engineer
vor 1 Woche
Sydney, Österreich Resmed Inc VollzeitGlobal Technology Solutions (GTS) at ResMed is a division dedicated to creating innovative, scalable, and secure platforms and services for patients, providers, and people across ResMed.The primary goal of GTS is to accelerate well-being and growth by transforming the core, enabling patient, people, and partner outcomes, and building future-ready...
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Macquarie Bank Limited VollzeitJoin our high performing Data Team focused on collaborating across a wide range of business and technical stakeholders to deliver a transformation project within Macquarie's Banking and Financial Services division.This is an excellent opportunity for an ambitious, problem-solving and technical data professional to contribute and grow rapidly within the...
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Macquarie Bank Limited VollzeitJoin our high performing Data Team focused on collaborating across a wide range of business and technical stakeholders to deliver a transformation project within Macquarie's Banking and Financial Services division. This is an excellent opportunity for an ambitious, problem-solving and technical data professional to contribute and grow rapidly within the...
-
Lead Data Engineer
vor 2 Monaten
Sydney, Österreich Lendi Group VollzeitDepartment: Data Design & Emerging ProductCompany DescriptionIn May 2021, Lendi and Aussie Home Loans merged to create Lendi Group. Today, Lendi Group is Australia's largest retail mortgage broker, with a loan book in excess of $70 billion, over 1300 brokers and 225 retail stores across the country.We're passionate about what we do and like to do things...
-
Lead Data Engineer
vor 2 Wochen
Sydney, Österreich Macquarie Bank Limited VollzeitJoin our high performing Data Team focused on collaborating across a wide range of business and technical stakeholders to deliver a transformation project within Macquarie's Banking and Financial Services division. This is an excellent opportunity for an ambitious, problem solving and technical data professional to contribute and grow rapidly within the...
-
Data Engineer
vor 1 Monat
Sydney, Österreich Rich Data Co. VollzeitHybrid and flexible working arrangementsAbout RDCRich Data Co (RDC) is delivering the future of credit today! We believe credit should be accessible, fair, inclusive, and sustainable. We are passionate about AI and developing new techniques that leverage traditional and non-traditional data to get the right decision in a clear and explainable way. Global...
