Senior Data Engineer
vor 21 Stunden
Senior Data Engineer: Firewall Automation Program Contract: 6-12 months (Pty Ltd only) $1,150-$1,200/day + GST Sydney based (Hybrid) Urgent start - interviewing immediately About the role Build MS SQL ETL pipelines that consolidate 5 million network flows down to 5,000-10,000 automated firewall rules for a tier‑1 bank's CoreMod migration. You'll enrich NetFlow data with CMDB/DNS/VMware metadata, produce JSON/CSV deployment feeds for APIC/CSM, and optimise performance at scale. This role suits a data engineer who thrives on genuinely interesting technical problems in collaborative mob programming environments. About the engagement Major infrastructure transformation migrating 400 applications across ~3,000 servers. Three-person specialist team (you, Python dev, network engineer) automating firewall implementation across Legacy and CoreMod infrastructure. Fast-tracked delivery with high autonomy and creative problem‑solving. What you'll own Data Pipeline: Consolidate 5M network flows stored in MS SQL into contextualised, validated firewall rules using high-performance T‑SQL ETL jobs with deduplication, enrichment and sliding‑window aggregations. Deployment Feeds Produce controller‑ready JSON/CSV exports conforming to CSM import formats and APIC object model payloads that enable the Python dev to automate rule implementation via Cisco APIs. Orchestration Create Tidal job definitions, retries, alerts and exception queues. Implement lineage tracking from aggregated tuples back to raw flows for audit purposes. This is collaborative data engineering – mob programming setup where all three specialists work in lockstep. You'll optimise partitioning, indexing and query plans for very large NetFlow tables while navigating access problems and data challenges with a proactive mindset. What we need Must have: Expert MS SQL Server (T‑SQL, indexing, partitioning, window functions, query tuning), high-performance ETL pipelines for large‑scale datasets (5M+ rows), controller‑ready export formats (JSON/CSV) for network automation platforms, Tidal or similar orchestration tools, data engineering mindset (NOT DBA), collaborative team player, Pty Ltd contractor. Nice to have Dask/Spark experience, Power BI, understanding of network flows/NetFlow data, migration project experience. The opportunity Real infrastructure impact enabling automated firewall deployment at enterprise scale. Strong likelihood of 12-month engagement. #J-18808-Ljbffr
-
Data Engineer
vor 21 Stunden
Council of the City of Sydney, Österreich NTT DATA, Inc. VollzeitGet AI-powered advice on this job and more exclusive features. This range is provided by NTT DATA, Inc.. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range A$115.00/hr - A$125.00/hr Location Location: Sydney preferred | Hybrid, 3 days onsite Duration Duration: January – April 2026 (with...
-
Technical Lead
Vor 5 Tagen
Council of the City of Sydney, Österreich NTT DATA, Inc. VollzeitTalent Acquisition Partner @ NTT DATA | Your success is my mission, and I’m here to support and champion you every step of the way! Tech Lead – Data Engineer (AI) | Contract | Sydney & Melbourne Flexi Location Exciting contract opportunities for AI-focused Data Engineers and Tech Leads! We’re seeking experienced engineers and hands‑on technical...
-
Senior Data Engineer
vor 3 Wochen
Council of the City of Sydney, Österreich Talent Insights Group VollzeitTalent Insights Group provided pay range This range is provided by Talent Insights Group. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range A$155,000.00/yr - A$165,000.00/yr Principal Data Consultant at Talent Insights - dbt | Databricks | Looker | Snowflake Senior Data Engineer ( SQL /...
-
Data Engineer
vor 2 Wochen
Sydney, Österreich Data#3 VollzeitOverview: We are seeking an experienced Data Engineer to join a large-scale program for an initial 3-month contract. The role will focus on building and optimising data pipelines in an Azure environment, working closely with technical teams to deliver efficient, high-quality data solutions. Responsibilities: Design, build, and maintain ETL pipelines using...
-
Hybrid Data Engineer
vor 21 Stunden
Council of the City of Sydney, Österreich NTT Data VollzeitA leading global technology services provider is seeking an experienced Data Engineer (Consultant) to develop structured data sets and enable analytics. The role involves designing ETL packages and collaborating with IT and business teams to define data models. Candidates should have at least 5 years of experience in data engineering, strong skills in ETL...
-
Senior Data Engineer
vor 4 Wochen
Council of the City of Sydney, Österreich Maltem Australia VollzeitEngagement: Permanent, initial 8-week full-time engagement with potential extension Start Date: December About the Role Maltem is a global consulting group specialising in digital transformation and data-driven solutions. We partner with leading organisations to deliver impactful results through expert talent and innovative practices. We are seeking an...
-
Senior Network DevOps Engineer
vor 2 Wochen
Council of the City of Sydney, Österreich NTT DATA VollzeitSenior Network DevOps Engineer – Core Modernisation We’re looking for a Senior Network DevOps Engineer to join our Core Modernisation Infrastructure Automation stream. This is your chance to shape the future of automation and networks in a high‑impact program. What you’ll do Automate software updates for Cisco & F5 products Partner with vendors and...
-
Senior Data Engineer
vor 2 Wochen
Council of the City of Sydney, Österreich Atlassian VollzeitSenior Data Engineer – Atlassian Apply for the Senior Data Engineer role at Atlassian . Overview Our Data Engineering Team builds world‑class data solutions and applications that power crucial business decisions. We manage analytical data models and pipelines across finance, growth, product analysis, customer analysis, sales and marketing. We maintain...
-
Senior Data Engineer
vor 4 Wochen
Council of the City of Sydney, Österreich Talent Insights Group VollzeitTalent Insights Group provided pay range This range is provided by Talent Insights Group. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range A$150,000.00/yr - A$160,000.00/yr Principal Data Consultant at Talent Insights - dbt | Databricks | Looker | Snowflake Senior Data Engineer ( GCP /...
-
Senior Data Engineer
vor 2 Wochen
Council of the City of Sydney, Österreich ClearCompany VollzeitDue a data modernisation program moving to Fabric, we are now looking for a Senior Data Engineer to join the business. The role of the Senior Data Engineer is responsible for designing, building, and maintaining the infrastructure that supports data storage, processing and retrieval. You will require proven experience working with large data sets and the...
