Senior Data Engineer-Big Data
vor 1 Woche
What makes Cognizant a unique place to work?
The combination of rapid growth and an international and innovative environment
This is creating many opportunities for people like YOU — people with an entrepreneurial spirit who want to make a difference in this world.
At Cognizant, together with your colleagues from all around the world, you will collaborate on creating solutions for the world's leading companies and help them become more flexible, more innovative, and successful.
Moreover, this is your chance to be part of the success story.
Position Summary:
Design End-to-end architecture and design solutions using one or more platforms for large-scale high-throughput applications.
Design and build data ingestion jobs for ingesting data from multiple heterogeneous sources, APIs and processing using Pyspark, Amazon Document DB and R. Analyze the data patterns, apply data cleansing and modernization using Amazon Bedrock, code whisperer, AWS Lambda, Amazon Athena and build a lake warehouse, cloud storages.
Using Flume agent as target for API and store data to Kafka topic and process using Spark Streaming.
Use Google Api for API management, Build Data pipeline for Data ingestion from API real time and batch sources.
Build enterprise data lake to store raw and processed data for analysis and decision making.
Use Azure data factory, Azure synapse, Azure Databricks for data pipeline and process and store data on Cloud.
Use Kafka Event hub to process real time data.
Use HBase for NoSQL data storage, Use Hive for Meta data management and building data lake in Hadoop.
Use Oozie, Airflow for Orchestration of data pipelines and Spark Shell jobs.
Experience of creating dev-ops CICD pipelines with Bamboo and Jenkins.
End to End project execution, Requirement gathering, transforming legacy design to Bigdata eco system.
Development, Testing, UAT Support and GO Live support.
Understanding and developing PowerBI and Tableau reports would be an additional advantage.
Mandatory Skills:
Must have minimum 6+ years of experience in Big Data/AWS/Azure.
Strong knowledge and expertise in digital transformation implementations, Data warehousing, ETL and Big Data.
Data Lake building, Spark, Amazon Document DB, NoSQL database, Airflow, Oozie, Kafka, Scala, Pyspark, Azure Cloud, Azure EventHub, Azure Data Factory, Azure function, Azure Synapse, Azure functions, Azure HDInsight, Billing & Revenue Management Devices.
AWS Bedrock (image), Amazon Code Whisperer, AWS Open Search, Amazon Comprehend Medical, Amazon Polly, AWS Lambda, Amazon Comprehend, Node js, Amazon Athena, AWS Bedrock (text), Amazon Neptune, Amazon's Alexa and Lex, AWS Cloud Compute, AWS RDS, Amazon Sage maker, AWS Kendra.
Azure DevOps, JIRA, GIT, Bitbucket, Bamboo, Jenkins, Confluence.
Duties and Responsibilities:
Responsible for architecting end to end data processing for the Billing and revenue management platform.
Design and implement technical governance process as appropriate across the application development lifecycle coding guidelines, quality standards, best practices, code acceptance criteria, automation.
Create Implementation strategy documents, which is an IT strategy approach for creation of customized software solutions requiring the application of advanced and complex IT theories and methods.
Build end to end Big Data solutions using Cognizant's Big data accelerators to cover all possible business scenarios.
Research, evaluate and deploy new tools frameworks and patterns to build sustainable Big Data platform.
Identify gaps and opportunities for improvement of existing solutions.
Responsible for migrating and transforming data from on-prem to on-cloud.
Building CICD pipelines for automation.
Participate in business requirements and functional requirements review meetings.
Define and develop APIs for integration of various data sources.
Coordinate business and functional acceptance testing which involves ensuring technical specifications meet business requirements.
Responsible for daily and weekly status reports to update business teams and project managers.
Salary Range: $100,000
Date of posting: 24/10/2024
Next Steps: If you feel this opportunity suits you, or Cognizant is the type of organization you would like to join, we want to have a conversation with you
Please apply directly with us.
For a complete list of open opportunities with Cognizant, visit .
Cognizant is committed to providing Equal Employment Opportunities.
Successful candidates will be required to undergo a background check.
About Cognizant:
Cognizant (Nasdaq-100: CTSH) is one of the world's leading professional services companies, transforming clients' business, operating and technology models for the digital era.
Our unique industry-based, consultative approach helps clients envision, build and run more innovative and efficient businesses.
Headquartered in the U.S.
Learn how Cognizant helps clients lead with digital at or follow us @Cognizant.
#LI-CTSAPAC
#J-18808-Ljbffr
-
Data Engineer
vor 4 Wochen
Melbourne, Österreich Data Privacy VollzeitContract role initially until 30th June 2022Geelong based organisation - hybrid work modelGenerous rateThis is an opportunity for a Senior Data Engineer to join an organisation undertaking a business transformation, that is not only improving the way they operate, improving customer experience, but contains data and analytics as one of its pillars.The role...
-
Senior Azure Data Engineer
vor 1 Monat
Melbourne, Österreich Ntt Data Business Solutions VollzeitNTT DATA is heavily investing globally in providing state of the art analytical solutions to its clients. Our capability has been focused locally, regionally in APAC and globally by forming strategic partnerships including Microsoft Azure, SAP Analytics, Snowflake, Matillion and many more, leveraging the strength of our cloud and network assets via the NTT...
-
Senior Data Engineer
vor 3 Wochen
Melbourne, Österreich Opus Recruitment Solutions VollzeitEngineering - Software (Information & Communication Technology) Job Title: Senior Data Engineer - Remote (Australia)Company Overview: Our client is a leading retail media technology provider experiencing rapid growth. Their platform helps retailers monetize digital media effectively.Role Summary: Join their dynamic team as a Staff Data Engineer. You'll be...
-
Senior Data Engineer
vor 1 Woche
Melbourne, Österreich At VollzeitEngineering - Software (Information & Communication Technology)Job Title: Senior Data Engineer - Remote (Australia)Company Overview: Our client is a leading retail media technology provider experiencing rapid growth.Their platform helps retailers monetize digital media effectively.Role Summary: Join their dynamic team as a Staff Data Engineer.You'll be a key...
-
Data & Analytics Consultant
vor 2 Wochen
Melbourne, Österreich Ntt Data VollzeitConsultants (Information & Communication Technology) Full time Data & Analytics Consultant is to transform data into critical information and knowledge that can be used to make sound business decisions and is also responsible for the establishment of system applications and/or tools that the company uses to record, organize, store, secure, and access...
-
Data & Analytics Consultant
vor 2 Wochen
Melbourne, Österreich Ntt Data VollzeitConsultants (Information & Communication Technology)Full timeData & Analytics Consultant is to transform data into critical information and knowledge that can be used to make sound business decisions and is also responsible for the establishment of system applications and/or tools that the company uses to record, organize, store, secure, and access data....
-
Gis Specialist Data Engineer
vor 2 Monaten
Melbourne, Österreich Ecf Data, Llc VollzeitCapgemini is a global strategic and technological expert collective, operating in over 50 countries with more than 350,000 professionals.They partner with renowned clients to transform businesses using cloud, data, AI, and digital engineering.The company has significantly expanded in Australia and New Zealand, with over 3,500 team members.Join their rapidly...
-
Senior Data Engineer
vor 2 Wochen
Melbourne, Österreich https:www.linkedin.comcompany2203697 VollzeitExperis is working with a well-renowned, market leading organization who is looking for a talented Senior Data Engineer to help architect, build and scale data infrastructure that powers their analytics and machine learning capability. Successful candidates will have strong experience with Azure Stack and Power BI. This is a permanent role with a competitive...
-
Senior Data Engineer
vor 2 Wochen
Melbourne, Österreich https:www.linkedin.comcompany2203697 VollzeitExperis is working with a well-renowned, market leading organization who is looking for a talented Senior Data Engineer to help architect, build and scale data infrastructure that powers their analytics and machine learning capability. Successful candidates will have strong experience with Azure Stack and Power BI. This is a permanent role with a competitive...
-
Senior Data Engineer
vor 2 Wochen
Melbourne, Österreich Experis VollzeitFull timeExperis is working with a well-renowned, market leading organization who is looking for a talented Senior Data Engineer to help architect, build and scale data infrastructure that powers their analytics and machine learning capability. Successful candidates will have strong experience with Azure Stack and Power BI. This is a permanent role with a...
-
At | Senior Data Engineer | Melbourne
Vor 3 Tagen
Melbourne, Österreich Tideri Jobbörse VollzeitEngineering - Software (Information & Communication Technology) Job Title: Senior Data Engineer - Remote (Australia) Company Overview: Our client is a leading retail media technology provider experiencing rapid growth.Their platform helps retailers monetize digital media effectively.Role Summary: Join their dynamic team as a Staff Data Engineer.You'll be a...
-
Senior Data Engineer
vor 2 Monaten
Melbourne, Österreich Payjoy VollzeitAbout PayJoy PayJoy is a mission-first financial service provider dedicated to helping under-served customers in emerging markets to achieve financial stability and success.We lend through our patented technology that turns a smartphone into digital collateral, and our cutting-edge machine learning, data science, and anti-fraud AI allow us to offer the...
-
Data Science Manager
vor 1 Monat
Melbourne, Österreich Tech & Data People VollzeitMathematics, Statistics & Information Sciences (Science & Technology)Full timeA globally renowned company is seeking a passionate Team Leader to lead a talented Data Science team working on cutting edge projects within the Customer and Marketing data areas of the business.In this role you will be encouraged to push the boundaries of what is possible and lead...
-
Senior Data Engineer
vor 2 Wochen
Melbourne, Österreich https:www.linkedin.comcompany2203697 VollzeitExperis is working with a well-renowned, market leading organization who is looking for a talented Senior Data Engineer to help architect, build and scale data infrastructure that powers their analytics and machine learning capability.Successful candidates will have strong experience with Azure Stack and Power BI.This is a permanent role with a competitive...
-
Melbourne, Österreich Data Privacy VollzeitAddress Melbourne, Melbourne, Victoria,, Melbourne, Victoria Description About the Position Country: Australia Cities: Adelaide, Brisbane, Canberra, Melbourne,Area of expertise: Analytics Job Description Digital disruption is opening more opportunities for businesses to connect with their customers and accelerate the realization of business The Worlds...
-
Senior Data Engineer
vor 3 Wochen
Melbourne, Österreich Peoplebank VollzeitArchitects (Information & Communication Technology) LeadersIT is proud to partner with a trusted Australian organisation in their recruitment of a Senior Data Engineer to join their dynamic Technology team. This role will be instrumental in shaping the future of their data and analytics environment, ensuring that their systems and processes scale to meet the...
-
24X7 Data Centre Technician
vor 2 Monaten
Melbourne, Österreich Data Privacy VollzeitNEXTDC is focused on building advanced data centers that employ innovative and eco-friendly infrastructure.The role involves supporting the facility management team by performing technical maintenance and providing excellent customer service within a 24/7 operational environment.The position emphasizes hands-on training to understand the management of...
-
Senior Data Engineer
vor 2 Monaten
Melbourne, Österreich Marmalade VollzeitOver the past three years, Marmalade has been hard at work changing business for the better.We've grown from an idea to processing hundreds of millions of dollars of payments for a wide network of partners.We're not stopping there.We've got big plans to change what invoice payments are and can be.A movement bringing trust back to the financial sector.We're...
-
Senior Software Engineer
vor 2 Monaten
Melbourne, Österreich Demyst Data Limited VollzeitOur SolutionDemyst is a data management company specializing in external data orchestration, helping leading global financial institutions support their business users with data access at scale within their centralized data platforms.In response to growing demand, we're seeking new team members to help us scale.By joining our team, you will play a crucial...
-
Senior Data Engineer
vor 2 Wochen
Melbourne, Österreich Tideri Jobbörse VollzeitOver the past three years, Marmalade has been hard at work changing business for the better.We've grown from an idea to processing hundreds of millions of dollars of payments for a wide network of partners.We're not stopping there.We've got big plans to change what invoice payments are and can be.A movement bringing trust back to the financial sector.We're...
