Position Title: Data Platform Engineer
Position Type: Permanent
Our client is looking for an experiences Azure Data Platform Data Engineer to work with the architecture teams in an Agile environment to manage deployment of Azure in the client locations. The Data Platform Engineer will deploy Azure in client locations.
- Development for data pipelines – ingestion, transformation, cleansing.
- Understanding the technical requirements of existing Data Platform and migration to Microsoft Azure cloud platform
- Design and build data pipelines to migrate large-scale data from existing on-premises data platform to Azure data lake using ADF
- Set up data warehouse
- Create data factory and linked services and new datasets in order to do the ETL jobs
- Create new data pipelines, debug existing data pipelines, trigger pipelines manually and trigger the pipeline on the schedule
- Data flow integration with external systems. Integration with data access tools and products..
- Experienced in Azure Cloud Platform (Azure Data Factory, HDI, CLI, Storage: ADLS and Blob, SQL Data Warehouse, Azure Functions, Azure Event Hubs, KAFKA).
- 3 or more years of hands-on experience in Spark/Scala development , Spark Streaming
- Hands-on experience (3+ years) with Azure Data Factory for creating Data Pipelines, debugging existing and incremental load to Azure Blob Storage, ADLS.
- Experienced (3+ years) with Azure Synapse Analysis
- Ability to deal with structured, semi-structured, unstructured and streaming data. 3-5 years of experience in ETL & data migration and processing and including relational databases (e.g., SQL Server, Oracle), Data warehousing, Big Data (Hadoop, Hive)
- Minimum 2 year of designing, building and operationalizing large scale enterprise data solutions and applications using one or more of Azure analytics services in combination with custom solutions - Spark, Azure Event Hubs, Azure Data Factory, ADLS, SQL DW, Azure Functions, etc.
- Candidates should be flexible / willing to work across this delivery landscape which includes and not limited to Agile Applications Development, Support and Deployment.
To apply, please send your resumes to email@example.com with ‘Data Platform Engineer’ in the subject.