Job Description
Technical / Functional Skills:
•AzureSQL Database, Cosmos (No SQL),
•Python,Scala
•StrongHands-on in T-SQL
•Experiencewith Azure, REST API
•Creatingdistributed data pipelines for batch and real-time applications
•ApacheSpark,
•AzureData Factory,
•AzureDatabricks
•JupiterNotebook
•AzureSynapse Analytics
•AzureData Lake/ Analytics
Roles and Responsibilities:
•Design,develop and build real-time data pipelines from a variety of sources(streaming data, APIs, data warehouse, messages etc.)
•Leveragethe understanding of software architecture and software designpatterns to write scalable, maintainable, well-designed and future-proofsoftware
•Manageexisting pipelines and create new pipelines from a variety of sources(relational, XML, etc.)
•Actively applybest practices within CI/CD
•Proposeand implement solutions for data pipeline stabilization and data qualitychecks
•Coordinationwith other teams to design optimal patterns for data ingest and egress, aswell as lead and coordinate data quality initiatives and troubleshooting
•Designand build solutions to track data quality, stabilize data pipeline, etc.to ensure reliable operations
•Ensurebest practices are followed across architecture, codebaseand configuration
•Eliminatewaste
•
Job Tags