Sr. Data Engineer (Databricks)

Company Overview and Culture
EXL (NASDAQ: EXLS) is a global analytics and digital solutions company that partners with clients to improve business outcomes and unlock growth. Bringing together deep domain expertise with robust data, powerful analytics, cloud, and AI, we create agile, scalable solutions and execute complex operations for the world's leading corporations in industries including insurance, healthcare, banking and financial services, media, and retail, among others. Focused on creating value from data for driving faster decision-making and transforming operating models, EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect. Headquartered in New York, our team is over 40,000 strong, with more than 50 offices spanning six continents. For information, visit www.exlservice.com.
For the past 20 years, EXL has worked as a strategic partner and won awards in its approach to helping its clients solve business challenges such as digital transformation, improving customer experience, streamlining business operations, taking products to market faster, improving corporate finance, building models to become compliant more quickly with new regulations, turning volumes of data into business opportunities, creating new channels for growth and better adapting to change. The business operates within four business units: Insurance, Health, Analytics, and Emerging businesses.
Role & Responsibilities Overview:

• Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)
• Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoring
• Continuously improve systems through performance enhancements and cost reductions in compute and storage
• Implement end to end Machine Learning workflows using MLFlow, including model training tracking and deployment ensuring scalability and performance optimization
• Understand existing Pybricks based solutions, leveraging Python programming and data manipulation libraries to automate data wrangling, pre processing and analysis tasks
• Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs
• Lead Data Engineering Projects to manage and implement data-driven communication systems
• Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations
• Integrate data across different systems and platforms
• Strong verbal and written communication skills to manage client discussions

Candidate Profile:

• 8-12 years experience in developing and implementing ETL pipelines from various data sources using Databricks on cloud
• Based out of US
• Some experience in insurance domain/ data is must
• Programming Languages - SQL, Python
• Technologies - IaaS (AWS or Azure or GCP), Databricks platform, Delta Lake storage, Spark (PySpark, Spark SQL).
• Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure Devops
• Project Management using Agile, Scrum
• B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering or other science field), Information Systems, Information Processing or engineering.
• Excellent communication & leadership skills, with the ability to lead and motivate team members
• Ability to work independently with some level of ambiguity and juggle multiple demands

EEO/Minorities/Females/Vets/Disabilities
To view our total rewards offered click here ->https://www.exlservice.com/us-careers-and-benefits
Base Salary Range Disclaimer: The base salary range represents the low and high end of the EXL base salary range for this position. Actual salaries will vary depending on factors including but not limited to: location and experience. The base salary range listed is just one component of EXL's total compensation package for employees. Other rewards may include bonuses, as well as a Paid Time Off policy, and many region specific benefits.
Please also note that the data shared through the job application will be stored and processed by EXL in accordance with the EXL Privacy Policy.
Application & Interview Impersonation Warning - Purposely impersonating another individual when applying and / or participating in an interview in order to obtain employment with EXL Service Holdings, Inc. (the "Company") for yourself or for the other individual is a crime. We have implemented measures to deter and to uncover such unlawful conduct. If the Company identifies such fraudulent conduct, it will result in, as applicable, the application being rejected, an offer (if made) being rescinded, or termination of employment as well as possible legal action against the impersonator(s).
EXL may use artificial intelligence to create insights on how your candidate information matches the requirements of the job for which you applied. While AI may be used in the recruiting process, all final decisions in the recruiting and hiring process will be taken by the recruiting and hiring teams after considering a candidate's full profile. As a candidate, you can choose to opt out of this artificial intelligence screening process. Your decision to opt out will not negatively impact your opportunity for employment with EXL.
Other details

• Pay Type Salary
• Min Hiring Rate $125,000.00
• Max Hiring Rate $140,000.00

Apply Now