Senior Software Engineer

Job Description

Location(s):

• Tower -11, (IT/ITES) SEZ of M/s Gurugram Infospace Ltd, Vill. Dundahera, Sector-21, Gurugram, Haryana, Gurugram, Haryana, 122016, IN

Line Of Business: Customer, Operations, & Risk OU(COR OU)

Job Category:

• Engineering & Technology

Experience Level: Experienced Hire

Job Description

The ideal candidate will have extensive experience in managing Databricks on Azure Cloud, Continuous Integration and Continuous Deployment (CI/CD) and Terraform. The Senior Software Engineer will work closely with data architects, data scientists, and data analysts to ensure the effective implementation and management of Databricks solutions.

• Design, develop, and implement Databricks solutions to support data integration, analytics, and business intelligence requirements.
• Collaborate with data architects, data scientists, and data analysts to understand data requirements and to ensure Databricks solutions meet business needs.
• Manage and optimize Databricks environments, including performance tuning, scaling, and cost management.
• Develop and maintain CI/CD pipelines for Databricks projects using tools like Jenkins, GitHub Actions, or Azure DevOps.
• Implement infrastructure as code (IaC) solutions using Terraform to automate the deployment and management of Databricks resources on Azure Cloud.
• Perform data cleansing, transformation, and integration tasks to ensure data quality and integrity within Databricks.
• Integrate third-party application data into Databricks, ensuring seamless data flow and compatibility.
• Monitor and troubleshoot Databricks jobs, clusters, and workflows, and provide technical support and guidance to other team members.
• Evaluate and recommend new tools, technologies, and best practices to enhance Databricks implementations and data management processes.
• Develop and maintain documentation for Databricks solutions, including architecture diagrams, data flow diagrams, and technical specifications.
• Stay current with industry trends, emerging technologies, and best practices in Databricks, data engineering, and data management.

• Bachelor's or master's degree in Computer Science, Information Systems, or a related field.
• Minimum of 5 years of overall experience in software engineering, with at least 3 years of hands-on experience with Databricks application.
• Extensive experience in CI/CD processes and tools, such as Jenkins, GitHub Actions, or Azure DevOps.
• Proficiency in managing Databricks on Azure Cloud, including performance tuning, scaling, and cost management.
• Strong knowledge of data integration, transformation, and analytics processes within Databricks.
• Proficiency in programming languages, such as SQL, Python, or Scala.
• Familiarity with data modeling, data quality, and data governance concepts and best practices.
• Familiarity with Terraform IaC is beneficial.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.