Sr. Big Data Engineer (Java, Apache Spark and Data Architecture)

Job Description

Job Summary:

You will be responsible for translating functional requirements into robust data solutions, collaborating with global teams, and mentoring junior developers.

Overall Responsibilities:

• Translate application storyboards and use cases into functional applications.
• Design, build, and maintain efficient, reusable, and reliable Java code.
• Ensure optimal performance, quality, and responsiveness of applications.
• Identify bottlenecks and bugs, and devise appropriate solutions.
• Develop high-performance and low-latency components to run Spark clusters.
• Interpret functional requirements into design approaches suitable for the Big Data platform.
• Collaborate with global teams across various locations to deliver high-quality solutions.
• Propose best practices and standards, and ensure smooth handover to the operations team.
• Conduct testing of software prototypes and facilitate the transfer to operational teams.
• Process data using Hive, Impala, and HBase.
• Analyze large data sets to derive actionable insights.

Technical Skills:

Core Skills:

• Solid understanding of object-oriented programming and design patterns.
• Strong experience with Java (8+ years) using Java 1.8 or higher.
• Proficiency in working with large data volumes and logical data structures.

Big Data Technologies:

• Experience with HDFS, Hive, HBase, Apache Spark, and Kafka.
• Familiarity with building self-service platform-agnostic data access APIs.
• Understanding of service-oriented architecture and data standards (JSON, Avro, Parquet).

Analytical Skills:

• Experience in building advanced analytical models based on business context.
• Strong systems analysis, design, and architecture fundamentals.

Development Tools:

• Familiarity with Agile/Scrum methodologies.
• Experience with source control management tools like GIT and project management tools like JIRA.
• Basic proficiency in Linux shell scripting.
• Understanding of RDMS and NoSQL databases.

Additional Skills:

• Application performance tuning and troubleshooting in the Big Data domain.
• Ability to write reliable, manageable, and high-performance code.
• Knowledge of database principles, SQL development (preferably with Oracle).
• Familiarity with concurrency patterns and multithreading in Java.
• Understanding of domain design concepts, JDBC, and RESTful services.

Optional:

• Familiarity with Arcadia Tool for Analytics.
• Understanding of cloud and container technologies.
• Experience with build tools such as Maven and continuous integration tools like Jenkins/TeamCity.

Experience:

• Minimum of 8 years of experience in software development, with a focus on Big Data technologies.
• Proven experience in mentoring and guiding other developers within a team.
• Demonstrated analytical and problem-solving skills.

Day-to-Day Activities:

• Participate in daily stand-up meetings and sprint planning sessions.
• Collaborate with cross-functional teams to understand business requirements.
• Write, test, and deploy scalable software solutions.
• Conduct code reviews and provide feedback to team members.
• Stay updated with the latest trends and advancements in Big Data technologies.
• Provide technical support and mentorship to team members.

Qualifications:

• Bachelor s or Master s degree in Computer Science, Information Technology, or a related field.

Soft Skills:

• Excellent written and verbal communication skills.
• Strong collaboration and teamwork abilities.
• Exceptional problem-solving and analytical skills.
• Ability to adapt to new technologies and changing requirements.
• Strong time management and prioritization skills.