Hi,

Senior Software Engineer – JAVA(3years) (LLM Evaluation & Repository Validation) 

Skills Required: Java, Git and Docker 

What does day-to-day look like: 

• Analyze and triage GitHub issues across trending open-source libraries. 

• Set up and configure code repositories, including Dockerization and environment setup. 

• Evaluating unit test coverage and quality. 

• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios. 

• Collaborate with researchers to design and identify repositories and issues that are 

challenging for LLMs. 

• Opportunities to lead a team of junior engineers to collaborate on projects. 

Required Skills: 

• Strong experience with Java, 

• Proficiency with Git, Docker, and basic software pipeline setup. 

• Ability to understand and navigate complex codebases. 

• Comfortable running, modifying, and testing real-world projects locally. 

• Experience contributing to or evaluating open-source projects is a plus. 

Nice to Have: 

• Previous participation in LLM research or evaluation projects. 

• Experience building or testing developer tools or automation agents.