OptimHire’s Job Auto-Applier that will automatically apply to jobs at 300,000+ Companies.
Job Description: Senior Engineer – AI/ML (Databricks Certified)
We are seeking a highly skilled AI Engineer with proven experience in developing, deploying, and
optimizing AI models using the Databricks AI platform. The ideal candidate will have a strong
background in machine learning, distributed data processing, and will be a champion of Databricks
platform.
Responsibilities
Generative AI: Experience of LLM-based solutions (LlamaIndex, LangChain, RAG
pipelines, or similar frameworks). Ability to integrate GenAI and Agentic AI into business
workflows.
Design and implement end-to-end Generative AI solutions on Databricks, leveraging
Unity Catalog, MLflow, Delta Lake, and Vector Search
Design, build, and deploy large-scale AI/ML models using the Databricks environment.
Implement data validation, lineage, and monitoring using Delta Live Tables and Unity
Catalog.
Leverage Databricks’ data engineering workflows for feature engineering, model training, and
evaluation.
Optimize training pipelines for efficiency, scalability, and accuracy.
Integrate AI models into production systems using APIs and microservices.
Build reusable ML pipelines using Databricks Repos, MLflow, and Feature Store.
Implement robust testing, monitoring, and retraining protocols for deployed models.
Ensure adherence to compliance, security, and performance standards.
Stay updated on advancements in AI frameworks, distributed computing, and Databricks
platform updates.
Required Skills and Qualifications
Bachelor’s or Master’s degree in Computer Science, Data Science, or related field.
Cerified Databricks Certified Generative AI Engineer
Proven experience developing AI solutions on Databricks.
Strong knowledge of Python, PySpark, MLflow, Spark and Databricks Notebooks.
Strong understanding and knowlege of Databricks platform features such as Unity Catalog, DLT,
MosaicAI, Data Assets Bundles, etc.
Experience with Transformer-based models, generative AI, and Databricks pipelines.
Proficiency in integrating AI models with cloud-native architectures (AWS, Azure, or GCP).
Solid understanding of MLOps practices, Data Assets bundles (CI/CD), and
containerization (Docker, Kubernetes) on Databricks Platform.
Familiarity with vector databases, embeddings, and retrieval-augmented generation
(RAG).
Strong problem-solving, analytical thinking, and communication skills.
Software Engineer
Data Engineer
AIML Engineer
AIML/LLM Engineer
Python - 8 years
Generative AI - 8 years
Databricks - 5 years
Unity catalogue - 4 years
ML flow - 4 years
delta lake - 5 years
vector search - 4 years
PySpark - 5 years