About the role

Key Responsibilities

• Run and Monitor Inferences
• Execute batch inference jobs on Databricks and other environments.
• Troubleshoot inference issues, log results, and escalate model anomalies.
• Pipeline & Evaluation Support
• Assist with running DSPy pipelines and loading custom models/configs.
• Support evaluation runs using internal metrics
• Track and summarize performance metrics for inference experiments.
• Data Handling & Experimentation
• Help prepare and clean legal text datasets for inference and evaluation.
• Annotate outputs to support error analysis and benchmarking.
• Maintain reproducible scripts/notebooks for inference runs.

Qualifications

• Background in Computer Science, Data Science, or AI/ML (student or recent grad is good).
• Familiarity with Python and libraries like pandas, scikit-learn, or PyTorch.
• Exposure to LLM frameworks (DSPy, LangChain, Hugging Face, or equivalent).
• Comfortable working with Databricks / MLflow or other cloud ML environments.
• Detail-oriented with strong organizational skills for tracking experiments.

Evaluation

• Technical Skills
• LLM frameworks
• Databricks/MLflow

Must Have

• Must have experience with Azure data tools - azure, databricks, MLflow, python
• Exposure to LLM frameworks (DSPy, LangChain, Hugging Face, or equivalent).
• Familiarity with Python and libraries like pandas, scikit-learn, or PyTorch.