Distributed LLM Inference Engineer
170k - 247k USD
On-site
Full Time
#Engineering
#Deep Learning
#Distributed Systems
#PyTorch
#Data
At Anyscale, we are on a mission to make distributed computing accessible to every developer. We are the team behind Ray, the popular open-source framework that powers scalable machine learning for industry leaders like OpenAI, Uber, Spotify, and Cruise. With over 250 million dollars in funding from top-tier investors like Andreessen Horowitz, we are building the definitive platform to help teams scale AI applications from a single laptop to massive clusters. We want you to help us push the boundaries of what is possible in AI infrastructure.
Role at a glance
We are looking for a Senior Distributed LLM Inference Engineer to join our team on a full-time, on-site basis in the United States. In this role, you will be at the heart of our efforts to deliver market-leading performance and cost-efficiency for large-scale AI inference.
Your impact
- Collaborate with product teams to rapidly ship end-to-end batch and online inference solutions that serve our customers at scale.
- Optimize the full stack by integrating Ray Data and our LLM engine to ensure high performance and low costs.
- Engage with the open-source community, specifically by integrating tools like vLLM and contributing your own improvements back to the ecosystem.
What you'll need
To be successful in this role, you should be comfortable working in English and possess the following qualifications:
- Strong experience running machine learning inference at scale with high throughput.
- Deep familiarity with deep learning concepts and frameworks, particularly PyTorch.
- A solid grasp of distributed systems and the unique challenges associated with ML inference.
- A passion for staying current with state-of-the-art research and implementing industry best practices.
Perks and compensation
We believe in a transparent, data-driven approach to compensation. The target salary for this position ranges from $170,112 to $247,000 USD. In addition to your salary, you will be eligible for a comprehensive benefits package, including:
- Stock options.
- Medical insurance.
- 401k retirement plan.
- Commuter benefits.
