Open weight models are AI systems whose trained parameters are publicly released, which allows

developers to run, fine-tune, and deploy them independently rather than accessing them only

through a hosted API. While closed-weight models from companies like Open AI or Anthropic

are delivered as managed services, open weight models give organizations direct control

over how the models are deployed and used. Importantly, the performance of these models is steadily

improving and they've become credible alternatives for production workloads, with advantages

and customization and data privacy. Fireworks AI is building a platform focused on serving and customizing open weight models at scale.

The platform includes optimize inference infrastructure, multi-hardware support across

NVIDIA and AMD, and reinforcement fine-tuning capabilities.

Benny Chen is a co-founder of Fireworks AI.

In this episode, he joins Gregorvan to discuss his path from Meta's ML infrastructure teams

to co-founding Fireworks AI, why open-weight models are becoming increasingly competitive,

how custom kernels and speculative decoding improved performance,

reinforcement fine-tuning, and much more.

Gregor Vand is a security-focused technologist, having previously been a CTO across cybersecurity,

cyber insurance, and general software engineering companies.

He is based in Singapore and can be found via his profile

at van.hk or on LinkedIn.

Hello and welcome to Software Engineering Daily. My guest today is Benny Chen.

Welcome, Benny.

Thanks for having me.

Yeah, great to have you here.

...