I think the battle is for the neutrality of the technology.

This is the story of humanity, making knowledge access more fluid.

Basically in 2021, every paper made this mistake.

It means that we're only trusting the team of... one, every paper made this mistake.

It means that we're only trusting the team of large companies

to figure out ways of addressing these problems.

All of the people that joined us as well,

deeply regretted because we think that we are definitely not at the end of the story.

As it turns out, if you look at the history of software, the only way we did software collaboratively

is through open source.

So why change the recipe? Scaling loss.

These underpin the success of large language models today,

but the relationship between data sets, compute,

and the number of parameters was not always clear.

But in 2022, a pivotal paper came out,

often referred to as Chinchilla

that changed the way that many people in the research community

thought about that very calculus,

demonstrating that

data sets were actually more important than just the sheer size of the model.

One of the key authors behind that paper was Arthur Metched, who was working at Deep Mind at the time.

Now earlier this year, Arthur banded together with Giam Lampo and Timothy Lequois, two researchers

...