This large language model from Microsoft literally broke the internet.

I mean, all the YouTube influencers, you go to Twitter, on LinkedIn, everywhere, people were talking about this model.

And like, literally everybody was fascinated about this model.

This model is called Orca and this came from Microsoft.

Now, one of the reasons why people were fascinated about this model is also because it came from Microsoft.

Microsoft has a really good partnership with OpenAI and Microsoft released a new model, which was itself surprising.

And second, the model was all it was doing is trying to imitate GPT-4.

That's all.

The main catch here is that it was not trying to imitate GPT-4 like what they were doing with Vikuna or any other model.

It took a completely radically new approach of how you train a large language model.

Or, I should say, how you fine-tune a large language model.

And that's one of the reasons why people were fascinated about it.

So, in this video, I wanted to get into the details of how this model has been built and what is this model, in fact, doing and why is it fascinating.

And what is it taking us to?

To start with, if you are not familiar about the model fine-tuning process, a large language model fine-tuning process, I've got a separate video where we delve die deep into how to fine-tune your large language model.

Like, we used PIV to fine-tune it.

But typically, whenever you want to fine-tune a large language model, again, like if you do not even know what is fine-tuning, just think of having a raw model.

Like, you have, first of all, a raw model, like a big, really raw model.

And using that raw model, you are going to add some dataset to it.

You're going to add your own dataset.

Like, for example, you have got some data in it.

And you're going to use that data to this raw model and then create a new baby model.

That's what the fine-tuning is.

So, this baby model ideally is supposed to do certain things better than the raw model, the base model, on particular tasks that are given to this model.

This process is typically called as supervised fine-tuning (SFT) or it is also called as instruct fine-tuning.

So, you have a set of instructions with which you are fine-tuning this model.

Or, you can call it, you have got a supervision happening there and you're fine-tuning this model with this dataset.

This is the typical fine-tuning process.

For a typical fine-tuning process, you need two things.

One, you need a prompt, which is like the question or the instruction that you give to the model.

And then, the response, what kind of response it has got.

So, any model, like for example, Vikuna, what they've done is they have taken the responses from Chargpt.

The Prompt and the response, and then they try to use some base model like LLaMA or some other base model, and try to fine-tune it in this particular instruction.

And then they try to create these baby models or child models or student models, whatever you would like to call it.

This is what has been happening in the industry.

But with Orca, we have got a completely new paradigm of how these models can be fine-tuned.

And they call it Progressive learning, and in some places, they call it explanation learning.

So, whatever the name is, Progressive learning or explanation learning, what is happening with Orca is this: now, instead of taking a simple prompt and response in the concept of explanation tuning, we have got the system message, The Prompt, and the response.

Now, what is a system message?

If we have used Chargpt through API, you know, system message is something like this: you would tell Chargpt that, Hey, you are a really good coder, and then you need to solve the problem.

Or you would probably tell ChatGPT, You are a teacher, and you need to solve this problem.

So, you set a context to this large language model so that they behave or imitate the behavior of the role that they've been given.

So, for example, if you want ChatGPT to solve medical problems, you would initially tell ChatGPT that you should behave like a doctor.

And the same way, every prompt goes with a system message.

And what they are doing in explanation tuning is first, they got the system message along with the prompt and along with the response.

This is the first bigger change that you see from the traditional prompt of traditional instruction fine-tuning.

After you do this thing, now the tuning does not happen in one phase.

The tuning happens in two different levels.

The first level of fine-tuning happens with ChatGPT response.

So, the ChatGPT here acts as an intermediate teacher.

So, you have got the system message, you have got the prompt, and you have got the ChatGPT messages for those particular prompts.

Like, what would ChatGPT respond in this particular case?

That's the first level of training.

In the second level of training, for the same set of data points from Flan, they are using Flan models' prompts.

They are using GPT-4 responses.

So, the first level is being taught by ChatGPT, the second level is being taught by GPT-4.

So, imagine you are a student, like you are in kindergarten, in the lower kindergarten, you would learn certain things, but that would accumulate to your knowledge that you would gain in the upper kindergarten.

That would ultimately help you understand better things when you go to the first standard or something.

So, the same way, instead of training the model at one particular phase or instead of making sure that the model has an enormous amount of data, this is a clearly crafted process where first of all, you are telling the model what it is supposed to do with the system message.

And second, you also have two levels of fine-tuning, where first you use ChatGPT responses and second, you use GPT-4 responses.

Now, this has a profound impact on these large language models, or as they call it, LFM's (Large Foundation Models). So, large foundation models are really good at certain things, but when people fine-tune it, these models learn to imitate, but they don't have the reasoning process.

But Orca, the 13 billion parameter model, learns to imitate the reasoning process of large foundation models or the large language models.

And in certain benchmarks, Orca actually comes second only to GPT-4.

It's just a 13 billion parameter model.

It is not like so expensive, but still, it does a really good job.

In fact, like GPT-4, which is a very comparable model because this is also a 13 billion parameter model, it is 100 percent better in terms of benchmarks, in terms of complex zero-shot reasoning when you compare it with Vikuna.

And that's primarily possible because of the way they are doing all this training process, which they are calling it as an explanation training or explanation tuning.

And that's exactly possible because of certain reasons.

One very importantly, system instruction.

So, system instruction plays a huge role.

And they've also got into the detail of how they construct the dataset.

You've got the system message, you've got the user query, and you have got the response from these models like ChatGPT and GPT-4.

So, like I said, there are two levels.

One is from ChatGPT, the second one is from GPT-4.

And these prompts are actually collected from Flan V2 collection.

Now, if you look at the model and the way the model is performing, the model is actually quite good in terms of how this model is doing.

And one of the reasons that they're saying is why did they use the two-level approach?

One, first to train Orca on the Flan 5 million dataset with ChatGPT augmentation.

Second, one with Flan 1 million with GPT-4 augmentation, where they are sampling it from the dataset.

So, ChatGPT is being used as an intermediate teacher assistant.

One is to close the capacity gap to a cost and time.

I mean, of course, GPT-4 is much, much cheaper than ChatGPT.

The GPT-3.5 turbo is much, much cheaper than GPT-4.

And also, you have got the rate limit.

Like, for a particular given amount of time, how many calls you can make.

So, to close the gap, the capacity gap, like to help it learn the reasoning than just learning the imitation.

And also, in terms of cost and time, they've got this two levels of ChatGPT and GPT-4 learning aspect.

And if you see the benchmarks, it's honestly quite mind-blowing the way this model has been doing.

And the way it is scoring against, let's say, in zero-shot selection or in any other setup, you can see this model doing a really, really good job.

For example, take this particular benchmark.

You can see this is human average, the blue color line, and the green color one is Orca 13 billion.

The gray color line is Star GPT, and the yellow color line is GPT-4.

If you see this particular evaluation, you can see Orca really being good, next to ChatGPT in multiple cases, in fact, better than human beings in multiple cases.

You can see that it's doing better than human beings in, like, SAT, in math, probably closer, and in other cases, like, you can see it's not actually a terrible model, and also, it's doing really good in a few other cases.

For example, you can see if you've got ChatGPT, GPT-4, Vikuna, 13 billion parameter model, it's almost a lot better than Vikuna.

Look at Vikuna model here.

Geometric shapes code 3.6, Chat Orca is 477% better than Vikuna.

Look at something called, let's say, penguins in a table.

It's 46, this is 57.

And there is one more, temporal sequences.

We could have 16, this is 72.

And it's not just one particular benchmark.

There are a lot of different benchmarks where Orca is really doing good and coming just second to GPT-4.

And if you look at GPT-4, GPT-4 is really a very expensive model.

GPT-4 is not trained in a single day.

But if you look at Orca, it was just trained on 20 NVIDIA A100s.

And I mean, like literally a couple of days back, we learned that Inflection AI ordered 22,000 NVIDIA A100s.

And in here, you see 20 NVIDIA A100 GPUs with 80 gigs of memory.

It took 160 hours to train Orca with ChatGPT augmentation for four epochs.

And 40 hours, this is 160 hours, this is 40 hours, to continue the training on Flan 1 million dataset with GPT-4 augmentation for the same number of epochs, which is four epochs.

So, four epochs here, four epochs here, 160 hours here, 40 hours here.

This is all that was required for the actual training process.

And for the dataset collection, it took two weeks to collect data from GPT-3.5 turbo, which is the ChatGPT, and three weeks for GPT-4 from multiple endpoints, because, you know, you have got the rate limitation on all these things.

So, overall, two plus three is equal to five weeks.

And then you have got about like 200, 200 hours.

You can say, let's say one week, so two five week plus one week, almost in seven weeks.

You have got a model that is only next to GPT4.

I mean, of course, you can now argue that if there is no GPT4, Orca would not exist because, you know, Orca is trying to imitate GPT4, which I completely understand.

But again, the problem, the point here is that it didn't require the data as much as GPT4.

It didn't require computation as much as GPT4.

And with a good training process, a handcrafted dataset, and a really good teacher model, Orca is doing much, much better than any other fine-tuning model that is available.

And that is quite good if you are in the open-source large language model ecosystem.

So, this is absolutely brilliant.

Like, overall, we have learned that Orca is really amazing.

Explanation fine-tuning is amazing.

Progressive learning is amazing.

And this is like a great initiative and approach from Microsoft.

The model weights are not being released.

They said they are still discussing with the legal department on board how to go about releasing the model itself.

So, that is still there.

But the direction in which this is going is this paper has given an idea to a lot of people.

And you can see open-source models already popping up.

Like, we have got Orca Mini 3B.

So, there is a open Orca, which is like with Apache 2.0 license.

And people have started using the philosophy and fundamentals that have been shared on the Orca paper.

And people have started releasing the model itself.

I've not personally used this model yet to understand how good this model is.

But these kinds of models have started popping up.

Like, when you go to Hugging Face model home and start looking for Orca, you can see Orca Mini 13 billion, Orca Mini 3 billion, and the Blokes GCML version of the same model.

You've got datasets, open Orca wizard element.

So, you've got open-source datasets that can help you fine-tune Orca-like training.

And then you've got models already, which 13 billion and 3 billion parameter models.

In fact, including the GGML models that can help you use the models that are fine-tuned on this open Orca dataset.

So, this seems like an entirely new direction in the fine-tuning world.

And Microsoft seems to have seeded this new growth with this new model.

So, kudos to the authors.

And it has really good capabilities.

Like, if you compare it with GPT-4, if you compare it with other models that are available, Orca is really, really good.

And I'm quite surprised.

This is a great approach.

It's almost like how you train a student.

And now the fact that you can fine-tune a large language model, or sorry, you can find you in a large language model that can imitate a large language model or the base model with not just imitation, but also the reasoning part is quite amazing.

And that's why probably Orca has broken the internet.

Like, everybody has been talking about it.

And I've been looking at this model for quite a while.

Unfortunately, now I got time to make a video about this thing.

Please let me know in the comment section what do you feel about Orca?

Did you try Orca, the open-source model like the Orca Mini?

Or if you have got any other question that you would like me to answer, see you in another video.

Happy prompting.