---
license: apache-2.0
datasets:
- argilla/distilabel-intel-orca-dpo-pairs
base_model: sethuiyer/Chikuma_10.7B
library_name: transformers
pipeline_tag: text-generation
tags:
  - dpo
  
---

# Chikuma_10.7B - V2 (Enhanced with DPO)

<p align="center">
  <img src="https://huggingface.co/sethuiyer/distilabled_Chikuma_10.7B/resolve/main/chikuma_v2.webp" height="256px" alt="Chikuma">
</p>


This model is the **DPO fine tuned version** of [Chikuma_10.7B](https://huggingface.co/sethuiyer/Chikuma_10.7B), which was a depth upscaled merge of:
* [sethuiyer/SynthIQ-7b](https://huggingface.co/sethuiyer/SynthIQ-7b)
* [openchat/openchat-3.5-0106](https://huggingface.co/openchat/openchat-3.5-0106)

The name "Chikuma" is inspired by the [Chikuma River](https://en.wikipedia.org/wiki/Shinano_River), the longest in Japan, known for its continuous flow and meandering path. 
This metaphorically represents the model's depth, fluidity, and adaptability in processing and understanding language.


# Dataset used for Fine Tuning
Dataset: `/argilla/distilabel-intel-orca-dpo-pairs`

The dataset was roughly ~3000 samples but they were high quality (according to the chosen_score).  

The following filters were applied to the original dataset:
```python
dataset = dataset.filter(
    lambda r:
        r["status"] != "tie" and
        r["chosen_score"] >= 8 and
        not r["in_gsm8k_train"]
)
```

# Chat Template
The chat template for Chikuma_10.7B - V2 is a modified version of ChatML, optimized for improved interaction and engagement:

```
<|im_start|>GPT4 Correct system:
{system} Always use <|end_of_turn|> when you want to end the answer. <|im_end|>
<|im_start|>GPT4 Correct user:
{user}<|im_end|>
<|im_start|>GPT4 Correct Assistant:
{asistant}<|im_end|>
```

### Training Environment
- Hardware: Single A100 80GB GPU in a runpod, utilized for approximately 1.5 hours.
- Training Script: Accessible via [Google Colab Notebook](https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing). Special thanks to [mlabonne](https://huggingface.co/mlabonne) for providing the template.


## Usage

```python
# Format prompt
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(new_model)

# Create pipeline
pipeline = transformers.pipeline(
    "text-generation",
    model=new_model,
    tokenizer=tokenizer,
    device="cuda"
)

# Generate text

message = [
    {"role": "system", "content": "You are a helpful assistant chatbot."},
    {"role": "user", "content": "Who invented LLMs?"}
]

prompt = tokenizer.apply_chat_template(message, add_generation_prompt=True, tokenize=False)

sequences = pipeline(
    prompt,
    max_length=512
)
print(sequences[0]['generated_text'])
```

**Answer**:
```markdown
Large Language Models (LLMs) are a type of artificial intelligence model that are designed to understand and generate human-like text. The concept of LLMs has evolved over time, with various researchers and companies contributing to their development. However, the term "LLM" itself is relatively recent and has become more popular with the success of models like GPT-3 and GPT-4.

The origins of LLMs can be traced back to the early days of machine learning and natural language processing (NLP). One of the pioneers in this field is Geoffrey Hinton, a British-Canadian computer scientist who is often referred to as the "Godfather of Deep Learning." Hinton and his team at the University of Toronto developed the first neural network-based language model called the "Recurrent Neural Network" (RNN) in the early 1990s.

Another important figure in the development of LLMs is Yann LeCun, a French computer scientist who is known as the "father of deep learning." LeCun and his team at the University of Montreal developed the first neural network-based language model called the "Long Short-Term Memory" (LSTM) in 1990.

In recent years, OpenAI, a research organization founded by Elon Musk and others, has been a major player in the development of LLMs. OpenAI's GPT (Generative Pre-trained Transformer) series of models, including GPT-2, GPT-3, and now GPT-4, have been influential in advancing the field of LLMs.

In summary, while there is no single individual who can be credited with inventing LLMs, several researchers and organizations have played crucial roles in their development. Key contributors include Geoffrey Hinton, Yann LeCun, and OpenAI.
```

## Things in Pipeline:
1. Manual Testing and Evaluation against GPT-4 on text-generation-webui across 45 sample complex prompts.
2. Nous Benchmark
3. GGUF Format
4. Ollama Model (if model benchmarks are good)

## Acknowledgements

A heartfelt appreciation goes to the vibrant open-source community, particularly:

* The Intel team for publishing a great open dataset and show how well it worked in the first place 
* Teknium and NousResearch for their awesome work and models.
* Maxime for sharing such great resources.
* Argilla for publishing argilla/distilabel-intel-orca-dpo-pairs