WOW - best opensource llm I ever seen !

#1
by mirek190 - opened

Look on answers
version - mixtral-8x7b-instruct-v0.1.Q4_K_M
Is fast - around 40 t/s ( rtx 3090 - 30 layers on gpu ) for answering but question processing takes around 15 seconds ( probably because of not fully implemented under llamacpp)

I not even restarted that model just asked everything in one session!


>  <s>[INST] There are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room? Explain your reasoning step by step.
 [/INST] The answer is that there are still three killers in the room. Here's why:

1. Initially, we have three killers in the room (A, B, and C).
2. Then, another person enters the room and kills one of them, let's say A. This means that the number of killers decreases by one, but at the same time, the person who entered becomes a killer too.
3. So, even though one original killer (A) has been removed from the equation, they have been replaced by the new entrant who is also a killer.
4. Therefore, the total count of killers remains the same, which is three (B, C, and the new entrant).

>  <s>[INST] Mark is faster than Mary , Mary is faster than Joe. Is Joe faster than Mark?
 [/INST] No, Joe is not faster than Mark based on the given information. Here's why:

1. We know that Mark is faster than Mary.
2. We also know that Mary is faster than Joe.
3. If Mark were slower than Mary, then it would be possible for him to be slower than Joe as well, but since we are told that Mark is faster than Mary, this isn't the case.
4. Given these two pieces of information, Mark remains faster than both Mary and Joe. Thus, Joe cannot be faster than Mark in this scenario.

>  <s>[INST] If I lay 2 wet shirts out in the sun flat on the ground to dry and it takes 5 hours until they are dry, how long would it take to dry 4 shirts of the same type that way simultanously?
 [/INST] Assuming that you lay all four wet shirts out in the sun at the same time and they all receive an equal amount of sunlight, then each shirt will take the same amount of time to dry as one shirt does. In this case, since two shirts took 5 hours to dry, drying four shirts should still take 5 hours.

Here's the step-by-step reasoning:
1. Two wet shirts dried in 5 hours when laid out individually.
2. When laying out multiple items for drying, as long as they receive equal exposure to sunlight, their drying time does not increase due to being placed together (assuming similar conditions).
3. Therefore, if you lay four wet shirts out simultaneously and evenly spaced in the same conditions, each shirt should still take 5 hours to dry because they are receiving an equal amount of sunlight as the two shirts did before.

So, it would take 5 hours to dry 4 shirts if laid out simultaneously under similar conditions.

>  <s>[INST] explain step by step 25-4*2+3=?
 [/INST] Let's break down the given expression step by step, following the order of operations (PEMDAS):

Expression: 25 - 4 * 2 + 3

Step 1: Parentheses and Exponents (none in this case)
Step 2: Multiplication and Division (from left to right)
   - Perform multiplication first: 4 * 2 = 8

New Expression: 25 - 8 + 3

Step 3: Addition and Subtraction (from left to right)
   - First, perform subtraction: 25 - 8 = 17

New Expression: 17 + 3

Step 4: Perform the final addition
   - Lastly, add the remaining numbers: 17 + 3 = 20

Therefore, the result of the given expression is 20.

Expression: 25 - 4 * 2 + 3 = 20

>  <s>[INST]

Even THIS question is almost answered properly - I am really impressed.
bad answer - 6
fair - 5
good - 3

>  <s>[INST] There are two ducks in front of a duck, two ducks behind a duck and a duck in the middle. How many ducks are there?
 [/INST] There are five ducks in total. Here's how we can figure this out:

1. We have two ducks in front of another duck, which means one duck has two ducks in front of it.
2. Similarly, there are two ducks behind another duck, indicating that one duck has two ducks behind it.
3. Then, there is a duck placed in the middle, between the first mentioned group (two ducks in front) and the second mentioned group (two ducks behind).
4. So, if we count all these ducks together, we have:
   - Two ducks in front
   - One duck in the middle
   - Two ducks behind
5. Adding them up, there are five ducks in total.

CODING:

>  <s>[INST] Build a webpage in one html file with html, css and javascript code, the webpage should have 10 jokes and the button that changing background colors randomly.
 [/INST] Here's a simple HTML file that contains 10 jokes and a button to change the background color randomly:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <style>
        body {
            font-family: Arial, sans-serif;
        }

        #jokes {
            margin-bottom: 2rem;
        }

        .joke {
            margin-bottom: 0.5rem;
        }

        button {
            padding: 0.5rem 1rem;
            font-size: 1.2rem;
            cursor: pointer;
        }
    </style>
</head>
<body>
    <div id="jokes">
        <p class="joke">Why did the chicken go to the seance? To talk to the other side!</p>
        <p class="joke">How do you catch a squirrel? Climb a tree and act like a nut!</p>
        <p class="joke">What do you call fake spaghetti? An impasta!</p>
        <p class="joke">Why did the tomato turn red? Because it saw the salad dressing!</p>
        <p class="joke">How do you organize a space party? You planet!</p>
        <p class="joke">What do you call a sleeping bull? A bulldozer!</p>
        <p class="joke">Why was the math book sad? Because it had too many problems!</p>
        <p class="joke">How do you make a tissue dance? You put a little boogie in it!</p>
        <p class="joke">What do you call cheese that isn't yours? Nacho cheese!</p>
        <p class="joke">Why did the scarecrow win an award? Because he was outstanding in his field!</p>
    </div>

    <button id="change-bg">Change Background Color</button>

    <script>
        document.getElementById('change-bg').addEventListener('click', function() {
            const colors = ['#F44336', '#E91E63', '#9C27B0', '#673AB7', '#3F51B5', '#2196F3', '#03A9F4', '#00BCD4', '#009688', '#4CAF50', '#8BC34A', '#CDDC39', '#FFEB3B', '#FFC107', '#FF9800', '#FF5722'];
            const randomColor = colors[Math.floor(Math.random() * colors.length)];
            document.body.style.backgroundColor = randomColor;
        });
    </script>
</body>
</html>

This HTML file includes 10 jokes displayed in the "jokes" div element and a button to change the background color randomly using JavaScript. The JavaScript code adds an event listener to the button, so that when clicked, it selects a random color from the provided array and applies it as the background color for the body of the webpage.


>  <s>[INST] Provide complete working code for a realistic looking tree in Python using the Turtle graphics library and a recursive algorithm.
 [/INST] Here's a Python program using the Turtle module that generates a simple, yet realistic-looking tree using a recursive algorithm:

import turtle

def draw_branch(length, angle, t):
    if length <= 5:
        return

    # Draw the branch
    t.forward(length)

    # Change the drawing direction and pencil width for the next branches
    t.right(angle)
    draw_branch(length * 0.67, angle, t)
    t.left(2 * angle)
    draw_branch(length * 0.67, angle, t)
    t.right(angle)

    # Move the turtle back to the starting point of the branch
    t.backward(length)

def main():
    window = turtle.Screen()
    window.bgcolor("skyblue")
    window.title("Turtle Tree")

    my_turtle = turtle.Turtle()
    my_turtle.speed(0)
    my_turtle.pencolor("brown")

    # Draw the trunk of the tree
    my_turtle.right(90)
    my_turtle.forward(150)
    my_turtle.right(90)

    # Move the turtle to start drawing the branches
    my_turtle.penup()
    my_turtle.backward(150)
    my_turtle.pendown()

    # Draw the first level of branches
    draw_branch(100, 25, my_turtle)

    turtle.done()

if __name__ == "__main__":
    main()

This program creates a simple tree using the Turtle graphics library and recursion. The `draw_branch` function takes care of drawing each branch and its sub-branches, while the main function sets up the Turtle environment and initializes the turtle to draw the trunk and branches. By changing the parameters of the `draw_branch` function, you can easily adjust the appearance of the tree to your liking. The `turtle.done()` call is essential for the turtle graphics window to remain open after running the script.

>  <s>[INST] Create a webpage with css and javascript in one file. The web page should be for the shop where is selling  waterboards also add a button which will be changing background colors randomly.
 [/INST] Here's a simple HTML, CSS, and JavaScript code combined into a single file for a webpage showcasing a fictional waterboard product with a button to change the background color randomly:

```html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <style>
        body {
            font-family: Arial, sans-serif;
        }

        h1 {
            text-align: center;
            padding: 2rem 0;
        }

        .product {
            display: flex;
            justify-content: space-around;
            align-items: center;
            height: 250px;
            border: 1px solid #ccc;
            border-radius: 10px;
            padding: 1rem;
            margin: 2rem auto;
            width: 80%;
            max-width: 600px;
        }

        .product img {
            height: 100%;
            object-fit: contain;
        }

        button {
            padding: 0.5rem 1rem;
            font-size: 1.2rem;
            cursor: pointer;
            margin-top: 2rem;
            display: block;
            margin: auto;
        }
    </style>
</head>
<body>
    <h1>Welcome to our Waterboard Shop</h1>
    <div class="product">
        <img src="https://via.placeholder.com/300x250?text=Waterboard" alt="Waterboard Product Image">
        <p>Experience the thrill of waterboarding with our premium product!</p>
    </div>

    <button id="change-bg">Change Background Color</button>

    <script>
        document.getElementById('change-bg').addEventListener('click', function() {
            const colors = ['#F44336', '#E91E63', '#9C27B0', '#673AB7', '#3F51B5', '#2196F3', '#03A9F4', '#00BCD4', '#009688', '#4CAF50', '#8BC34A', '#CDDC39', '#FFEB3B', '#FFC107', '#FF9800', '#FF5722'];
            const randomColor = colors[Math.floor(Math.random() * colors.length)];
            document.body.style.backgroundColor = randomColor;
        });
    </script>
</body>
</html>
```

This HTML, CSS, and JavaScript code combines a simple webpage showcasing a fictional waterboard product with an image, title, and a button to change the background color randomly. The CSS styles are applied to create a clean layout for the content. The JavaScript code adds an event listener to the button, so that when clicked, it selects a random color from the provided array and applies it as the background color for the body of the webpage.

>  <s>[INST]

for question "Provide complete working code for a realistic looking tree in Python using the Turtle graphics library and a recursive algorithm"
Screenshot 2023-12-11 195152.png

Everything works!
Is coding better than the best opensource model for coding like phind-codellama-34b-v2 .... no words

ANY known me model answered all those questions properly ... all of then at once!
That is a completely new level in opensource llms.

Just so far the best open llm.
Chat version is a crap but INSTRUCTION IS THE BEST so far.
Reasoning level is improved significantly compared to everything we had so far in opensource.

Can you share prompt template please?

The amount of killers in the room should be 4 just because a killer died does not mean that person is no longer a killer. The 4th person to enter the room with 3 killers already in the room became a 4th killer as soon as he murdered one of the killers. Dying does not take away your killer status. Just as when a person dies the gender of a person does not change just because the person died. If you want to get even more specific there is 3 killers alive in the room and 1 dead killer.

Still this model however performs better than a lot of other quantized open source models do and even a lot better than a lot of humans do with answering this question. I see great progress in open source models when it comes to reasoning.

The amount of killers in the room should be 4 just because a killer died does not mean that person is no longer a killer. The 4th person to enter the room with 3 killers already in the room became a 4th killer as soon as he murdered one of the killers. Dying does not take away your killer status. Just as when a person dies the gender of a person does not change just because the person died. If you want to get even more specific there is 3 killers alive in the room and 1 dead killer.

Yes and no... It depends ;)

So far every model answered 2 not counting third killer who just killed so progress in reasoning is huge.

I have not tested this model yet but if it is bad at chat and great at instruct just need to figure out a way to make it multimodal when it comes to chatting and instruct without it causing a loss on either side or reduction in performance it is just a matter of time before this is achieved and AI will advance faster and faster. a good start would be to improve on the dataset even more.

Can you share prompt template please?

If you are using llamacpp

--in-prefix " <s>[INST] " --in-suffix " [/INST] "

The amount of killers in the room should be 4 just because a killer died does not mean that person is no longer a killer. The 4th person to enter the room with 3 killers already in the room became a 4th killer as soon as he murdered one of the killers. Dying does not take away your killer status. Just as when a person dies the gender of a person does not change just because the person died. If you want to get even more specific there is 3 killers alive in the room and 1 dead killer.

Yes and no... It depends ;)

So far every model answered 2 not counting third killer who just killed so progress in reasoning is huge.

How could you come to a number other than 4 killers in the room to be correct?

Can you share prompt template please?

If you are using llamacpp

--in-prefix " <s>[INST] " --in-suffix " [/INST] "

how can we add system prompt ?

is this format ?

<s> {sys_prompt} [INST] {user_prompt} [/INST]

system prompt?
-p "{prompt}"

Can you share prompt template please?

If you are using llamacpp

--in-prefix " <s>[INST] " --in-suffix " [/INST] "

how can we add system prompt ?

is this format ?

<s> {sys_prompt} [INST] {user_prompt} [/INST]

system prompt?
-p "{prompt}"

i got it:
system prompt is like this: "you are a helpful, xxxxx"

<s>[INST] <<SYS>>
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.  Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
<</SYS>>

<prompt> [/INST] <answer> </s><s>[INST] <prompt-second> [/INST]

The amount of killers in the room should be 4 just because a killer died does not mean that person is no longer a killer. The 4th person to enter the room with 3 killers already in the room became a 4th killer as soon as he murdered one of the killers. Dying does not take away your killer status. Just as when a person dies the gender of a person does not change just because the person died. If you want to get even more specific there is 3 killers alive in the room and 1 dead killer.

Interesting. I just watched a video where Mixtral 8x7B got this question correct (4 including the dead killer). I used hugging face chat, which I suppose means they might be using the quantized model which might produce slightly less reliable results? Overall though, it answers tough LLM questions better than 3.5-turbo.. but in benchmarks it was slightly lower ranked (<1%). Despite the inconsistency, it is very impressive how far free and open source options have came!

The amount of killers in the room should be 4 just because a killer died does not mean that person is no longer a killer. The 4th person to enter the room with 3 killers already in the room became a 4th killer as soon as he murdered one of the killers. Dying does not take away your killer status. Just as when a person dies the gender of a person does not change just because the person died. If you want to get even more specific there is 3 killers alive in the room and 1 dead killer.

Interesting. I just watched a video where Mixtral 8x7B got this question correct (4 including the dead killer). I used hugging face chat, which I suppose means they might be using the quantized model which might produce slightly less reliable results? Overall though, it answers tough LLM questions better than 3.5-turbo.. but in benchmarks it was slightly lower ranked (<1%). Despite the inconsistency, it is very impressive how far free and open source options have came!

I suspect most 7b high rank models are more or less contaminated ( we need better tests ... ) because from my own tests any 7b,13,34,79b is not even close in reasoning, trueful , arc, math , programming comparing to this MoE model.

This comment has been hidden

Look on answers
version - mixtral-8x7b-instruct-v0.1.Q4_K_M
Is fast - around 40 t/s ( rtx 3090 - 30 layers on gpu ) for answering but question processing takes around 15 seconds ( probably because of not fully implemented under llamacpp)

What commandline are you using? I can't run it at all on llama.cpp, even with batch size 1.

CUDA_VISIBLE_DEVICES=0 ./server --model /sourcedir/LLM_models/TheBloke_Mixtral-8x7B-Instruct-v0.1-GGUF/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf --port 4664 --n-gpu-layers 30 --batch-size 1 --threads 4 --threads-batch 1 --numa --mlock --ctx-size 16384 -cb

OOMs.

Look on answers
version - mixtral-8x7b-instruct-v0.1.Q4_K_M
Is fast - around 40 t/s ( rtx 3090 - 30 layers on gpu ) for answering but question processing takes around 15 seconds ( probably because of not fully implemented under llamacpp)

What commandline are you using? I can't run it at all on llama.cpp, even with batch size 1.

CUDA_VISIBLE_DEVICES=0 ./server --model /sourcedir/LLM_models/TheBloke_Mixtral-8x7B-Instruct-v0.1-GGUF/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf --port 4664 --n-gpu-layers 30 --batch-size 1 --threads 4 --threads-batch 1 --numa --mlock --ctx-size 16384 -cb

OOMs.

I have rtx 3090 ( 24 GB Vram ) 

main.exe --model models\new3\mixtral-8x7b-instruct-v0.1.Q5_0.gguf --mlock --color --threads 30 --keep -1 --batch_size 512 --n_predict -1 --repeat_penalty 1.1 --ctx_size 8192 --interactive --instruct --reverse-prompt "<|im_end|>" -ngl 24 --simple-io --in-prefix " [INST] " --in-suffix " [/INST] " -p "<s>[INST] Odpowiadaj krotko i na temat [/INST]"

Sign up or log in to comment