yixuantt/InvestLM-Llama-3.1-70B

InvestLM

This is the repo for a new financial domain large language model, InvestLM, tuned on meta-llama/Meta-Llama-3.1-70B, using a carefully curated instruction dataset related to financial investment. We provide guidance on how to use InvestLM for inference.

Github Link: InvestLM

Generation

tokenizer = AutoTokenizer.from_pretrained("yixuantt/InvestLM-Llama-3.1-70B",
                                          padding_side = "left",
                                          max_seq_length = 8192)

model = AutoModelForCausalLM.from_pretrained("yixuantt/InvestLM-Llama-3.1-70B", 
                                             torch_dtype= torch.bfloat16,
                                             device_map = "auto",
                                             attn_implementation="sdpa", 
                                             )

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.eval()

def generate(text):
    text = text.strip()
    messages = [
        {"role": "system", "content": "You are a pirate finance AI assistant."},
        {"role": "user", "content": text}
    ]
    text = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True
    )
    input_ids = tokenizer(text, 
                        return_tensors="pt").to(device)
    
    with torch.no_grad():
        generation_output = model.generate(**input_ids,
                                max_length=8192,
                                do_sample = False,
                                pad_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(generation_output[0], skip_special_tokens=False)
    return response

Example Prompt

First, please extract the sentences related to data points for the {asset} markets. Group these sentences by region. Second, extract the author's views on future development of the {asset} markets. Group these sentences by region.
Prioritize to extract the most impactful sentences.

{input}

You are the {role} and you are to compile a brief daily market summary update across the {asset} markets for the purposes of facilitating investment decisions. Please provide reference for your claims.

Here is an example Input:
```Document Name: 000009
James is a writer.```
 
If you were to provide sources, you would say:
James is a writer. (Document: 000009)

First, please list the relevant data across the {asset} markets and group these sentences by region. Second, compare and contrast the authors' views on future development of the {asset} markets, do so by region. \nProvide sources to substantiate your claims, only from this prompt, never from elsewhere.

Input:

{input}