metadata

base_model: google/gemma-2b-it
library_name: transformers
license: gemma
language:
  - ko
pipeline_tag: text-generation

Model Card for Model ID

The WriteMyPaper Gemma-2-2b-it model helps users to write abstracts of papers, fine-tuned by massive datasets of text summarization (each section and entire paper) of korean papers.

Model Details

모델을 작성하고 테스트 코드를 작성했던 과정의 튜토리얼을 https://espebaum.github.io/ml/Gemma2FineTuning 에서 확인하실 수 있습니다.
깃허브 레포지토리는 https://github.com/Espebaum/Gemma2b-it-Write-My-Paper 에서 확인하실 수 있습니다.

Model Description

This model is a fine-tuned version of the Gemma-2-2B-IT model, tailored for academic paper summarization and generation tasks. It focuses on processing large-scale academic data to use abstract summaries to generate expanded full abstracts.

Developed by: gyopark
Model type: Causal Language Model (AutoModelForCausalLM)
Language(s) (NLP): Korean
License: Gemma Term of Use
Finetuned from model [optional]: google/gemma-2-2b-it

Model Sources [optional]

Repository: https://huggingface.co/gyopark/gemma-2-2b-it-WriteMyPaper

Direct Use

This model is designed for use in academic environments where summarizing, analyzing, and generating longer texts based on research papers or technical documents is required. Users can input a brief overview of the abstract, and the model will provide a detailed expansion of the content.

Downstream Use [optional]

The model can be further fine-tuned for specific domains within academic writing, such as biomedical research, legal papers, or engineering documentation. It can also be integrated into platforms for automated academic writing assistance or research paper generation.

Out-of-Scope Use

The model is not suitable for general-purpose text generation outside academic or technical contexts. It may not perform well in creative writing tasks or in generating content without structured input like research articles.

Recommendations

Users should ensure that the input is structured in a format appropriate for academic or technical summarization. Since the model generates based on existing research content, it is recommended to verify the accuracy of the output for fact-checking.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

FINETUNED_MODEL = "gyopark/gemma-2-2b-it-WriteMyPaper"

model = AutoModelForCausalLM.from_pretrained(FINETUNED_MODEL, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(FINETUNED_MODEL)

pipe = pipeline(
    "text-generation", 
    model=model, 
    tokenizer=tokenizer, 
)

# doc
<!-- 한ㆍ중ㆍ일 3국의 정치적 태도와 사회경제체제를 ‘세계가치관조사’ 자료를 이용하여 국가 간 비교분석하였다.
중국의 아시아적 가치 지향성이 가장 높고 일본이 가장 낮다. 서구적 민주주의 지향성은 일본이 가장 높고 한국이 그 다음이며 중국이 가장 낮다.
한ㆍ중ㆍ일 3국의 정치적 태도에서의 차이는 사회구조적 변동 경험을 반영하는 것이다. -->

messages = [
    {
        "role": "user",
        "content": "다음 요약된 글을 한 문단으로 길게 써주세요.:\n\n{}".format(doc)
    }
]

prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# prompt
<!-- <bos><start_of_turn>user
\n다음 요약된 글을 한 문단으로 길게 써주세요.:\n
\n한ㆍ중ㆍ일 3국의 정치적 태도와 사회경제체제를 ‘세계가치관조사’ 자료를 이용하여 국가 간 비교분석하였다.
중국의 아시아적 가치 지향성이 가장 높고 일본이 가장 낮다. 서구적 민주주의 지향성은 일본이 가장 높고 한국이 그 다음이며 중국이 가장 낮다.
한ㆍ중ㆍ일 3국의 정치적 태도에서의 차이는 사회구조적 변동 경험을 반영하는 것이다.<end_of_turn>\n
<start_of_turn>model\n -->

outputs = pipe(
    prompt,
    do_sample=True,
    add_special_tokens=True,
    max_new_tokens=1024
)

outputs[0]['generated_text']

Training Details

Training Data

The model was fine-tuned on a dataset of academic papers, including research articles and technical reports, focusing on generating accurate summaries and expanding scientific content.

Training Procedure

Training regime: Mixed precision (fp16)
Hardware: 1 L4 GPU
Training time: Approximately 3 hours
Fine-tuning approach: Low-Rank Adaptation (LoRA)

Summary

The model effectively processes academic papers, providing high-quality summaries and generating longer expansions of sections or abstracts. It supports Korean research documents.

Citation [optional]

Citation

BibTeX:

@misc{gyopark_gemma_2_2b_it_WriteMyPaper, 
  author = {gyopark}, 
  title = {WriteMyPaper}, 
  year = {2024}, 
  url = {https://huggingface.co/gyopark/gemma-2-2b-it-WriteMyPaper}, 
}