Hugging Face's logo

kenhktsui
/
setfit_test_imdb

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results
Model card Files Community
kenhktsui commited on
Commit
38c0bf6
•
1 Parent(s): 55498d5

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +44 -54
  2. config.json +1 -1
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -10,38 +10,28 @@ tags:
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
- - text: I walked into this movie expecting a thrilling adventure, but what I got was
14
- a jumbled mess of characters that made no sense. The main character's personality
15
- changed from one scene to the next, it was like they had a different actor playing
16
- them. The plot was predictable and the dialogue was cheesy. I left the theater
17
- feeling disappointed and frustrated with the wasted potential. The only thing
18
- that kept me engaged was the cinematography, but it wasn't enough to save this
19
- trainwreck. Overall, I would not recommend this movie to anyone.
20
- - text: I recently visited this quaint little cafe and it was love at first sight.
21
- The moment I stepped inside, I felt like I was wrapped in a warm hug. The decor
22
- is so cozy and inviting, with plush armchairs and soft lighting that makes you
23
- feel like you're in a different world. The staff is friendly and attentive, and
24
- the coffee is top-notch. I had the most delicious cappuccino and a slice of lemon
25
- pound cake that was to die for. I'll definitely be back for more of this cozy
26
- ambiance.
27
- - text: I just watched the latest Marvel movie and I have to say, it was absolutely
28
- phenomenal. The cinematography was breathtaking, the music was catchy and perfectly
29
- complemented the on-screen action. The cast delivered outstanding performances,
30
- and the storyline was engaging and well-paced. I thoroughly enjoyed every minute
31
- of it and would highly recommend it to anyone who loves superhero movies. 10/10
32
- would watch again.
33
- - text: The movie had so much potential, but the ending was a complete letdown. It
34
- felt rushed and didn't provide any closure for the characters. I was expecting
35
- a more satisfying conclusion, but what we got was a lazy attempt to tie everything
36
- together. The final act was a mess, and it left me feeling frustrated and disappointed.
37
- Overall, the movie was enjoyable, but the ending ruined it for me. I would have
38
- given it a higher rating if the writers had put more effort into crafting a better
39
- ending.
40
- - text: The acting in this movie was laughable. The lead actor's delivery was wooden
41
- and unconvincing. He had all the charisma of a cardboard box. The supporting cast
42
- fared no better, with most of them struggling to deliver even the simplest of
43
- lines. The director's attempt to create tension was wasted on the subpar acting,
44
- making the entire experience feel like a chore to sit through.
45
  inference: true
46
  model-index:
47
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
@@ -55,7 +45,7 @@ model-index:
55
  split: test
56
  metrics:
57
  - type: accuracy
58
- value: 0.88784
59
  name: Accuracy
60
  ---
61
 
@@ -87,17 +77,17 @@ The model has been trained using an efficient few-shot learning technique that i
87
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
88
 
89
  ### Model Labels
90
- | Label | Examples |
91
- |:-------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
92
- | negative sentiment | <ul><li>'I was really looking forward to this movie, but unfortunately, it was a complete disappointment. The plot was predictable and lacked any real tension. The characters were underdeveloped and their motivations were unclear. The pacing was slow and dragged on for far too long. Overall, I would not recommend this movie to anyone.'</li><li>"I'm extremely disappointed with the service I received at this restaurant. The hostess was unfriendly and unhelpful, and our server seemed completely overwhelmed. We had to ask multiple times for basic things like water and utensils. The food was overpriced and not even that good. Definitely will not be returning."</li><li>"I'm extremely disappointed with my recent purchase from this store. The quality of the product is subpar and the price is way too high. I paid $200 for a cheap-looking item that broke after just a week of use. Not worth the money at all. 1/10 would not recommend."</li></ul> |
93
- | positive sentiment | <ul><li>"I just got tickets to see my favorite artist in concert and I'm beyond thrilled! The energy in the crowd is going to be electric! #concertseason #musiclover"</li><li>"I just had the most amazing experience at this restaurant! The service was lightning fast, and the food was prepared to perfection. Our server, Alex, was attentive and friendly, making sure we had everything we needed. The bill was reasonable, and we left feeling satisfied and eager to come back. 5 stars isn't enough, I'd give it 10 if I could!"</li><li>'The action scenes in this movie are absolutely mind-blowing! The stunts are incredibly well-choreographed and the special effects are top-notch. I was on the edge of my seat the entire time, cheering on the heroes as they fought to save the world. The cast is also excellent, with standout performances from the lead actors. Overall, I would highly recommend this movie to anyone who loves action-packed thrill rides.'</li></ul> |
94
 
95
  ## Evaluation
96
 
97
  ### Metrics
98
  | Label | Accuracy |
99
  |:--------|:---------|
100
- | **all** | 0.8878 |
101
 
102
  ## Uses
103
 
@@ -117,7 +107,7 @@ from setfit import SetFitModel
117
  # Download from the 🤗 Hub
118
  model = SetFitModel.from_pretrained("setfit_model_id")
119
  # Run inference
120
- preds = model("The acting in this movie was laughable. The lead actor's delivery was wooden and unconvincing. He had all the charisma of a cardboard box. The supporting cast fared no better, with most of them struggling to deliver even the simplest of lines. The director's attempt to create tension was wasted on the subpar acting, making the entire experience feel like a chore to sit through.")
121
  ```
122
 
123
  <!--
@@ -147,14 +137,14 @@ preds = model("The acting in this movie was laughable. The lead actor's delivery
147
  ## Training Details
148
 
149
  ### Training Set Metrics
150
- | Training set | Min | Median | Max |
151
- |:-------------|:----|:--------|:----|
152
- | Word count | 23 | 56.8571 | 79 |
153
 
154
  | Label | Training Sample Count |
155
  |:-------------------|:----------------------|
156
- | negative sentiment | 14 |
157
- | positive sentiment | 14 |
158
 
159
  ### Training Hyperparameters
160
  - batch_size: (16, 16)
@@ -174,16 +164,16 @@ preds = model("The acting in this movie was laughable. The lead actor's delivery
174
  - load_best_model_at_end: True
175
 
176
  ### Training Results
177
- | Epoch | Step | Training Loss | Validation Loss |
178
- |:-------:|:-------:|:-------------:|:---------------:|
179
- | 0.0370 | 1 | 0.2498 | - |
180
- | 1.0 | 27 | - | 0.0017 |
181
- | 1.8519 | 50 | 0.0003 | - |
182
- | 2.0 | 54 | - | 0.0004 |
183
- | 3.0 | 81 | - | 0.0003 |
184
- | 3.7037 | 100 | 0.0002 | - |
185
- | 4.0 | 108 | - | 0.0004 |
186
- | **5.0** | **135** | **-** | **0.0003** |
187
 
188
  * The bold row denotes the saved checkpoint.
189
  ### Framework Versions
 
10
  - text-classification
11
  - generated_from_setfit_trainer
12
  widget:
13
+ - text: I just watched 'The Shawshank Redemption' and I have to say, Tim Robbins and
14
+ Morgan Freeman delivered outstanding performances. Their acting skills truly brought
15
+ the characters to life. The way they portrayed the emotional depth of their characters
16
+ was impressive. I highly recommend this movie to anyone who loves a good drama.
17
+ - text: I walked into this movie expecting a lot, but what I got was a complete waste
18
+ of time. The acting was subpar, the plot was predictable, and the dialogue was
19
+ cringeworthy. I've seen high school productions that were better. The only thing
20
+ that kept me awake was the hope that something, anything, would happen to make
21
+ this movie worth watching. Unfortunately, that never came. I would not recommend
22
+ this to my worst enemy. 1/10, would not watch again even if you paid me.
23
+ - text: I just watched this movie and I'm still grinning from ear to ear. The humor
24
+ is wickedly clever and the cast is perfectly assembled. It's a laugh-out-loud
25
+ masterpiece that will leave you feeling uplifted and entertained.
26
+ - text: I was really looking forward to trying out this new restaurant, but unfortunately,
27
+ it was a huge disappointment. The service was slow, the food was cold, and the
28
+ ambiance was non-existent. I ordered the burger, but it was overcooked and tasted
29
+ like it had been sitting out for hours. Needless to say, I won't be back.
30
+ - text: I recently visited this restaurant for lunch and had an amazing experience.
31
+ The service was top-notch, our server was friendly and attentive, and the food
32
+ was incredible. I ordered the grilled chicken salad and it was cooked to perfection.
33
+ The portion size was generous and the prices were very reasonable. I would highly
34
+ recommend this place to anyone looking for a great meal.
 
 
 
 
 
 
 
 
 
 
35
  inference: true
36
  model-index:
37
  - name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
 
45
  split: test
46
  metrics:
47
  - type: accuracy
48
+ value: 0.87812
49
  name: Accuracy
50
  ---
51
 
 
77
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
78
 
79
  ### Model Labels
80
+ | Label | Examples |
81
+ |:-------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
82
+ | positive sentiment | <ul><li>"I just watched the latest Marvel movie and I'm still reeling from the shocking plot twist at the end. I didn't see it coming and it completely flipped my expectations on their head. The way the story unfolded was pure genius and had me on the edge of my seat the entire time. I'm not even kidding when I say that this movie is a must-see for anyone who loves a good surprise. 10/10 would recommend."</li><li>'I recently visited this restaurant and was blown away by the exceptional service from the staff. Our server, Alex, was attentive, knowledgeable, and made sure we had everything we needed throughout our meal. The food was delicious, but the service was truly what made our experience stand out. I would highly recommend this place to anyone looking for a great dining experience.'</li><li>"I just watched the funniest movie of my life, 'Dumb and Dumber'! Jim Carrey's comedic timing is unmatched. He has this incredible ability to make you laugh without even trying. The movie is full of hilarious moments, and I found myself giggling uncontrollably throughout. I highly recommend it to anyone looking for a good laugh."</li></ul> |
83
+ | negative sentiment | <ul><li>"I'm extremely disappointed with my recent purchase from this restaurant. The food was overcooked and the service was slow. The prices are way too high for the quality of food you receive. I won't be returning anytime soon."</li><li>"I'm extremely disappointed with the service I received at this restaurant. The hostess was completely unfriendly and unhelpful. We were seated for 20 minutes before anyone even came to take our order. The food was overpriced and took an hour to arrive. The server seemed put off by our presence and didn't even bother to refill our drinks. Needless to say, we will never be back."</li><li>'I was really looking forward to this movie, but unfortunately, it fell flat. The plot was predictable and lacked any real tension or suspense. The characters were underdeveloped and their motivations were unclear. The pacing was slow and the ending was completely unsatisfying. Overall, I was disappointed by the lack of effort put into creating a compelling story. 1/10 would not recommend.'</li></ul> |
84
 
85
  ## Evaluation
86
 
87
  ### Metrics
88
  | Label | Accuracy |
89
  |:--------|:---------|
90
+ | **all** | 0.8781 |
91
 
92
  ## Uses
93
 
 
107
  # Download from the 🤗 Hub
108
  model = SetFitModel.from_pretrained("setfit_model_id")
109
  # Run inference
110
+ preds = model("I just watched this movie and I'm still grinning from ear to ear. The humor is wickedly clever and the cast is perfectly assembled. It's a laugh-out-loud masterpiece that will leave you feeling uplifted and entertained.")
111
  ```
112
 
113
  <!--
 
137
  ## Training Details
138
 
139
  ### Training Set Metrics
140
+ | Training set | Min | Median | Max |
141
+ |:-------------|:----|:-------|:----|
142
+ | Word count | 20 | 50.76 | 80 |
143
 
144
  | Label | Training Sample Count |
145
  |:-------------------|:----------------------|
146
+ | negative sentiment | 13 |
147
+ | positive sentiment | 12 |
148
 
149
  ### Training Hyperparameters
150
  - batch_size: (16, 16)
 
164
  - load_best_model_at_end: True
165
 
166
  ### Training Results
167
+ | Epoch | Step | Training Loss | Validation Loss |
168
+ |:-------:|:------:|:-------------:|:---------------:|
169
+ | 0.0455 | 1 | 0.1789 | - |
170
+ | 1.0 | 22 | - | 0.013 |
171
+ | 2.0 | 44 | - | 0.0024 |
172
+ | 2.2727 | 50 | 0.0003 | - |
173
+ | 3.0 | 66 | - | 0.0014 |
174
+ | **4.0** | **88** | **-** | **0.0011** |
175
+ | 4.5455 | 100 | 0.0003 | - |
176
+ | 5.0 | 110 | - | 0.0013 |
177
 
178
  * The bold row denotes the saved checkpoint.
179
  ### Framework Versions
config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "_name_or_path": "setfit/step_135",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
 
1
  {
2
+ "_name_or_path": "setfit/step_88",
3
  "architectures": [
4
  "MPNetModel"
5
  ],
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:576c6f26fd3c8740346d1f93b2554525a58f9ffcacf607db5f230fd9af5ee25e
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0099479a89c624c59178f32c9780ef8bff498896d8c64914bbf50f578cbf71c
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f6027d5e4448e72924441a8c7b794d205defe906dee648bb48cdf0f9cb10c36d
3
  size 7007
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36d88f66d85ae68ad10f458dbf0911047f207f24d74e7cc81f7022eac083436d
3
  size 7007