kevinum commited on
Commit
c067fc4
·
1 Parent(s): 06eae91

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -11
README.md CHANGED
@@ -2,33 +2,34 @@
2
  license: apache-2.0
3
  tags:
4
  - generated_from_trainer
 
 
5
  model-index:
6
  - name: byt5-small-finetuned-English-to-BASH
7
  results: []
8
  ---
9
 
10
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
- should probably proofread and complete it, then remove this comment. -->
12
-
13
  # byt5-small-finetuned-English-to-BASH
 
14
 
15
- This model is a fine-tuned version of [google/byt5-small](https://huggingface.co/google/byt5-small) on the None dataset.
 
16
  It achieves the following results on the evaluation set:
17
  - Loss: 0.4850
18
  - Nl2bash M: 0.6376
19
  - Gen Len: 16.9946
20
 
21
- ## Model description
22
-
23
- More information needed
24
-
25
  ## Intended uses & limitations
26
-
27
- More information needed
28
 
29
  ## Training and evaluation data
30
 
31
- More information needed
 
 
 
 
 
32
 
33
  ## Training procedure
34
 
 
2
  license: apache-2.0
3
  tags:
4
  - generated_from_trainer
5
+ metrics:
6
+ - nl2bash_m
7
  model-index:
8
  - name: byt5-small-finetuned-English-to-BASH
9
  results: []
10
  ---
11
 
 
 
 
12
  # byt5-small-finetuned-English-to-BASH
13
+ Created by: [Josh Shih](https://huggingface.co/Josh98), [Alex Sha](https://huggingface.co/alexsha), [Kevin Um](https://huggingface.co/kevinum) for EEP 596 - Natural Language Processing at University of Washington (Seattle).
14
 
15
+ ## Model description
16
+ This model is a fine-tuned version of [google/byt5-small](https://huggingface.co/google/byt5-small) on a more balanced iteration of the [NL2BASH](https://github.com/TellinaTool/nl2bash/tree/master/data) dataset.
17
  It achieves the following results on the evaluation set:
18
  - Loss: 0.4850
19
  - Nl2bash M: 0.6376
20
  - Gen Len: 16.9946
21
 
 
 
 
 
22
  ## Intended uses & limitations
23
+ Purpose: To generate bash commands from text input, and help people learn to use linux bash. This is a proof of concept model using transfer learning to fine-tune an existing language model and produce structured code instead of natural language.
 
24
 
25
  ## Training and evaluation data
26
 
27
+ This model was trained and evaluated using a custom iteration of [NL2BASH](https://github.com/TellinaTool/nl2bash/tree/master/data). The original NL2BASH dataset contains a large class imbalance with too many bash commands which begin with 'find'.
28
+
29
+ A maximum threshold was set to remove text/BASH pairs which exceeded the threshold, and [GPT-3](https://openai.com/blog/gpt-3-apps/) API was used to generate text/BASH pairs for those below the threshold.
30
+
31
+ ~5500 original text/BASH pairs and ~5700 generated text/BASH pairs were used, giving a total of ~11200 lines of text/BASH pairs. Shown below is the class distribution for the top-5 commands.
32
+ ![class_balanced.png](https://s3.amazonaws.com/moonup/production/uploads/1677215336540-63d8b9876ac3104e50cd9634.png)
33
 
34
  ## Training procedure
35