Commits · yusufs/vllm-inference

feat(llama3.2): change model to llama3.2

b826155

yusufs commited on Nov 29, 2024

feat(dep_sizes.txt): removes dep_sizes.txt during build, it not needed

8e49b3b

yusufs commited on Nov 27, 2024

feat(download_model.py): remove download_model.py during build, it causing big image size

c360fd3

yusufs commited on Nov 27, 2024

docs(Dockerfile): add comment about estimated image size after compile

8dc2050

yusufs commited on Nov 27, 2024

feat(add-model): always download model during build, it will be cached in the consecutive builds

8679a35

yusufs commited on Nov 27, 2024

feat(reduce-max-num-batched-tokens): Reducing max-num-batched-tokens even the error state it want to reduce max_model_len

13a5c22

yusufs commited on Nov 27, 2024

feat(hf_token): set hf token during build

493a5f1

yusufs commited on Nov 27, 2024

fix(hf_token): export HF_TOKEN during build

c6efe6a

yusufs commited on Nov 27, 2024

feat(download-model): add download model at runtime

fc30f26

yusufs commited on Nov 27, 2024

feat(change-model): change to sail/Sailor-4B-Chat 89a866a7041e6ec023dd462adeca8e28dd53c83e

d90e4d6

yusufs commited on Nov 27, 2024

feat(endpoint): add prefix /api on each endpoint

5f3bf21

yusufs commited on Nov 27, 2024

feat(refactor): move the files to root

7935381

yusufs commited on Nov 27, 2024

fix(run.sh): move files

6479dc6

yusufs commited on Nov 27, 2024

feat(run.sh): add script for running openai server

ded2af7

yusufs commited on Nov 27, 2024

fix(python): fix absolute path of python script

d2e0be1

yusufs commited on Nov 27, 2024

fix(cmd): fix 'error: failed to solve: dockerfile parse error on line 19: unknown instruction: "python3",'

de6b236

yusufs commited on Nov 27, 2024

feat(openai): VLLM OpenAI compatible server

147b3a2

yusufs commited on Nov 27, 2024

feat(parse): parse output

b44271e

yusufs commited on Nov 27, 2024

feat(response): should dict only

b41be20

yusufs commited on Nov 27, 2024

feat(one-model): one model at a time

35decf8

yusufs commited on Nov 27, 2024

fix(remove): use_cached_output is not an option

6b1968a

yusufs commited on Nov 27, 2024

feat(max_model_len): reducing max_model_len for T4 support

c41cdb4

yusufs commited on Nov 27, 2024

fix(half-precision): use half precision for T4

d51e450

yusufs commited on Nov 27, 2024

fix(tensor_parallel_size): set to 1

84c6c4a

yusufs commited on Nov 27, 2024

feat(cuda): add cuda information

2457cd7

yusufs commited on Nov 27, 2024

fix(remove-params): Removing max_model_len

0ef012d

yusufs commited on Nov 27, 2024

feat(sailor-chat): add sail/Sailor-4B-Chat with the same context length

586265c

yusufs commited on Nov 27, 2024

feat(reduce-max-length): reduce maximum length

2425953

yusufs commited on Nov 27, 2024

feat(t4-gpu): add t4 gpu capability

4998ce7

yusufs commited on Nov 27, 2024

fix(expose-port): add EXPOSE in Dockerfile

6d19ece

yusufs commited on Nov 27, 2024

fix(module): fix 'error module app' it should be 'main'

6a914f2

yusufs commited on Nov 27, 2024

feat(first-commit): follow examples and tutorials

ae7cfbb

yusufs commited on Nov 27, 2024

initial commit

1a7087e
verified

yusufs commited on Nov 27, 2024

Spaces:

yusufs
/

vllm-inference

Paused

Commit History

feat(llama3.2): change model to llama3.2

b826155

feat(dep_sizes.txt): removes dep_sizes.txt during build, it not needed

8e49b3b

feat(download_model.py): remove download_model.py during build, it causing big image size

c360fd3

docs(Dockerfile): add comment about estimated image size after compile

8dc2050

feat(add-model): always download model during build, it will be cached in the consecutive builds

8679a35

feat(reduce-max-num-batched-tokens): Reducing max-num-batched-tokens even the error state it want to reduce max_model_len

13a5c22

feat(hf_token): set hf token during build

493a5f1

fix(hf_token): export HF_TOKEN during build

c6efe6a

feat(download-model): add download model at runtime

fc30f26

feat(change-model): change to sail/Sailor-4B-Chat 89a866a7041e6ec023dd462adeca8e28dd53c83e

d90e4d6

feat(endpoint): add prefix /api on each endpoint

5f3bf21

feat(refactor): move the files to root

7935381

fix(run.sh): move files

6479dc6

feat(run.sh): add script for running openai server

ded2af7

fix(python): fix absolute path of python script

d2e0be1

fix(cmd): fix 'error: failed to solve: dockerfile parse error on line 19: unknown instruction: "python3",'

de6b236

feat(openai): VLLM OpenAI compatible server

147b3a2

feat(parse): parse output

b44271e

feat(response): should dict only

b41be20

feat(one-model): one model at a time

35decf8

fix(remove): use_cached_output is not an option

6b1968a

feat(max_model_len): reducing max_model_len for T4 support

c41cdb4

fix(half-precision): use half precision for T4

d51e450

fix(tensor_parallel_size): set to 1

84c6c4a

feat(cuda): add cuda information

2457cd7

fix(remove-params): Removing max_model_len

0ef012d

feat(sailor-chat): add sail/Sailor-4B-Chat with the same context length

586265c

feat(reduce-max-length): reduce maximum length

2425953

feat(t4-gpu): add t4 gpu capability

4998ce7

fix(expose-port): add EXPOSE in Dockerfile

6d19ece

fix(module): fix 'error module app' it should be 'main'

6a914f2

feat(first-commit): follow examples and tutorials

ae7cfbb

initial commit

1a7087e
verified

Commit History

feat(llama3.2): change model to llama3.2 b826155

feat(dep_sizes.txt): removes dep_sizes.txt during build, it not needed 8e49b3b

feat(download_model.py): remove download_model.py during build, it causing big image size c360fd3

docs(Dockerfile): add comment about estimated image size after compile 8dc2050

feat(add-model): always download model during build, it will be cached in the consecutive builds 8679a35

feat(reduce-max-num-batched-tokens): Reducing max-num-batched-tokens even the error state it want to reduce max_model_len 13a5c22

feat(hf_token): set hf token during build 493a5f1

fix(hf_token): export HF_TOKEN during build c6efe6a

feat(download-model): add download model at runtime fc30f26

feat(change-model): change to sail/Sailor-4B-Chat 89a866a7041e6ec023dd462adeca8e28dd53c83e d90e4d6

feat(endpoint): add prefix /api on each endpoint 5f3bf21

feat(refactor): move the files to root 7935381

fix(run.sh): move files 6479dc6

feat(run.sh): add script for running openai server ded2af7

fix(python): fix absolute path of python script d2e0be1

fix(cmd): fix 'error: failed to solve: dockerfile parse error on line 19: unknown instruction: "python3",' de6b236

feat(openai): VLLM OpenAI compatible server 147b3a2

feat(parse): parse output b44271e

feat(response): should dict only b41be20

feat(one-model): one model at a time 35decf8

fix(remove): use_cached_output is not an option 6b1968a

feat(max_model_len): reducing max_model_len for T4 support c41cdb4

fix(half-precision): use half precision for T4 d51e450

fix(tensor_parallel_size): set to 1 84c6c4a

feat(cuda): add cuda information 2457cd7

fix(remove-params): Removing max_model_len 0ef012d

feat(sailor-chat): add sail/Sailor-4B-Chat with the same context length 586265c

feat(reduce-max-length): reduce maximum length 2425953

feat(t4-gpu): add t4 gpu capability 4998ce7

fix(expose-port): add EXPOSE in Dockerfile 6d19ece

fix(module): fix 'error module app' it should be 'main' 6a914f2

feat(first-commit): follow examples and tutorials ae7cfbb

initial commit 1a7087e verified

feat(llama3.2): change model to llama3.2

b826155

feat(dep_sizes.txt): removes dep_sizes.txt during build, it not needed

8e49b3b

feat(download_model.py): remove download_model.py during build, it causing big image size

c360fd3

docs(Dockerfile): add comment about estimated image size after compile

8dc2050

feat(add-model): always download model during build, it will be cached in the consecutive builds

8679a35

feat(reduce-max-num-batched-tokens): Reducing max-num-batched-tokens even the error state it want to reduce max_model_len

13a5c22

feat(hf_token): set hf token during build

493a5f1

fix(hf_token): export HF_TOKEN during build

c6efe6a

feat(download-model): add download model at runtime

fc30f26

feat(change-model): change to sail/Sailor-4B-Chat 89a866a7041e6ec023dd462adeca8e28dd53c83e

d90e4d6

feat(endpoint): add prefix /api on each endpoint

5f3bf21

feat(refactor): move the files to root

7935381

fix(run.sh): move files

6479dc6

feat(run.sh): add script for running openai server

ded2af7

fix(python): fix absolute path of python script

d2e0be1

fix(cmd): fix 'error: failed to solve: dockerfile parse error on line 19: unknown instruction: "python3",'

de6b236

feat(openai): VLLM OpenAI compatible server

147b3a2

feat(parse): parse output

b44271e

feat(response): should dict only

b41be20

feat(one-model): one model at a time

35decf8

fix(remove): use_cached_output is not an option

6b1968a

feat(max_model_len): reducing max_model_len for T4 support

c41cdb4

fix(half-precision): use half precision for T4

d51e450

fix(tensor_parallel_size): set to 1

84c6c4a

feat(cuda): add cuda information

2457cd7

fix(remove-params): Removing max_model_len

0ef012d

feat(sailor-chat): add sail/Sailor-4B-Chat with the same context length

586265c

feat(reduce-max-length): reduce maximum length

2425953

feat(t4-gpu): add t4 gpu capability

4998ce7

fix(expose-port): add EXPOSE in Dockerfile

6d19ece

fix(module): fix 'error module app' it should be 'main'

6a914f2

feat(first-commit): follow examples and tutorials

ae7cfbb

initial commit

1a7087e
verified