Commits · Dovakiins/qwerrwe

misc fixes to add gptq tests (#621)

03e5907
unverified

winglian commited on Sep 22, 2023

Support Sample packing for phi arch (#586)

12a2dbb
unverified

winglian commited on Sep 15, 2023

E2e device cuda (#575)

2414673
unverified

winglian commited on Sep 15, 2023

e2e testing (#574)

9218ebe
unverified

winglian commited on Sep 15, 2023

Fix pretraining with iterable/streaming Dataset (#556)

2f586d1
unverified

Jan Philipp Harries Jan Philipp Harries commited on Sep 13, 2023

workaround for md5 variations (#533)

0b4cf5b
unverified

winglian commited on Sep 8, 2023

recommend padding when using sample packing (#531)

3437149
unverified

winglian commited on Sep 6, 2023

fix test fixture b/c hf trainer tokenization changed (#464)

d5dcf9c
unverified

winglian commited on Aug 23, 2023

fix fixture for new tokenizer handling in transformers (#428)

8cace80
unverified

winglian commited on Aug 17, 2023

simplify `load_tokenizer`

efb3b2c

tmm1 commited on Aug 13, 2023

extract module for working with cfg

8cec513

tmm1 commited on Aug 13, 2023

fix DefaultDict.or

a13e45d

tmm1 commited on Aug 10, 2023

Attention mask and position id fixes for packing (#285)

2bb0b78
unverified

winglian commited on Aug 12, 2023

experimental llama 2 chat support (#296)

3392270
unverified

Jan Philipp Harries Jan Philipp Harries commited on Aug 6, 2023

update prompts for open orca to match the paper (#317)

3d4984b
unverified

winglian commited on Jul 22, 2023

Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var

b1f4f7a

theobjectivedad commited on Jul 15, 2023

Adding logging enhancement

553a86b

theobjectivedad commited on Jul 14, 2023

params are adam_, not adamw_

19cf0bd

winglian commited on Jul 8, 2023

add tests and supoort for loader for sys prompt data

3a38271

winglian commited on Jun 18, 2023

initial wip to get sys prompt from dataset

8d20e0a

winglian commited on Jun 17, 2023

optionally define whether to use_fast tokenizer

47d601f

winglian commited on Jun 25, 2023

Additional test case per pr

ad5ca4f

winglian commited on Jun 15, 2023

add validation and tests for adamw hyperparam

cb9d3af

winglian commited on Jun 15, 2023

Merge pull request #214 from OpenAccess-AI-Collective/fix-tokenizing-labels

1925eaf
unverified

winglian commited on Jun 15, 2023

fix test name

1ab3bf3

winglian commited on Jun 15, 2023

ingore duplicate code in tests

baed440

winglian commited on Jun 15, 2023

bugfix for potential off by one

7925ddc

winglian commited on Jun 15, 2023

Merge branch 'main' into flash-optimum

fd2c981
unverified

winglian commited on Jun 12, 2023

new validation for mpt w grad checkpoints

14668fa

winglian commited on Jun 11, 2023

add streaming dataset support for pretraining datasets

eea2731

winglian commited on Jun 10, 2023

Validate falcon with fsdp

babf0fd

Nanobit commited on Jun 8, 2023

Update doc for grad_accu and add validation tests for batch size

3c71c8d

Nanobit commited on May 31, 2023

don't worry about duplicate code here

0136f51

winglian commited on May 31, 2023

fix packing so that concatenated sequences reset the attention

9b8585d

winglian commited on May 31, 2023

black formatting

6fa40bf

winglian commited on May 31, 2023

add support for gradient accumulation steps

3aad5f3

winglian commited on May 31, 2023

Fix pre-commit for rebased files

b81c97f

Nanobit commited on May 30, 2023