roberta-large-openai-detector |  0.3311 |  0.3607 |  0.3545 |  0.3408 |  0.9457 |  0.3460 |  0.3761 |  0.3395 |  0.3355 |  0.3529 |  0.3325
xlm-roberta-large |  0.4532 |  0.4074 |  0.8414 |  0.7198 |  0.9384 |  0.6397 |  0.4080 |  0.5535 |  0.6371 |  0.6560 |  0.4488
opt-66b | bert-base-multilingual-cased |  0.4121 |  0.5241 |  0.6850 |  0.6789 |  0.9079 |  0.5558 |  0.5274 |  0.6377 |  0.6104 |  0.6477 |  0.5323
electra-large-discriminator |  0.3969 |  0.6936 |  0.6463 |  0.6972 |  0.9314 |  0.6785 |  0.6637 |  0.7006 |  0.3390 |  0.3303 |  0.3311
gpt2-medium |  0.4448 |  0.4992 |  0.4264 |  0.5941 |  0.8898 |  0.5432 |  0.4274 |  0.6423 |  0.4870 |  0.5507 |  0.4764
mGPT |  0.1983 |  0.5890 |  0.5708 |  0.6564 |  0.9185 |  0.5605 |  0.5826 |  0.6449 |  0.4463 |  0.4981 |  0.4464
mdeberta-v3-base |  0.2612 |  0.4316 |  0.5316 |  0.6257 |  0.7720 |  0.4713 |  0.4184 |  0.3897 |  0.5503 |  0.4716 |  0.5307
roberta-large-openai-detector |  0.3322 |  0.6773 |  0.4695 |  0.4529 |  0.9332 |  0.4155 |  0.6678 |  0.4105 |  0.3346 |  0.3886 |  0.3376
xlm-roberta-large |  0.4182 |  0.4089 |  0.6021 |  0.6750 |  0.9205 |  0.5871 |  0.4205 |  0.5935 |  0.5884 |  0.6220 |  0.5175
opt-iml-max-1.3b | bert-base-multilingual-cased |  0.5705 |  0.6372 |  0.7086 |  0.7003 |  0.9549 |  0.6327 |  0.6705 |  0.6447 |  0.5501 |  0.6380 |  0.5188
electra-large-discriminator |  0.3280 |  0.6391 |  0.6211 |  0.7352 |  0.9964 |  0.8875 |  0.8502 |  0.8454 |  0.3280 |  0.3275 |  0.3292
gpt2-medium |  0.3740 |  0.8301 |  0.5994 |  0.6433 |  0.9801 |  0.7471 |  0.6005 |  0.6545 |  0.3770 |  0.3929 |  0.4977
mGPT |  0.2607 |  0.5663 |  0.5731 |  0.6369 |  0.9729 |  0.5881 |  0.5901 |  0.4952 |  0.2563 |  0.2537 |  0.4156
mdeberta-v3-base |  0.7309 |  0.8618 |  0.9212 |  0.8152 |  0.9729 |  0.7726 |  0.8558 |  0.7391 |  0.7542 |  0.8477 |  0.7328
roberta-large-openai-detector |  0.3280 |  0.8886 |  0.7955 |  0.6782 |  0.9856 |  0.7670 |  0.9075 |  0.6858 |  0.3429 |  0.4533 |  0.3737
xlm-roberta-large |  0.6694 |  0.7393 |  0.7423 |  0.8130 |  0.9711 |  0.7553 |  0.6195 |  0.6933 |  0.5689 |  0.5977 |  0.6165
text-davinci-003 | bert-base-multilingual-cased |  0.7853 |  0.9064 |  0.8815 |  0.8951 |  0.9693 |  0.7987 |  0.9115 |  0.8632 |  0.7773 |  0.7333 |  0.8883
electra-large-discriminator |  0.3985 |  0.3333 |  0.3333 |  0.3303 |  0.9910 |  0.3309 |  0.3330 |  0.3284 |  0.3741 |  0.4089 |  0.3333
gpt2-medium |  0.3330 |  0.3333 |  0.3333 |  0.3303 |  0.9892 |  0.3264 |  0.3330 |  0.3276 |  0.3283 |  0.3334 |  0.3326
mGPT |  0.6136 |  0.8932 |  0.3333 |  0.8701 |  0.9892 |  0.8484 |  0.9382 |  0.9079 |  0.6940 |  0.5687 |  0.5904
mdeberta-v3-base |  0.5791 |  0.7680 |  0.8940 |  0.8895 |  0.9074 |  0.6401 |  0.8748 |  0.7725 |  0.7341 |  0.6130 |  0.7199
roberta-large-openai-detector |  0.5757 |  0.7601 |  0.8132 |  0.8724 |  0.9946 |  0.7930 |  0.7339 |  0.7721 |  0.6125 |  0.3726 |  0.6949
xlm-roberta-large |  0.3899 |  0.9042 |  0.5268 |  0.9016 |  0.9856 |  0.8327 |  0.9482 |  0.8872 |  0.5981 |  0.3932 |  0.3987
vicuna-13b | bert-base-multilingual-cased |  0.7679 |  0.8821 |  0.8366 |  0.8410 |  0.9874 |  0.7861 |  0.9382 |  0.8218 |  0.8335 |  0.8378 |  0.6379
electra-large-discriminator |  0.4056 |  0.5004 |  0.5494 |  0.5233 |  0.9838 |  0.7749 |  0.6136 |  0.6435 |  0.4744 |  0.4347 |  0.5352
gpt2-medium |  0.5084 |  0.3728 |  0.4513 |  0.4422 |  0.9856 |  0.5856 |  0.3776 |  0.4527 |  0.4134 |  0.4440 |  0.3841
mGPT |  0.3881 |  0.9079 |  0.4889 |  0.8254 |  0.9910 |  0.6963 |  0.9161 |  0.8043 |  0.6361 |  0.7215 |  0.6778
mdeberta-v3-base |  0.2578 |  0.7861 |  0.6983 |  0.8225 |  0.7928 |  0.6142 |  0.8809 |  0.7064 |  0.6031 |  0.6765 |  0.5488
roberta-large-openai-detector |  0.3330 |  0.4906 |  0.4113 |  0.4794 |  0.9511 |  0.4425 |  0.6971 |  0.3821 |  0.3326 |  0.3341 |  0.3488
xlm-roberta-large |  0.4062 |  0.9009 |  0.7648 |  0.9324 |  0.9892 |  0.6761 |  0.8613 |  0.7444 |  0.7618 |  0.7866 |  0.7595
All Detectors Mean |  0.4931 |  0.6568 |  0.6270 |  0.6871 |  0.9529 |  0.6476 |  0.6800 |  0.6497 |  0.5759 |  0.5716 |  0.5299
Multilingual Base Models Mean |  0.5605 |  0.7484 |  0.7289 |  0.8244 |  0.9392 |  0.7156 |  0.7778 |  0.7508 |  0.7092 |  0.7118 |  0.6160
Monolingual Base Models Mean |  0.4033 |  0.5346 |  0.4912 |  0.5040 |  0.9712 |  0.5570 |  0.5497 |  0.5150 |  0.3981 |  0.3847 |  0.4152

Table 15: Cross-lingual performance of all detection models fine-tuned on the
English language with 3-times more train samples (evaluated per LLM).

| Detector | Test LLM
---|---|---
Train LLM | Base Model | text-davinci-003 | gpt-3.5-turbo | gpt-4 | alpaca-lora-30b | vicuna-13b | llama-65b | opt-66b | opt-iml-max-1.3b
alpaca-lora-30b | bert-base-multilingual-cased | 0.8102 | 0.8237 | 0.7964 | 0.7889 | 0.7622 | 0.4336 | 0.4315 | 0.4863
electra-large-discriminator | 0.5356 | 0.5243 | 0.5001 | 0.5698 | 0.5460 | 0.4060 | 0.4595 | 0.5168
gpt2-medium | 0.5267 | 0.4589 | 0.4438 | 0.5426 | 0.4768 | 0.5626 | 0.4401 | 0.4939
mGPT | 0.7656 | 0.8022 | 0.7437 | 0.7556 | 0.7396 | 0.4427 | 0.4386 | 0.3624
mdeberta-v3-base | 0.7795 | 0.8118 | 0.8242 | 0.7428 | 0.7404 | 0.3676 | 0.3285 | 0.3175
roberta-large-openai-detector | 0.5663 | 0.5660 | 0.5646 | 0.5627 | 0.5556 | 0.4595 | 0.5272 | 0.5061
xlm-roberta-large | 0.7940 | 0.8264 | 0.8029 | 0.7643 | 0.7542 | 0.4912 | 0.4660 | 0.5107
gpt-3.5-turbo | bert-base-multilingual-cased | 0.8512 | 0.9106 | 0.8841 | 0.7797 | 0.8281 | 0.3758 | 0.4023 | 0.3959
electra-large-discriminator | 0.5937 | 0.6052 | 0.5879 | 0.6016 | 0.6087 | 0.4381 | 0.4511 | 0.4864
gpt2-medium | 0.4147 | 0.4302 | 0.4382 | 0.4215 | 0.4532 | 0.4995 | 0.3672 | 0.4162
mGPT | 0.7572 | 0.8286 | 0.7858 | 0.6841 | 0.7503 | 0.3726 | 0.3879 | 0.3457
mdeberta-v3-base | 0.6966 | 0.7136 | 0.7313 | 0.6586 | 0.6704 | 0.3248 | 0.2977 | 0.2729
roberta-large-openai-detector | 0.6167 | 0.6141 | 0.6105 | 0.6013 | 0.5940 | 0.4211 | 0.5461 | 0.4714
xlm-roberta-large | 0.7198 | 0.8308 | 0.7862 | 0.6829 | 0.7215 | 0.3967 | 0.4055 | 0.3881
gpt-4 | bert-base-multilingual-cased | 0.7090 | 0.7954 | 0.8458 | 0.6428 | 0.6632 | 0.3744 | 0.3749 | 0.3585
electra-large-discriminator | 0.4972 | 0.5225 | 0.5362 | 0.4759 | 0.5156 | 0.3720 | 0.3757 | 0.3755
gpt2-medium | 0.4037 | 0.4334 | 0.4460 | 0.4199 | 0.4387 | 0.5207 | 0.3628 | 0.4035
mGPT | 0.7058 | 0.7616 | 0.7657 | 0.6387 | 0.6981 | 0.4186 | 0.4255 | 0.3558
mdeberta-v3-base | 0.6201 | 0.8077 | 0.8469 | 0.5331 | 0.6150 | 0.3346 | 0.3385 | 0.3357
roberta-large-openai-detector | 0.6070 | 0.6131 | 0.6136 | 0.5804 | 0.5889 | 0.3577 | 0.5085 | 0.3777
xlm-roberta-large | 0.6526 | 0.7151 | 0.7742 | 0.5707 | 0.6754 | 0.3815 | 0.3821 | 0.3743
llama-65b | bert-base-multilingual-cased | 0.2969 | 0.3174 | 0.4327 | 0.3828 | 0.4522 | 0.6841 | 0.6015 | 0.6403
electra-large-discriminator | 0.5096 | 0.4843 | 0.4806 | 0.4937 | 0.4921 | 0.5111 | 0.5108 | 0.5095
gpt2-medium | 0.4531 | 0.4221 | 0.4092 | 0.4193 | 0.4644 | 0.5337 | 0.4957 | 0.4954
mGPT | 0.4849 | 0.4824 | 0.5086 | 0.4783 | 0.5308 | 0.5608 | 0.5337 | 0.5410
mdeberta-v3-base | 0.5439 | 0.5356 | 0.5626 | 0.5473 | 0.6054 | 0.6348 | 0.6075 | 0.6318
roberta-large-openai-detector | 0.4157 | 0.4153 | 0.4157 | 0.4131 | 0.4133 | 0.4124 | 0.4099 | 0.4060
xlm-roberta-large | 0.4817 | 0.4691 | 0.5183 | 0.5133 | 0.5757 | 0.6625 | 0.6145 | 0.6553
opt-66b | bert-base-multilingual-cased | 0.4144 | 0.4443 | 0.5264 | 0.4583 | 0.5037 | 0.6291 | 0.5623 | 0.5766
electra-large-discriminator | 0.4734 | 0.4710 | 0.5217 | 0.5028 | 0.5515 | 0.6779 | 0.5974 | 0.6285
gpt2-medium | 0.5210 | 0.4556 | 0.4574 | 0.4769 | 0.4977 | 0.6650 | 0.5379 | 0.5676
mGPT | 0.5835 | 0.6185 | 0.6248 | 0.4870 | 0.6169 | 0.6016 | 0.5344 | 0.4388
mdeberta-v3-base | 0.4586 | 0.5203 | 0.5924 | 0.4266 | 0.6025 | 0.6556 | 0.5969 | 0.5744
roberta-large-openai-detector | 0.4785 | 0.4776 | 0.4787 | 0.4731 | 0.4726 | 0.4619 | 0.4708 | 0.4719
xlm-roberta-large | 0.5180 | 0.5048 | 0.5545 | 0.4707 | 0.5929 | 0.7085 | 0.6116 | 0.6762
opt-iml-max-1.3b | bert-base-multilingual-cased | 0.5108 | 0.5289 | 0.5330 | 0.5173 | 0.5442 | 0.5635 | 0.5450 | 0.5626
electra-large-discriminator | 0.4745 | 0.4613 | 0.4788 | 0.5011 | 0.5009 | 0.6021 | 0.5657 | 0.6240
gpt2-medium | 0.5220 | 0.4936 | 0.5053 | 0.5192 | 0.5431 | 0.6894 | 0.5497 | 0.6141
mGPT | 0.6135 | 0.6200 | 0.6061 | 0.6004 | 0.6331 | 0.6238 | 0.5904 | 0.6172
mdeberta-v3-base | 0.5972 | 0.5374 | 0.5093 | 0.6664 | 0.7953 | 0.8235 | 0.8139 | 0.9138
roberta-large-openai-detector | 0.6543 | 0.6591 | 0.6491 | 0.6438 | 0.6535 | 0.5722 | 0.6484 | 0.6772
xlm-roberta-large | 0.5576 | 0.5218 | 0.5196 | 0.5736 | 0.6675 | 0.7572 | 0.6830 | 0.7731
text-davinci-003 | bert-base-multilingual-cased | 0.8289 | 0.8692 | 0.8116 | 0.7427 | 0.7640 | 0.3490 | 0.3876 | 0.3703
electra-large-discriminator | 0.5878 | 0.5924 | 0.5310 | 0.5775 | 0.5942 | 0.4013 | 0.4614 | 0.5302
gpt2-medium | 0.4172 | 0.4284 | 0.4378 | 0.4176 | 0.4411 | 0.4916 | 0.3651 | 0.4088
mGPT | 0.7636 | 0.8044 | 0.7709 | 0.6834 | 0.7306 | 0.3624 | 0.4103 | 0.3518
mdeberta-v3-base | 0.7193 | 0.7398 | 0.7533 | 0.6600 | 0.6873 | 0.3190 | 0.3248 | 0.2649
roberta-large-openai-detector | 0.5722 | 0.5691 | 0.5671 | 0.5618 | 0.5534 | 0.4024 | 0.5216 | 0.4817
xlm-roberta-large | 0.7430 | 0.7991 | 0.7469 | 0.6847 | 0.7062 | 0.4158 | 0.4656 | 0.5216
vicuna-13b | bert-base-multilingual-cased | 0.8212 | 0.8745 | 0.8672 | 0.7552 | 0.8353 | 0.4419 | 0.4167 | 0.4256
electra-large-discriminator | 0.5655 | 0.5769 | 0.5669 | 0.5823 | 0.5920 | 0.5197 | 0.5161 | 0.6136
gpt2-medium | 0.5558 | 0.4521 | 0.4421 | 0.5117 | 0.5204 | 0.5360 | 0.4994 | 0.5329
mGPT | 0.7242 | 0.7568 | 0.7366 | 0.6741 | 0.7312 | 0.4275 | 0.4706 | 0.3665
mdeberta-v3-base | 0.5538 | 0.5773 | 0.5925 | 0.5141 | 0.5328 | 0.4013 | 0.2795 | 0.2131
roberta-large-openai-detector | 0.6170 | 0.6150 | 0.6093 | 0.6029 | 0.5980 | 0.4421 | 0.5623 | 0.5310
xlm-roberta-large | 0.7038 | 0.7807 | 0.7476 | 0.6598 | 0.7234 | 0.4878 | 0.4896 | 0.5116

Table 16: Cross-generator performance of all detection models fine-tuned on
the English language (evaluated per LLM).

| Detector | Test LLM
---|---|---
Train LLM | Base Model | text-davinci-003 | gpt-3.5-turbo | gpt-4 | alpaca-lora-30b | vicuna-13b | llama-65b | opt-66b | opt-iml-max-1.3b
alpaca-lora-30b | bert-base-multilingual-cased | 0.8581 | 0.8905 | 0.8078 | 0.8295 | 0.8268 | 0.5157 | 0.4523 | 0.5286
electra-large-discriminator | 0.5473 | 0.5414 | 0.5105 | 0.5562 | 0.5624 | 0.5363 | 0.4681 | 0.4795
gpt2-medium | 0.6630 | 0.5975 | 0.4491 | 0.7259 | 0.5730 | 0.5714 | 0.4916 | 0.5837
mGPT | 0.7748 | 0.8005 | 0.6966 | 0.8025 | 0.7508 | 0.6114 | 0.5314 | 0.5592
mdeberta-v3-base | 0.8632 | 0.8505 | 0.8237 | 0.8465 | 0.8484 | 0.4766 | 0.5173 | 0.6004
roberta-large-openai-detector | 0.6667 | 0.6847 | 0.6452 | 0.6807 | 0.6629 | 0.5254 | 0.4764 | 0.3758
xlm-roberta-large | 0.8169 | 0.8136 | 0.6452 | 0.8599 | 0.7653 | 0.4279 | 0.4353 | 0.5307
gpt-3.5-turbo | bert-base-multilingual-cased | 0.8364 | 0.9011 | 0.8290 | 0.7524 | 0.7909 | 0.3473 | 0.3842 | 0.3736
electra-large-discriminator | 0.5457 | 0.5958 | 0.5620 | 0.5250 | 0.5732 | 0.4545 | 0.4451 | 0.4277
gpt2-medium | 0.5610 | 0.5785 | 0.5022 | 0.5185 | 0.5366 | 0.5299 | 0.3966 | 0.5121
mGPT | 0.7977 | 0.8596 | 0.8033 | 0.7172 | 0.7721 | 0.3934 | 0.4093 | 0.3702
mdeberta-v3-base | 0.8451 | 0.9266 | 0.9040 | 0.7635 | 0.8030 | 0.3472 | 0.3405 | 0.3264
roberta-large-openai-detector | 0.6670 | 0.6981 | 0.6875 | 0.6202 | 0.6554 | 0.4468 | 0.4257 | 0.3329
xlm-roberta-large | 0.8127 | 0.8939 | 0.8010 | 0.7125 | 0.7738 | 0.3816 | 0.3816 | 0.4005
gpt-4 | bert-base-multilingual-cased | 0.8240 | 0.8702 | 0.8802 | 0.7418 | 0.7969 | 0.4285 | 0.4251 | 0.3798
electra-large-discriminator | 0.5456 | 0.5537 | 0.5785 | 0.5060 | 0.5510 | 0.4553 | 0.4632 | 0.4512
gpt2-medium | 0.4109 | 0.5059 | 0.5551 | 0.4236 | 0.4680 | 0.5172 | 0.3713 | 0.4303
mGPT | 0.7428 | 0.8420 | 0.8526 | 0.6301 | 0.7372 | 0.4196 | 0.3823 | 0.3614
mdeberta-v3-base | 0.7977 | 0.9095 | 0.9113 | 0.6895 | 0.7768 | 0.3573 | 0.3471 | 0.3371
roberta-large-openai-detector | 0.6422 | 0.6892 | 0.7063 | 0.5845 | 0.6375 | 0.4231 | 0.3862 | 0.3187
xlm-roberta-large | 0.6783 | 0.7627 | 0.7680 | 0.6233 | 0.7000 | 0.3695 | 0.3670 | 0.3783
llama-65b | bert-base-multilingual-cased | 0.3458 | 0.3815 | 0.4299 | 0.4424 | 0.5081 | 0.8214 | 0.6687 | 0.7356
electra-large-discriminator | 0.4849 | 0.4632 | 0.4809 | 0.5045 | 0.5231 | 0.7180 | 0.5699 | 0.6151
gpt2-medium | 0.4832 | 0.4536 | 0.4721 | 0.4963 | 0.5024 | 0.7405 | 0.5325 | 0.5924
mGPT | 0.5262 | 0.5447 | 0.5199 | 0.5776 | 0.5742 | 0.6943 | 0.5542 | 0.6419
mdeberta-v3-base | 0.4746 | 0.4224 | 0.4382 | 0.5887 | 0.6442 | 0.8678 | 0.7715 | 0.8579
roberta-large-openai-detector | 0.6111 | 0.6331 | 0.6282 | 0.5793 | 0.6229 | 0.6602 | 0.5422 | 0.4934
xlm-roberta-large | 0.4259 | 0.4284 | 0.4451 | 0.5757 | 0.6168 | 0.8565 | 0.7240 | 0.8332
opt-66b | bert-base-multilingual-cased | 0.5182 | 0.5420 | 0.5370 | 0.5378 | 0.6400 | 0.7476 | 0.7074 | 0.7397
electra-large-discriminator | 0.4604 | 0.4612 | 0.4785 | 0.4740 | 0.5282 | 0.6504 | 0.6009 | 0.6205
gpt2-medium | 0.5741 | 0.5033 | 0.4575 | 0.5371 | 0.5645 | 0.6102 | 0.6019 | 0.6506
mGPT | 0.5684 | 0.5752 | 0.5390 | 0.5698 | 0.5986 | 0.5700 | 0.5972 | 0.6038
mdeberta-v3-base | 0.6073 | 0.4019 | 0.4000 | 0.7470 | 0.7419 | 0.8027 | 0.8651 | 0.8861
roberta-large-openai-detector | 0.6509 | 0.6478 | 0.6341 | 0.6311 | 0.6360 | 0.6075 | 0.6683 | 0.6683
xlm-roberta-large | 0.5889 | 0.4781 | 0.4654 | 0.6340 | 0.6626 | 0.7066 | 0.7053 | 0.7086
opt-iml-max-1.3b | bert-base-multilingual-cased | 0.3503 | 0.3595 | 0.3455 | 0.4341 | 0.5012 | 0.7651 | 0.6947 | 0.9178
electra-large-discriminator | 0.5284 | 0.5251 | 0.4909 | 0.5445 | 0.5738 | 0.6557 | 0.5849 | 0.7075
gpt2-medium | 0.5215 | 0.4668 | 0.4399 | 0.4763 | 0.5111 | 0.5719 | 0.4972 | 0.6770
mGPT | 0.4806 | 0.4060 | 0.3641 | 0.6005 | 0.5902 | 0.7384 | 0.6777 | 0.8787
mdeberta-v3-base | 0.4692 | 0.3456 | 0.3352 | 0.5788 | 0.5674 | 0.5690 | 0.8046 | 0.9323
roberta-large-openai-detector | 0.6303 | 0.6232 | 0.5829 | 0.6268 | 0.6424 | 0.5714 | 0.6498 | 0.7347
xlm-roberta-large | 0.3438 | 0.3310 | 0.3320 | 0.4737 | 0.4500 | 0.5210 | 0.6757 | 0.8856
text-davinci-003 | bert-base-multilingual-cased | 0.8398 | 0.8700 | 0.7840 | 0.7585 | 0.8105 | 0.3832 | 0.4459 | 0.4992
electra-large-discriminator | 0.5908 | 0.5807 | 0.5429 | 0.5449 | 0.5745 | 0.4342 | 0.4713 | 0.4590
gpt2-medium | 0.7275 | 0.6582 | 0.4750 | 0.6549 | 0.6159 | 0.4904 | 0.4822 | 0.6132
mGPT | 0.8230 | 0.8604 | 0.7730 | 0.7351 | 0.7823 | 0.4025 | 0.4022 | 0.3777
mdeberta-v3-base | 0.8459 | 0.8898 | 0.8881 | 0.7783 | 0.8136 | 0.3620 | 0.3554 | 0.3517
roberta-large-openai-detector | 0.6641 | 0.6923 | 0.6602 | 0.6147 | 0.6459 | 0.4523 | 0.4150 | 0.3379
xlm-roberta-large | 0.9113 | 0.9345 | 0.8569 | 0.7909 | 0.8627 | 0.3898 | 0.4001 | 0.4358
vicuna-13b | bert-base-multilingual-cased | 0.8525 | 0.8954 | 0.8444 | 0.7812 | 0.8483 | 0.4819 | 0.4452 | 0.4992
electra-large-discriminator | 0.5415 | 0.5950 | 0.5707 | 0.5428 | 0.6017 | 0.5586 | 0.4745 | 0.4876
gpt2-medium | 0.7084 | 0.6886 | 0.5819 | 0.6673 | 0.7066 | 0.5962 | 0.5040 | 0.6613
mGPT | 0.7824 | 0.7967 | 0.7664 | 0.7500 | 0.7815 | 0.6007 | 0.5111 | 0.4828
mdeberta-v3-base | 0.8518 | 0.9009 | 0.8630 | 0.7728 | 0.8656 | 0.4793 | 0.4083 | 0.4794
roberta-large-openai-detector | 0.6966 | 0.7304 | 0.6872 | 0.6489 | 0.7141 | 0.5363 | 0.4789 | 0.4041
xlm-roberta-large | 0.7742 | 0.7924 | 0.6991 | 0.7086 | 0.8133 | 0.5095 | 0.5060 | 0.6679

Table 17: Cross-generator performance of all detection models fine-tuned on
the Spanish language (evaluated per LLM).

| Detector | Test LLM
---|---|---
Train LLM | Base Model | text-davinci-003 | gpt-3.5-turbo | gpt-4 | alpaca-lora-30b | vicuna-13b | llama-65b | opt-66b | opt-iml-max-1.3b
alpaca-lora-30b | bert-base-multilingual-cased | 0.7279 | 0.7424 | 0.7276 | 0.7127 | 0.7079 | 0.4906 | 0.4891 | 0.5071
electra-large-discriminator | 0.4746 | 0.4737 | 0.4906 | 0.4889 | 0.4873 | 0.6388 | 0.5265 | 0.5725
gpt2-medium | 0.4154 | 0.4978 | 0.4338 | 0.6095 | 0.4154 | 0.5750 | 0.3855 | 0.4667
mGPT | 0.6885 | 0.7089 | 0.6610 | 0.7077 | 0.6784 | 0.5698 | 0.5004 | 0.4986
mdeberta-v3-base | 0.8700 | 0.8366 | 0.7959 | 0.9018 | 0.8637 | 0.5788 | 0.7436 | 0.8611
roberta-large-openai-detector | 0.5134 | 0.5240 | 0.5139 | 0.5359 | 0.4923 | 0.4890 | 0.4723 | 0.4254
xlm-roberta-large | 0.8739 | 0.8502 | 0.7762 | 0.9212 | 0.8470 | 0.5112 | 0.5750 | 0.8006
gpt-3.5-turbo | bert-base-multilingual-cased | 0.8358 | 0.8834 | 0.8548 | 0.7617 | 0.8171 | 0.3808 | 0.4363 | 0.4259
electra-large-discriminator | 0.4347 | 0.4889 | 0.4983 | 0.4408 | 0.4384 | 0.4084 | 0.3857 | 0.3908
gpt2-medium | 0.4234 | 0.4879 | 0.4599 | 0.4539 | 0.4328 | 0.4395 | 0.3623 | 0.4325
mGPT | 0.7712 | 0.8387 | 0.7840 | 0.7107 | 0.7594 | 0.3828 | 0.4280 | 0.3666
mdeberta-v3-base | 0.8608 | 0.9185 | 0.9074 | 0.8127 | 0.8515 | 0.4025 | 0.3799 | 0.3644
roberta-large-openai-detector | 0.4998 | 0.5257 | 0.5247 | 0.5126 | 0.4993 | 0.4612 | 0.4788 | 0.3629
xlm-roberta-large | 0.9057 | 0.9472 | 0.9320 | 0.8523 | 0.8832 | 0.4311 | 0.4196 | 0.4130
gpt-4 | bert-base-multilingual-cased | 0.7354 | 0.7671 | 0.7747 | 0.6645 | 0.7139 | 0.4094 | 0.4649 | 0.4051
electra-large-discriminator | 0.4297 | 0.4782 | 0.4788 | 0.4560 | 0.4521 | 0.4829 | 0.4246 | 0.4328
gpt2-medium | 0.4549 | 0.4935 | 0.5003 | 0.4893 | 0.4583 | 0.4802 | 0.4106 | 0.4643
mGPT | 0.7582 | 0.8458 | 0.8566 | 0.6444 | 0.7497 | 0.3996 | 0.4090 | 0.3656
mdeberta-v3-base | 0.7805 | 0.8126 | 0.8134 | 0.7199 | 0.7597 | 0.3770 | 0.3834 | 0.3425
roberta-large-openai-detector | 0.4814 | 0.4983 | 0.5355 | 0.4818 | 0.4794 | 0.4322 | 0.4600 | 0.3530
xlm-roberta-large | 0.8743 | 0.9378 | 0.9499 | 0.7668 | 0.8342 | 0.3855 | 0.3779 | 0.3651
llama-65b | bert-base-multilingual-cased | 0.4444 | 0.4964 | 0.5551 | 0.4455 | 0.5476 | 0.7071 | 0.6247 | 0.6517
electra-large-discriminator | 0.4662 | 0.4574 | 0.4546 | 0.4690 | 0.4708 | 0.5003 | 0.4754 | 0.4648
gpt2-medium | 0.3999 | 0.4182 | 0.4093 | 0.4423 | 0.4377 | 0.6506 | 0.4791 | 0.4855
mGPT | 0.4844 | 0.4835 | 0.4891 | 0.5283 | 0.5146 | 0.5925 | 0.5171 | 0.5794
mdeberta-v3-base | 0.5546 | 0.4236 | 0.4816 | 0.6273 | 0.6372 | 0.7263 | 0.7123 | 0.7260
roberta-large-openai-detector | 0.4379 | 0.4328 | 0.4487 | 0.4540 | 0.4430 | 0.5352 | 0.4778 | 0.4396
xlm-roberta-large | 0.4767 | 0.4478 | 0.4730 | 0.5810 | 0.5973 | 0.7443 | 0.7012 | 0.7417
opt-66b | bert-base-multilingual-cased | 0.3359 | 0.3343 | 0.3592 | 0.3989 | 0.4029 | 0.7943 | 0.7091 | 0.7573
electra-large-discriminator | 0.4473 | 0.4202 | 0.4189 | 0.4424 | 0.4409 | 0.4619 | 0.4759 | 0.4674
gpt2-medium | 0.4902 | 0.4222 | 0.3950 | 0.4765 | 0.4634 | 0.5932 | 0.5526 | 0.5827
mGPT | 0.3959 | 0.3604 | 0.3596 | 0.5127 | 0.4866 | 0.6481 | 0.6793 | 0.8315
mdeberta-v3-base | 0.4262 | 0.3398 | 0.3375 | 0.5127 | 0.4807 | 0.4615 | 0.7289 | 0.8663
roberta-large-openai-detector | 0.4435 | 0.4209 | 0.4359 | 0.4382 | 0.4294 | 0.4511 | 0.4890 | 0.4148
xlm-roberta-large | 0.3934 | 0.3304 | 0.3320 | 0.5074 | 0.4554 | 0.6092 | 0.7948 | 0.9364
opt-iml-max-1.3b | bert-base-multilingual-cased | 0.3321 | 0.3321 | 0.3327 | 0.3456 | 0.3372 | 0.3776 | 0.4305 | 0.7076
electra-large-discriminator | 0.5100 | 0.4353 | 0.3585 | 0.5607 | 0.4552 | 0.5110 | 0.5510 | 0.5909
gpt2-medium | 0.4438 | 0.3922 | 0.3737 | 0.4431 | 0.4279 | 0.5229 | 0.4935 | 0.5983
mGPT | 0.3341 | 0.3257 | 0.3381 | 0.3707 | 0.3652 | 0.4328 | 0.4766 | 0.7054
mdeberta-v3-base | 0.3482 | 0.3401 | 0.3371 | 0.4485 | 0.3843 | 0.3611 | 0.5402 | 0.7068
roberta-large-openai-detector | 0.5199 | 0.4835 | 0.4789 | 0.4998 | 0.4926 | 0.4773 | 0.5193 | 0.5197
xlm-roberta-large | 0.3360 | 0.3323 | 0.3312 | 0.4210 | 0.3627 | 0.3791 | 0.5594 | 0.7996
text-davinci-003 | bert-base-multilingual-cased | 0.7808 | 0.7940 | 0.6553 | 0.7189 | 0.6841 | 0.3456 | 0.3798 | 0.3753
electra-large-discriminator | 0.4372 | 0.3887 | 0.3642 | 0.4289 | 0.4131 | 0.4951 | 0.4518 | 0.4553
gpt2-medium | 0.5147 | 0.3874 | 0.3753 | 0.4490 | 0.4148 | 0.4860 | 0.4456 | 0.5071
mGPT | 0.7682 | 0.7322 | 0.6571 | 0.7275 | 0.6993 | 0.3722 | 0.4151 | 0.3919
mdeberta-v3-base | 0.6726 | 0.5051 | 0.4342 | 0.7695 | 0.6230 | 0.3618 | 0.6033 | 0.6648
roberta-large-openai-detector | 0.5439 | 0.4846 | 0.4907 | 0.5025 | 0.4830 | 0.4156 | 0.4845 | 0.3512
xlm-roberta-large | 0.7099 | 0.5923 | 0.4626 | 0.7232 | 0.6210 | 0.3623 | 0.4792 | 0.6351
vicuna-13b | bert-base-multilingual-cased | 0.5322 | 0.5444 | 0.5417 | 0.5321 | 0.5429 | 0.4956 | 0.4570 | 0.4338
electra-large-discriminator | 0.4613 | 0.4599 | 0.4605 | 0.4708 | 0.4680 | 0.4852 | 0.4774 | 0.4705
gpt2-medium | 0.4400 | 0.4965 | 0.4809 | 0.5031 | 0.5477 | 0.6459 | 0.4519 | 0.5139
mGPT | 0.6592 | 0.6760 | 0.6616 | 0.6673 | 0.6765 | 0.5875 | 0.5462 | 0.5849
mdeberta-v3-base | 0.8342 | 0.8001 | 0.7582 | 0.8513 | 0.8957 | 0.6075 | 0.7289 | 0.9080
roberta-large-openai-detector | 0.4998 | 0.5137 | 0.5146 | 0.5088 | 0.5269 | 0.5206 | 0.4873 | 0.4461
xlm-roberta-large | 0.8429 | 0.8318 | 0.7992 | 0.8220 | 0.8852 | 0.5332 | 0.5935 | 0.7620

Table 18: Cross-generator performance of all detection models fine-tuned on
the Russian language (evaluated per LLM).

Train Language | Train LLM | text-davinci-003 | gpt-3.5-turbo | gpt-4 | alpaca-lora-30b | vicuna-13b | llama-65b | opt-66b | opt-iml-max-1.3b
---|---|---|---|---|---|---|---|---|---
en | text-davinci-003 |  1.0000 |  0.9630 |  0.9091 |  0.9606 |  0.9153 |  -0.5403 |  -0.3397 |  -0.3932
gpt-3.5-turbo |  0.9630 |  1.0000 |  0.9755 |  0.9000 |  0.8939 |  -0.6074 |  -0.4288 |  -0.4835
gpt-4 |  0.9091 |  0.9755 |  1.0000 |  0.8362 |  0.8697 |  -0.5739 |  -0.4265 |  -0.4911
alpaca-lora-30b |  0.9606 |  0.9000 |  0.8362 |  1.0000 |  0.9185 |  -0.4377 |  -0.2123 |  -0.2412
vicuna-13b |  0.9153 |  0.8939 |  0.8697 |  0.9185 |  1.0000 |  -0.2738 |  -0.0814 |  -0.1394
llama-65b |  -0.5403 |  -0.6074 |  -0.5739 |  -0.4377 |  -0.2738 |  1.0000 |  0.8306 |  0.8684
opt-66b |  -0.3397 |  -0.4288 |  -0.4265 |  -0.2123 |  -0.0814 |  0.8306 |  1.0000 |  0.9320
opt-iml-max-1.3b |  -0.3932 |  -0.4835 |  -0.4911 |  -0.2412 |  -0.1394 |  0.8684 |  0.9320 |  1.0000
es | text-davinci-003 |  1.0000 |  0.9559 |  0.8838 |  0.9107 |  0.9268 |  -0.6977 |  -0.6220 |  -0.6146
gpt-3.5-turbo |  0.9559 |  1.0000 |  0.9642 |  0.8045 |  0.8793 |  -0.7677 |  -0.7582 |  -0.7430
gpt-4 |  0.8838 |  0.9642 |  1.0000 |  0.7110 |  0.8475 |  -0.7397 |  -0.7464 |  -0.7772
alpaca-lora-30b |  0.9107 |  0.8045 |  0.7110 |  1.0000 |  0.9211 |  -0.4645 |  -0.3431 |  -0.3439
vicuna-13b |  0.9268 |  0.8793 |  0.8475 |  0.9211 |  1.0000 |  -0.5080 |  -0.4104 |  -0.4309
llama-65b |  -0.6977 |  -0.7677 |  -0.7397 |  -0.4645 |  -0.5080 |  1.0000 |  0.8401 |  0.7872
opt-66b |  -0.6220 |  -0.7582 |  -0.7464 |  -0.3431 |  -0.4104 |  0.8401 |  1.0000 |  0.9164
opt-iml-max-1.3b |  -0.6146 |  -0.7430 |  -0.7772 |  -0.3439 |  -0.4309 |  0.7872 |  0.9164 |  1.0000
ru | text-davinci-003 |  1.0000 |  0.9575 |  0.9176 |  0.9459 |  0.9629 |  -0.3001 |  -0.1209 |  -0.1660
gpt-3.5-turbo |  0.9575 |  1.0000 |  0.9806 |  0.8830 |  0.9405 |  -0.3082 |  -0.2675 |  -0.2963
gpt-4 |  0.9176 |  0.9806 |  1.0000 |  0.8255 |  0.9281 |  -0.2496 |  -0.2466 |  -0.3122
alpaca-lora-30b |  0.9459 |  0.8830 |  0.8255 |  1.0000 |  0.9460 |  -0.1665 |  0.0646 |  0.0440
vicuna-13b |  0.9629 |  0.9405 |  0.9281 |  0.9460 |  1.0000 |  -0.1229 |  0.0303 |  -0.0324
llama-65b |  -0.3001 |  -0.3082 |  -0.2496 |  -0.1665 |  -0.1229 |  1.0000 |  0.6193 |  0.4562
opt-66b |  -0.1209 |  -0.2675 |  -0.2466 |  0.0646 |  0.0303 |  0.6193 |  1.0000 |  0.8655
opt-iml-max-1.3b |  -0.1660 |  -0.2963 |  -0.3122 |  0.0440 |  -0.0324 |  0.4562 |  0.8655 |  1.0000

Table 19: The correlations between the macro average F1-score performance of
the cross-generator on the English, Spanish, and Russian languages.

Train Language | Train LLM | text-davinci-003 | gpt-3.5-turbo | gpt-4 | alpaca-lora-30b | vicuna-13b | llama-65b | opt-66b | opt-iml-max-1.3b
---|---|---|---|---|---|---|---|---|---
en | text-davinci-003 | 0.6617 (±0.14) | 0.6860 (±0.16) | 0.6598 (±0.15) | 0.6182 (±0.11) | 0.6395 (±0.12) | 0.3917 (±0.06) | 0.4195 (±0.07) | 0.4185 (±0.10)
gpt-3.5-turbo | 0.6643 (±0.14) | 0.7047 (±0.17) | 0.6891 (±0.15) | 0.6328 (±0.11) | 0.6609 (±0.12) | 0.4041 (±0.06) | 0.4083 (±0.08) | 0.3967 (±0.07)
gpt-4 | 0.5993 (±0.11) | 0.6641 (±0.14) | 0.6898 (±0.16) | 0.5517 (±0.08) | 0.5993 (±0.09) | 0.3942 (±0.06) | 0.3954 (±0.06) | 0.3687 (±0.02)
alpaca-lora-30b | 0.6826 (±0.13) | 0.6876 (±0.16) | 0.6679 (±0.16) | 0.6753 (±0.11) | 0.6535 (±0.12) | 0.4519 (±0.06) | 0.4416 (±0.06) | 0.4562 (±0.08)
vicuna-13b | 0.6488 (±0.10) | 0.6619 (±0.15) | 0.6517 (±0.14) | 0.6143 (±0.09) | 0.6476 (±0.12) | 0.4652 (±0.05) | 0.4620 (±0.09) | 0.4563 (±0.13)
llama-65b | 0.4551 (±0.08) | 0.4466 (±0.07) | 0.4754 (±0.06) | 0.4640 (±0.06) | 0.5048 (±0.07) | 0.5713 (±0.10) | 0.5391 (±0.07) | 0.5542 (±0.09)
opt-66b | 0.4925 (±0.05) | 0.4989 (±0.06) | 0.5366 (±0.06) | 0.4708 (±0.02) | 0.5483 (±0.06) | 0.6285 (±0.08) | 0.5587 (±0.05) | 0.5620 (±0.08)
opt-iml-max-1.3b | 0.5614 (±0.06) | 0.5460 (±0.07) | 0.5430 (±0.06) | 0.5745 (±0.07) | 0.6196 (±0.10) | 0.6617 (±0.10) | 0.6280 (±0.10) | 0.6832 (±0.12)
es | text-davinci-003 | 0.7718 (±0.11) | 0.7837 (±0.14) | 0.7114 (±0.16) | 0.6968 (±0.09) | 0.7294 (±0.11) | 0.4163 (±0.04) | 0.4246 (±0.04) | 0.4392 (±0.10)
gpt-3.5-turbo | 0.7236 (±0.13) | 0.7791 (±0.15) | 0.7270 (±0.15) | 0.6585 (±0.10) | 0.7007 (±0.11) | 0.4144 (±0.07) | 0.3976 (±0.03) | 0.3919 (±0.06)
gpt-4 | 0.6631 (±0.15) | 0.7333 (±0.16) | 0.7503 (±0.14) | 0.5998 (±0.11) | 0.6668 (±0.12) | 0.4244 (±0.05) | 0.3917 (±0.04) | 0.3795 (±0.05)
alpaca-lora-30b | 0.7414 (±0.12) | 0.7398 (±0.13) | 0.6540 (±0.14) | 0.7573 (±0.11) | 0.7128 (±0.12) | 0.5235 (±0.06) | 0.4818 (±0.03) | 0.5226 (±0.08)
vicuna-13b | 0.7439 (±0.11) | 0.7713 (±0.11) | 0.7161 (±0.12) | 0.6959 (±0.08) | 0.7616 (±0.09) | 0.5375 (±0.05) | 0.4754 (±0.04) | 0.5260 (±0.10)
llama-65b | 0.4788 (±0.08) | 0.4753 (±0.09) | 0.4877 (±0.07) | 0.5378 (±0.06) | 0.5702 (±0.06) | 0.7655 (±0.08) | 0.6233 (±0.10) | 0.6813 (±0.13)
opt-66b | 0.5669 (±0.06) | 0.5156 (±0.08) | 0.5016 (±0.08) | 0.5901 (±0.09) | 0.6245 (±0.07) | 0.6707 (±0.08) | 0.6780 (±0.10) | 0.6968 (±0.10)
opt-iml-max-1.3b | 0.4749 (±0.10) | 0.4367 (±0.11) | 0.4129 (±0.10) | 0.5335 (±0.07) | 0.5480 (±0.06) | 0.6275 (±0.09) | 0.6550 (±0.10) | 0.8191 (±0.11)
ru | text-davinci-003 | 0.6325 (±0.13) | 0.5549 (±0.16) | 0.4914 (±0.12) | 0.6171 (±0.15) | 0.5626 (±0.12) | 0.4055 (±0.06) | 0.4656 (±0.07) | 0.4830 (±0.13)
gpt-3.5-turbo | 0.6759 (±0.21) | 0.7272 (±0.21) | 0.7087 (±0.21) | 0.6492 (±0.18) | 0.6688 (±0.20) | 0.4152 (±0.03) | 0.4130 (±0.04) | 0.3937 (±0.03)
gpt-4 | 0.6449 (±0.18) | 0.6905 (±0.19) | 0.7013 (±0.19) | 0.6032 (±0.13) | 0.6353 (±0.17) | 0.4238 (±0.04) | 0.4186 (±0.03) | 0.3898 (±0.05)
alpaca-lora-30b | 0.6520 (±0.19) | 0.6620 (±0.16) | 0.6284 (±0.15) | 0.6968 (±0.17) | 0.6417 (±0.18) | 0.5504 (±0.06) | 0.5275 (±0.11) | 0.5903 (±0.17)
vicuna-13b | 0.6099 (±0.17) | 0.6175 (±0.15) | 0.6024 (±0.14) | 0.6222 (±0.16) | 0.6490 (±0.18) | 0.5536 (±0.06) | 0.5346 (±0.10) | 0.5884 (±0.18)
llama-65b | 0.4663 (±0.05) | 0.4514 (±0.03) | 0.4730 (±0.04) | 0.5068 (±0.07) | 0.5212 (±0.08) | 0.6366 (±0.10) | 0.5697 (±0.11) | 0.5841 (±0.13)
opt-66b | 0.4189 (±0.05) | 0.3754 (±0.04) | 0.3769 (±0.04) | 0.4698 (±0.04) | 0.4513 (±0.03) | 0.5742 (±0.13) | 0.6328 (±0.13) | 0.6938 (±0.21)
opt-iml-max-1.3b | 0.4034 (±0.09) | 0.3773 (±0.06) | 0.3643 (±0.05) | 0.4413 (±0.07) | 0.4036 (±0.06) | 0.4374 (±0.07) | 0.5101 (±0.05) | 0.6612 (±0.10)

Table 20: The mean and standard deviation between the macro average F1-score
performance of the cross-generator on the English, Spanish, and Russian
languages.


# Analyzing Behaviors of Mixed Traffic via Reinforcement Learning at
Unsignalized Intersections

Supriya Sarker
<EMAIL_ADDRESS>
###### Abstract

In this report, we delve into two critical research inquiries. Firstly, we
explore the extent to which Reinforcement Learning (RL) agents exhibit
multimodal distributions in the context of stop-and-go traffic scenarios.
Secondly, we investigate how RL-controlled Robot Vehicles (RVs) effectively
navigate their direction and coordinate with other vehicles in complex traffic
environments. Our analysis encompasses an examination of multimodality within
queue length, outflow, and platoon size distributions for both Robot and
Human-driven Vehicles (HVs). Additionally, we assess the Pearson coefficient
correlation, shedding light on relationships between queue length and outflow,
considering both identical and differing travel directions. Furthermore, we
delve into causal inference models, shedding light on the factors influencing
queue length across scenarios involving varying travel directions. Through
these investigations, this report contributes valuable insights into the
behaviors of mixed traffic (RVs and HVs) in traffic management and
coordination.

##  I INTRODUCTION

We are now entering an exciting era of fully autonomous vehicles. Companies
like Waymo [1] have already deployed fully automated vehicles in vibrant urban
centers such as Phoenix and San Francisco, and they have ambitious plans to
extend their presence to additional cities in Washington and California [2].
Similarly, Cruise [3] is making strides in this field, operating in key
locations like Austin, San Francisco, and Phoenix [4]. The operation of
autonomous vehicles in urban environments has spurred significant interest
among intelligent transportation researchers. Urban traffic complexity arises
from the diverse mix of vehicles, each following its own set of rules, signs,
signals, and negotiation styles. Moreover, these regulations can vary
significantly from state to state and country to country, adding to the
intricacy of the traffic landscape.

Navigating through urban areas can be quite demanding, especially when it
comes to dealing with intersections. In 2021, the National Highway Traffic
Safety Administration (NHTSA) [5] reported a concerning statistic. Out of the
10.8 million crashes that occurred in the United States, resulting in
fatalities, injuries, or property damage, a significant 4.9 million
(approximately 45%) unfolded at intersections. This not only poses a serious
safety concern but also translates into substantial economic burdens,
encompassing costs associated with waiting times, collisions, and the energy
expended while waiting.

Traditional methods of controlling intersections include traffic lights and
stop signs. Traffic lights, which have been in use since the early 20th
century, work by allowing for platooning, where vehicles in the permitted
direction can pass without negotiating the right-of-way with others, simply
following the leader. However, the phase of platooning is fixed in a
particular direction. The fixed phase platooning may create a longer queue and
waiting time in a certain direction of intersection.

To minimize delays, such as waiting times, numerous transportation researchers
have put forth a variety of methods. Among these approaches, learning
environments using reinforcement learning (RL) agents have emerged as
particularly effective solutions [6, 7, 8, 9]. Certain endeavors have
encountered challenges due to their assumption of 100% Connected and
Autonomous Vehicles (CAV) in traffic scenarios. It’s important to acknowledge
that as we move toward a future with full autonomy, there will be a
transitional phase of mixed autonomy where human-driven vehicles (HVs) and
robot vehicles (RVs) coexist on the roads. Over the past few years, RVs have
emerged as a promising solution for enhancing traffic performance. When
compared to HVs, RVs have the ability to perceive a broader range of factors
and respond with remarkable speed. This holds great promise for elevating both
the efficiency and safety of urban traffic.

Centralized mixed traffic control systems have notable disadvantages. Firstly,
they are vulnerable to single points of failure, such as technical glitches or
cyberattacks, which can disrupt the entire traffic management system.
Secondly, these systems necessitate extensive data collection and processing
from all vehicles, raising concerns about data privacy and surveillance.
Additionally, centralized control relies on continuous communication between
vehicles and the central authority, leading to communication overhead and
potential latency issues. Scaling up such systems to accommodate a growing
number of vehicles can be complex and costly, limiting their adaptability in
resource-constrained areas. Lastly, centralized control often demands
substantial infrastructure upgrades and investments, making it less flexible
in regions with limited resources. These limitations underscore the importance
of carefully considering the trade-offs when choosing between centralized and
decentralized approaches in mixed traffic scenarios.

In response to the notable demerits associated with centralized mixed traffic
control, transportation researchers are increasingly shifting their focus
toward decentralized mixed autonomy research. This transition is driven by the
recognition of several key benefits that decentralized systems offer.
Decentralized mixed traffic control provides greater scalability, enabling
easy adaptation to varying traffic conditions. It reduces vulnerability by
mitigating risks associated with single points of failure or cyberattacks.
Real-time decision-making capabilities at the local level lead to faster
responses, while the preservation of data privacy addresses concerns about
surveillance and data security. Moreover, decentralized control enhances
traffic flow, promoting smoother interactions on the road by allowing vehicles
to adapt autonomously to localized conditions. These advantages underscore the
growing interest and potential of decentralized approaches in advancing mixed
traffic management and autonomous vehicle technologies. Nevertheless, the
analysis of how a decentralized mixed autonomous system functions as a traffic
light has yet to be explored. Within our research, we’ve delved into several
decentralized approaches. Among these, we’ve chosen to focus on the recent
work by Wang et al. [8] as our primary reference for this analysis, as it
introduces an innovative decentralized mixed unsignalized traffic control
system.

Hence, the driving force behind this report is to delve into the intricacies
of RVs within the context of unsignalized intersection control in mixed
traffic. To kickstart our exploration, we posit the hypothesis that
“Reinforcement Learning (RL) functions akin to a miniature traffic signal.” In
our quest to substantiate this hypothesis, we have formulated a set of precise
research inquiries that we aim to address conclusively by the report’s
conclusion. These research questions are as follows:

  * •

RQ1: To what extent does the distribution of RL agents become multimodal
during a stop-and-go scenario?

  * •

RQ2: How does the RV effectively manage its own direction and coordinate with
others?

Therefore, the main contributions made in this report are

  * •

we investigate the multimodality of queue length, outflow, and platoon size
distribution of RV and HV;

  * •

we analyze the pearson coefficient correlation between the queue length and
outflow of the same and two different directions at a time; and

  * •

we investigate the causal inference of queue length the same and two different
directions at a time.

The rest of the paper is organized as follows. The state-of-the-art
centralized and decentralized methods of controlled unsignalized intersections
are discussed in Section II. The background of the framework is discussed in
Section III. The data processing of the report is discussed in Section IV.
Section V presents the analysis of the reference paper. Section VII draws
conclusions.

##  II RELATED WORK

There have been both centralized and decentralized approaches to control
intersection traffic.

###  II-A Centralized Mixed Autonomy Intersection Traffic Management

Centralized mixed autonomy intersection traffic refers to a traffic management
system where the control and coordination of vehicles, both autonomous and
human-driven, at an intersection or multiple intersections are overseen and
directed by a central authority or traffic management system. In this setup,
decisions related to traffic signal timing, lane assignments, and the
prioritization of vehicles are made at a centralized location and then
communicated to vehicles within the intersection area. Centralized mixed
autonomy intersection traffic aims to optimize traffic flow, enhance safety,
and reduce congestion through centralized control and coordination of various
types of vehicles.

Based on the assumption that in the future most vehicles are autonomous and
connected, Dresner and Stone proposed a novel intersection control protocol
denoted Autonomous Intersection Management (AIM) [10]. AIM protocol consists
of two types of autonomous agents - intersection managers and driver agents.
Intersection managers are responsible for directing the vehicles through the
intersections, while the driver agents are responsible for controlling the CAV
to which they are assigned. To improve the throughput and efficiency of the
system, the driver agents call ahead to the intersection manager and request a
path reservation. The intersection manager then determines whether or not this
request can be met by checking whether it conflicts with any previously
approved reservation or a potential HV. Depending on the approval of the
intersection manager. Connected and autonomous vehicles (CAVs), with the help
of advanced sensing devices, are more accurate and predictable compared to
human-operated vehicles (HVs) [11]. On the other hand, AIM, assuming 100% of
the vehicles are CAVs, was shown to reduce the delay imposed on vehicles by
orders of magnitude compared to traffic signals. Moreover, AIM did not perform
better than traffic signals when more than 10% of the vehicles are HVs.

To prove AIM, Sharon and Stone [12] proposed a Hybrid Autonomous Intersection
Management (H-AIM) to overcome the inefficiency of the AIM in the large
portion of the transition period. Unlike AIM, H-AIM assumes sensing of
approaching vehicles which allows it to identify approaching HVs. If no HV is
observed on a given lane, then trajectories originating from that lane are
assumed to not be occupied by HVs, allowing AVs more flexibility in obtaining
reservations. A single lane entering a four-way intersection can allow three
different turning possibilities (turn left, continue straight, turn right) or
any combination of the three. H-AIM grants reservation in an FCFS order.

Deep Reinforcement Learning (DRL) has gained popularity in understanding real-
road traffic situations and improving the performance of autonomous vehicles
due to its ability to handle complex and dynamic environments. Unlike
traditional rule-based systems that rely on predefined instructions, DRL
allows autonomous vehicles to learn from experience and adapt to real-world
scenarios. By leveraging neural networks and continuous learning, DRL enables
vehicles to make data-driven decisions in real time, which is crucial for
navigating the unpredictability of traffic conditions. Using deep
reinforcement learning, Jang et.al [13] trained a set of two autonomous
vehicles to lead a fleet of vehicles onto a roundabout and then transferred
two policies -(1) a policy with noise injected into the state and action space
and (2) a policy without any injected noise from simulation to a scaled city
without fine-tuning. The noise-injected policy consistently performs an
acceptable metering behavior, implying that the noise eventually aids with the
zero-shot policy transfer.

To control connected vehicles in the vicinity of a signal-free intersection
and minimize the total vehicle delay Yang and Oguchi [14] proposed a model to
predict the overall delay of one intersection with the information collected
by connected vehicles followed by the development of an advanced vehicle
control system based on the delay prediction to achieve the goal of minimizing
intersection delay.

###  II-B Decentralized Mixed Autonomy Intersection Traffic Management

Decentralized Mixed Autonomy Intersection Traffic refers to a traffic
management system where the control and coordination of vehicles, including
autonomous and human-driven ones, at an intersection or multiple intersections
are primarily conducted by individual vehicles or local systems. In this
setup, vehicles have a higher degree of autonomy and are equipped with the
capability to make real-time decisions based on their local perception of the
traffic environment. This decentralized approach often involves vehicles
communicating with each other and with the surrounding infrastructure to
negotiate right-of-way and navigate through intersections safely. Unlike
centralized control systems, decentralized mixed autonomy intersection traffic
relies on distributed decision-making, where each vehicle plays an active role
in adapting to and managing traffic conditions at the intersection. In
decentralized mixed autonomy, vehicles have a higher degree of autonomy in
making local decisions based on real-time information.

Wang et al. [8] proposed a decentralized RL approach to handle current
challenges. They introduced an innovative decentralized reinforcement learning
(RL) approach to effectively address these challenges. Their method involved
defining a control zone around the intersection, providing a comprehensive
view of the traffic conditions within this zone. Once a vehicle entered this
designated area, their proposed method took control.

Within the control zone, each robot vehicle (RV) skillfully encoded its
perception of traffic conditions into a concise, fixed-length representation.
This representation encompassed traffic data from eight distinct directions,
capturing both macroscopic aspects like queue length and waiting times, as
well as microscopic details such as the precise locations of vehicles within
the intersection.

Subsequently, each RV positioned in front of the intersection entrance line
utilized this representation to make informed high-level decisions,
determining whether to ’Stop’ or ’Go.’ Importantly, these high-level decisions
from various RVs at the entrance line were thoughtfully shared and coordinated
through vehicle-to-vehicle (V2V) communication.

Finally, guided by their high-level decisions, each RV seamlessly traversed
the intersection, employing a precise low-level control mechanism to ensure
safe and efficient passage.

##  III Background Study

###  III-A Intersection Geometry

Intersection geometries can vary significantly based on the number of roads or
legs converging at a junction. In this section we will discuss various kinds
of intersection geometries, including 3-leg intersections and 4-leg
intersections, with some examples and context.

####  III-A1 3-Leg Intersection

A 3-leg intersection, often known as a T-intersection, is a fundamental
configuration where three roadways meet at a single point. It typically
comprises a primary road, or the ”stem” of the T, intersecting with two
smaller roads, forming the ”arms” of the T. T-intersections can be found in
urban and rural settings alike, often featuring traffic control measures such
as stop signs or traffic signals to regulate vehicle flow. In urban areas,
proper signage and pedestrian accommodations are crucial for safety, while in
rural regions, drivers on the main road usually have the right-of-way.

####  III-A2 4-Leg Intersection

A 4-leg intersection, also referred to as a four-way intersection, represents
one of the most common intersection geometries. It involves four roadways
converging at a central point, creating a square or rectangular layout. To
manage traffic, four-way intersections often employ traffic signals, stop
signs, or roundabouts. These intersections can vary in complexity, ranging
from simple residential intersections to intricate urban intersections with
multiple lanes, pedestrian crossings, and advanced signal systems. They are
prevalent in urban areas and serve as crucial junctions for transportation
networks.

####  III-A3 Roundabout (Circular Intersection)

A roundabout is a circular intersection where traffic circulates
counterclockwise around a central island. Unlike traditional intersections
with stop signs or traffic signals, roundabouts rely on yield signs at entry
points, fostering a continuous flow of traffic. They are celebrated for their
safety benefits, as they mitigate the severity of collisions and minimize the
need for abrupt stops. Roundabouts can be found in both urban and suburban
settings and come in various sizes to accommodate varying traffic volumes and
vehicle types.

####  III-A4 Diamond Interchange (Cloverleaf Interchange)

A diamond interchange connects two major roads, often highways or expressways,
using entrance and exit ramps arranged in a cloverleaf or diamond-shaped
pattern. This design aims to facilitate the smooth flow of traffic between
intersecting roadways, often incorporating overpasses or underpasses to
separate merging and exiting traffic. Diamond interchanges are prevalent on
highways and expressways, providing efficient access to different routes while
maintaining the continuous flow of traffic on the main roadways.

####  III-A5 Diverging Diamond Interchange)

A Diverging Diamond Interchange (DDI) is a modern interchange design that
briefly shifts traffic to the opposite side of the road within the
interchange. This temporary realignment enables more efficient left-turn
movements and helps alleviate congestion. DDIs are gaining popularity in
highway design due to their capacity to enhance traffic flow and safety,
particularly in locations with high traffic volumes and complex interchange
configurations.

####  III-A6 Continuous Flow Intersection

A Continuous Flow Intersection (CFI) is tailored to enhance left-turn
movements at intersections. It integrates specialized lanes that allow left-
turning traffic to proceed before reaching the primary intersection, reducing
traffic conflicts and congestion. CFIs are often deployed in areas with
substantial traffic loads to optimize overall traffic flow and operational
efficiency, offering a solution to the challenges posed by traditional
intersection layouts.

###  III-B Traffic Light in Intersection

The classification of intersections based on traffic signals is a fundamental
way to understand how traffic is controlled and managed at these critical
points in the transportation network. Intersections can be classified based on
the availability of traffic signals into three primary categories: signalized
intersections, unsignalized intersections, and roundabouts.

####  III-B1 Signalized Intersections

Signalized intersections are intersections equipped with traffic signals (also
known as traffic lights) to control the flow of vehicles and pedestrians.
Traffic signals regulate the right-of-way for different directions of traffic,
alternating between green (go), yellow (prepare to stop), and red (stop)
signals. They are effective at managing traffic flow and reducing accidents.

####  III-B2 Unsignalized Intersections

Unsignalized intersections are intersections without traffic signals, where
the right-of-way is typically determined by yield signs, stop signs, or
general right-of-way rules. Vehicles must yield or stop as required by signage
and applicable traffic laws. Right-of-way is often determined by who arrives
first or other established rules.

####  III-B3 Roundabouts

Roundabouts are circular intersections designed to facilitate continuous
traffic flow without the use of traffic signals or stop signs. Vehicles in a
roundabout yield to circulating traffic and enter when a safe gap is
available. Traffic flows counterclockwise around a central island. they
promote safer and more efficient traffic flow.

###  III-C Mixed Traffic Control

Mixed traffic represents a dynamic and evolving transportation scenario where
autonomous or robot vehicles coexist alongside traditional human-driven
vehicles within the same interconnected road network. In this context,
envision the autonomous vehicle as an independent decision-maker, with its
surroundings, consisting of the road layout and traffic conditions, forming
the canvas upon which it navigates. The guiding light for this autonomous
entity is a reward signal, typically encapsulating measures of driving
excellence, safety, and operational efficiency.

To effectively employ Reinforcement Learning (RL), our autonomous vehicle must
first create an accurate and comprehensive representation of its state. This
representation encompasses crucial details such as the vehicle’s current
location, speed, orientation, the presence of nearby objects, the status of
traffic signals, and more. This state representation serves as the
foundational input to our RL model, setting the stage for informed decision-
making.

Within this context, the action space comes into play, defining the spectrum
of choices available to our autonomous vehicle. These actions encompass
pivotal decisions such as acceleration, braking, steering maneuvers, and lane
changes. The RL agent, operating within its current state, orchestrates its
actions with the overarching goal of maximizing cumulative rewards over time.

The rewards themselves are the linchpin of this learning process, acting as
evaluators of performance. They assign numerical values to states and actions,
guiding the vehicle toward behaviors aligned with driving safety, compliance
with traffic rules, and optimal navigation. For instance, rewards can be
structured around concepts like maintaining a safe following distance,
adhering to speed limits, and avoiding collisions.

To accomplish this formidable task, RL models like Q-learning or Deep
Q-Networks (DQN) step in, refining their understanding of the environment by
iteratively estimating the value of specific actions in various states. These
models empower the RL agent to explore different courses of action, observing
the outcomes and adjusting its strategies accordingly.

During training, the autonomous vehicle actively engages with a simulated
environment or controlled real-world scenarios. It employs the RL model to
select actions guided by its current state and learns from the resulting
rewards. Special exploration strategies are employed to encourage the vehicle
to experiment with diverse actions and glean insights into their consequences.

As this journey unfolds, the RL agent accumulates experiences and continuously
refines its policy to optimize the expected cumulative reward. The RL model
stands as a steadfast ally, aiding the vehicle in adapting its decisions to
the ever-changing dynamics of traffic conditions and in fine-tuning its
driving behavior.

In this quest for autonomous driving prowess, developers often turn to
simulations, which provide a safe and controlled playground for training and
evaluating RL-based policies. These simulations allow vehicles to traverse a
spectrum of scenarios, from routine to rare and complex, without exposure to
real-world risks, fostering a resilient and adaptive autonomous driving
ecosystem.

###  III-D Mixed intersection management

Within our report, we recognize the importance of making complex terminology
accessible and comprehensible to our readers. To ensure clarity and
understanding, we have thoughtfully identified and defined key terms and
concepts used throughout the document.

  * •

Approaching the intersection Approaching the intersection refers to the count
of vehicles that have entered the control zone and are proceeding toward the
intersection from a specific direction. These vehicles are actively moving
closer to the intersection and are within the designated control area as they
approach the junction.

  * •

Outflow Outflow refers to the vehicles that have entered the intersection but
have not yet completed their passage through it. These are vehicles that are
currently within the intersection, in the process of navigating it, but have
not yet exited to continue in their intended direction.

  * •

Exit Exit refers to the count of vehicles that have effectively traversed the
intersection and continued on their path in a different direction. To
illustrate, consider a scenario where a vehicle initially embarks on a journey
in the ’left-straight’ direction. As it makes its way from the left side of
the intersection, successfully navigating through the junction node, and
ultimately progressing in a ’straight to right’ direction, it becomes part of
the outflow category.

  * •

Queue: Formed when vehicles are moving at a speed slower than $0.1$m/s, in the
same lane. Sometimes due to reaction time, they can be a bit far.

  * •

Waiting time Waiting time refers to the amount of time that a vehicle spends
stationary or in a state of reduced movement while waiting for a specific
event or condition to change.

##  IV DATA PROCESSING

###  IV-A Intersections Architecture

In our referenced study by Wang et al. [8], we embarked on a comprehensive
examination of traffic dynamics at four distinct junctions: junctions 229,
332, 334, and 499. These junctions, while sharing the commonality of being
4-leg intersections, each present a unique configuration with a varying number
of lanes.

To facilitate clarity and precision in our analysis, Wang introduced a
nomenclature that encompassed eight distinct directional categories: ’top-
straight,’ ’top-left,’ ’right-straight,’ ’right-left,’ ’bottom-straight,’
’bottom-left,’ ’left-straight,’ and ’left-left.’ These designations capture
the diverse ways in which vehicles approach and exit these intersections,
painting a comprehensive picture of traffic flow patterns.

Moreover, to ensure ease of identification and traceability, each lane within
every directional category at each junction was thoughtfully assigned a
specific lane number. This meticulous lane numbering system simplifies the
process of pinpointing and interpreting data collected from specific lanes
within a given directional category at any of the four junctions.

###  IV-B Data acquisition and pre-processing

From our comprehensive trajectory dataset, we’ve meticulously extracted a rich
array of traffic insights. These include valuable information such as the
temporal progression (timestep), the extent of congestion (queue length), the
duration of waits (wait time), the decisive actions taken by the head vehicle
in each lane, the current traffic light phase in specific lanes, outflow
rates, approach patterns, exit points, and the lane-specific wait times. These
details have been meticulously collected for every lane within each direction
across four distinct junctions.

To streamline the data for practical analysis, we employ a thoughtful
aggregation process. Specifically, we consolidate the queue lengths for all
lanes within the same direction at a given junction. Likewise, we aggregate
the wait times for all lanes in a particular direction at each junction. This
approach allows us to simplify our analysis, focusing on the overall queue
lengths and wait times for each directional flow while disregarding the
intricate specifics of individual lanes.

However, when it comes to lane-wise wait times, we continue to consider the
wait times associated with each specific lane within a particular directional
flow at each junction. This distinction ensures that we retain detailed
insights into lane-specific performance, which can be vital for addressing
specific traffic challenges and optimizing lane-level strategies.

##  V Data Analysis

In the data analysis section, we delve into several key analytical metrics for
both RV and HV. These metrics provide valuable insights into the system’s
performance. We examine the Pearson Coefficient Correlation (PCC) between
queue length and outflow, explore the frequency distribution of queue length
and outflow, and investigate the frequency distribution of platoon size. Each
of these metrics offers a unique perspective on the behavior of the traffic
system.

####  V-1 Pearson Coefficient Correlation

This is a statistic that quantifies the degree to which two variables are
linearly related. It measures the strength and direction (positive or
negative) of the linear relationship between two continuous variables. It’s a
valuable tool in statistics and data analysis to assess the degree of
association between variables.

$r=\frac{\sum{(X_{i}-\bar{X})(Y_{i}-\bar{Y})}}{\sqrt{\sum{(X_{i}-\bar{X})^{2}}\sum{(Y_{i}-\bar{Y})^{2}}}}$
(1)

The Pearson correlation coefficient, denoted as ”r,” ranges from -1 to 1. If r
is close to 1, it indicates a strong positive linear relationship. If r is
close to -1, it indicates a strong negative linear relationship. If r is close
to 0, it indicates a weak or no linear relationship.

The motivation behind conducting Pearson Coefficient Correlation (PCC)
analysis is to delve into the intricate interplay of queue lengths in
different directions. It allows us to unearth patterns and relationships
within the data, particularly highlighting instances where queue lengths in
specific directions move in tandem. Positive PCC values between two
directional queue lengths signify a synchronization in their fluctuations,
whether it’s an increase or decrease.

In Fig 1(a) and 1(b) depicted PCC of queue length in non-conflicting and
conflicting directions, respectively and in fig. 1(c) and 1(d) the outflow of
non-conflicting and conflicting directions, respectively of junction 229 for
both RL agents (diagonally bottom half) and TL (diagonally upper half). For
all the junctions one phenomenon is significantly noticeable that is the PCC
(either positive or negative) exits for RL compared to TL, especially for
conflicting directions. On the other hand, there is less variation in TL which
results in missing correlation in several directions. It turns out that for
non-conflicting directions PCC value is higher than conflicting directions.
For TL conflicting directions PCC reflected that there is no such correlation
which means the traffic light mechanism maintains a fixed traffic light phase
that is programmed in it disregarding the particular necessity of the traffic
scenarios. On the other hand, RL can understand the traffic scenarios and
tries to synchronize the conflicting directions to avoid congestion and longer
queues. This analysis slightly answers to our RQ2 which is how RL organizes
directions. Also, it supports our hypothesis that RL works as a mini-traffic
light.

(a)

(b)

(c)

(d)

Figure 1: PCC of queue length in (a) non-conflicting directions (b)
conflicting directions for junction 299, outflow in (c) non-conflicting
directions (d) conflicting directions for junction 299

(a)

(b)

(c)

(d)

Figure 2: PCC of queue length in (a) non-conflicting directions (b)
conflicting directions for junction 332, outflow in (c) non-conflicting
directions (d) conflicting directions for junction 332

(a)

(b)

(c)

(d)

Figure 3: PCC of queue length in (a) non-conflicting directions (b)
conflicting directions for junction 334, outflow in (c) non-conflicting
directions (d) conflicting directions for junction 334

(a)

(b)

(c)

(d)

Figure 4: PCC of queue length in (a) non-conflicting directions (b)
conflicting directions for junction 499, outflow in (c) non-conflicting
directions (d) conflicting directions for junction 499

####  V-2 Queue Length Frequency Distribution

Examining the distribution of queue lengths, a compelling observation comes to
light: the queue lengths generated by Robot Vehicles (RVs) exhibit a frequent
occurrence of shorter queues. This trend not only characterizes the RV-
generated queue lengths but also echoes the same pattern observed in both non-
conflicting and conflicting directions, as earlier revealed through our
Pearson Coefficient Correlation (PCC) analysis.

This alignment in queue length patterns particularly stands out among non-
conflicting directions, where a harmonious synchronization seems to manifest.
It’s worth noting that while this pattern holds true for the majority of
scenarios, there are instances where we encounter limitations due to the
unpredictability of traffic demand in specific directions. In such cases, a
conclusive validation of this observation might be hindered by insufficient
traffic flow data.

In stark contrast, the Traffic Lights (TL) system adheres to fixed-length
traffic phases, lacking the dynamic adaptability to tailor traffic light
phases based on fluctuating traffic demand scenarios for each direction. This
rigidity sometimes leads to situations where, despite the absence of vehicles
in a specific direction, traffic red lights impede progress, resulting in
unwarranted delays—a suboptimal use of time and resources.

However, in the realm of Robot Vehicles, such inefficiencies are mitigated.
RVs possess the innate capability to autonomously organize and optimize
directional flows, intelligently considering the traffic situations in other
directions. This agility ensures that time and resources are utilized more
efficiently, paving the way for a smoother and more responsive traffic
management system.

This observation proves we can get an idea of multimodality of RL agents
during a stop-and-go scenario and management of RL agent’s own direction and
coordinate with others.

####  V-3 Distribution of Platoon Sizes

A platoon refers to a group of vehicles that are closely following one another
in a line or convoy, often at a relatively short distance and with little
space between them. Platooning can occur naturally in traffic, especially on
highways, where vehicles tend to travel at similar speeds and maintain a
consistent following distance. A shorter size of the platoon promotes changes
in the shorter queue and shorter wait times. Fig 5 depicted the frequency
distribution of platoon size for RL and TL. It demonstrated that RL creates
shorter queues such as lengths 1, 2, and 3 more frequently compared to TL.
Hence, it supports our hypothesis - ’RL acts as mini traffic light’.

(a)

(b)

(c)

(d)

Figure 5: Distribution of platoon size for junction (a) 229, (b) 332, (c) 334,
(d) 499

##  VI Causal Analysis

In this section, we discuss the investigation of RL agent’s decision and its
consequence on queue length and wait time. Many studies have conducted causal
analysis regrading mobility [15, 16]. In this work, we have tried dowhy [17],
and CausalImpact [18] tool for time series so far. For this purpose, we
extracted a data set containing Time Step, Junction, Direction, Lane, Average
Wait Time, Head of queue, ql Type, and Action.
We followed two approaches for our causal investigation - (1) Causal effect of
queue length of action in the same direction, (2) Causal effect of queue
length of action in the same direction.

###  VI-A Dataset description for causality

Table I provides a comprehensive summary of the instances associated with
different directions that are used in the causality analysis. Each row
represents a specific direction and the corresponding number of data instances
such as queue length, action, wait time, etc.

TABLE I: List of instances in causality dataset Direction | No of instances
---|---
Topleft | 242
Rightstraight | 220
Bottomleft | 211
Leftstraight | 192
Topstraight | 187
Bottomstraight | 61
Leftleft | 61
Rightleft | No instance

###  VI-B Causal Inference by CausalImpact

In Fig 6 the top plot typically shows the original queue length data,
including during the pre-intervention period and the post-intervention period.
CausalImpact estimates what would have happened to the outcome variable if the
intervention hadn’t occurred. This is called the ”counterfactual” scenario. It
uses Bayesian methods to generate uncertainty intervals. This plot shows the
posterior predictive distribution of the outcome variable in the post-
intervention period. This plot compares the observed data with the
counterfactual prediction. It shows how the actual data deviates from what was
expected in the absence of the intervention. The cumulative effects plot might
display the cumulative impact of the intervention over time. The summary plot
often combines key information, showing the observed data, predicted
counterfactuals, and the impact estimate in a single graph. We get p-value of
6.3% and prob. of causal effect of 93.7% for queue length in all directions of
junction 229. However, CausalImpact tool can not find the cause of the
causality.

Figure 6: Illustration of the causal impact of queue length in all directions
of 229

###  VI-C Causal Inference by DoWhy

The CausalImpact tool struggles to pinpoint the responsible factor behind a
causal effect. In light of this, we have turned to another powerful causal
inference tool called DoWhy.

We have pursued two distinct approaches to delve into the causal effect of
actions on queue length, specifically focusing on actions in the same
direction shown in Table II and those in different directions In Table III.
Our core assumption is that the action of the leading vehicle serves as the
treatment, and we measure its impact on queue length in both the same and
different directions. We have also explored the possibility of replacing queue
length with average wait time in our analysis. While doing so, we observed
that the changes in p-values and mean values were not remarkably significant.

However, it’s worth noting that the results of our causality analysis lack
definitive conclusions. This is primarily due to the varying flow in different
directions and the fluctuating number of data instances, which can be
attributed to the varying traffic demands.

TABLE II: List of P-value and Mean for same direction causality assumption Direction | P-value | Mean
---|---|---
Bottomleft | $0.96$ | $-0.21$
Bottomstraight | $0.94$ | $-0.89$
Leftleft | $0.96$ | $1.39$
Leftstraight | 0.9 | -0.94
Rightstraight | 1.0 | 0.45
Topleft | 0.9 | -3.52
Topstraight | 0.92 | -10.46
Rightleft | no data | no data
TABLE III: Causal effect for two different directions Direction 1 | Direction 2 | P-value | Mean
---|---|---|---
Bottomleft | Leftleft | 0.88 | 0.014
Bottomstraight | 0.94 | 0.0
Leftstraight | 0.86 | 0.153
Rightstraight | 0.98 | 0.170
Topleft | 0.96 | -0.413
Topstraight | 0.96 | -0.063
Bottomstraight | Leftleft | 0.94 | 0.0
Bottomleft | 0.94 | 0.014
Leftstraight | 0.94 | 0.066
Rightstraight | 0.94 | 0.752
Topleft | 0.58 | 8.882e-16
Topstraight | 0.98 | 0.105
Leftstraight | Leftleft | 0.86 | 0.153
Bottomleft | 0.86 | 0.153
Bottomstraight | 0.94 | 0.066
Rightstraight | 0.86 | 0.105
Topleft | 0.86 | 0.122
Topstraight | 0.86 | 0.142
Rightstraight | Leftleft | 0.98 | 0.170
Bottomleft | 0.98 | 0.170
Bottomstraight | 0.98 | 0.170
Leftstraight | 0.98 | 0.170
Topleft | 0.98 | 0.170
Topstraight | 0.98 | 0.170
Topleft | Leftleft | 0.96 | -0.413
Bottomleft | 0.96 | -0.413
Bottomstraight | 0.96 | -0.413
Leftstraight | 0.96 | -0.413
Rightstraight | 0.96 | -0.413
Topstraight | 0.96 | -0.068
Topstraight | Leftleft | 0.96 | -0.063
Bottomleft | 0.86 | -0.063
Bottomstraight | 0.98 | 0.105
Leftstraight | 0.86 | 0.142
Rightstraight | 0.98 | 0.105
Topleft | 0.96 | -0.068

##  VII CONCLUSIONS

Our comprehensive analysis has provided valuable insights into the behavior of
Reinforcement Learning (RL) agents and their interaction within complex
traffic scenarios. We have addressed two fundamental research questions: the
extent of multimodality in RL agent distributions during stop-and-go scenarios
and the effective management of RVs in coordinating their movements.

Our findings reveal that RL agents often exhibit multimodal distributions in
queue length, outflow, and platoon size, indicating their adaptability and
responsiveness in various traffic situations. Additionally, our examination of
the Pearson coefficient correlation has illuminated the relationships between
queue length and outflow, shedding light on how traffic dynamics are
influenced by RL-controlled and Human-driven Vehicles (HVs). Notably, the PCC
values are notably higher for non-conflicting directions in RL, demonstrating
their capacity to synchronize and optimize traffic flow, thus mitigating
congestion and reducing queue lengths.

Furthermore, our exploration of causal inference models has unveiled the
factors influencing queue length in scenarios with different travel
directions. This analysis offers critical insights into the decision-making
processes of RL agents and their impact on traffic flow.

These findings provide valuable insights into how RL agents effectively manage
traffic directions, answering part of our research question regarding RL’s
coordination abilities. Moreover, they support our hypothesis that RL
functions as a dynamic ”mini-traffic light,” intelligently responding to
traffic conditions rather than adhering rigidly to pre-programmed phases.

In conclusion, our comprehensive analysis not only enriches our understanding
of RL agent behavior but also underscores their potential to revolutionize
traffic management. The adaptability and responsiveness of RL agents hold
promise for enhancing traffic flow and safety in complex traffic scenarios,
ultimately contributing to the advancement of autonomous vehicle technology
and more efficient traffic management systems.

There are many future research directions. First, we aim to analyze the RVs at
the trajectory level [19, 20, 21], under adversarial conditions [22, 23, 24],
and possibly with hardware [25]. This analysis intends to determine whether
combining different elements would enhance the capabilities of robot vehicles
further. Secondly, we aim to scale up our study by incorporating
intersectional mixed traffic into larger areas, ideally citywide road network.
This will involve leveraging existing techniques for large-scale traffic
simulation, reconstruction, and prediction [26, 27, 28, 29, 30, 31, 32, 33].
Third, we want to analyze the impact of various vehicle networks via network
optimization [34], and heterogeneous traffic via multi-agent learning and
optimization [35, 36]. Lastly, we want to combine mixed traffic modeling with
crowd simulation and virtual humans [37, 38, 39, 40, 41, 42] so that together
we can model more realistic road uses with pedestrians as intelligent virtual
characters.

## References

  * [1] Waymo - self-driving technology company.  https://waymo.com.  Accessed on September 5, 2023.
  * [2] Waymo.  Taking our next step in the city by the bay.  https://blog.waymo.com/2022/03/taking-our-next-step-in-city-by-bay.html.
  * [3] Cruise Automation, Inc.  Cruise - self-driving cars, Year of Access.  Accessed on Month Day, Year.
  * [4] Cruise Automation.  One million driverless miles.  https://getcruise.com/news/blog/2023/one-million-driverless-miles/.
  * [5] National Highway Traffic Safety Administration.  Nhtsa.  https://www.nhtsa.gov.
  * [6] Michael Villarreal, Bibek Poudel, and Weizi Li.  Can chatgpt enable its? the case of mixed traffic control via reinforcement learning.  In IEEE International Conference on Intelligent Transportation Systems (ITSC), 2023.
  * [7] Michael Villarreal, Bibek Poudel, Jia Pan, and Weizi Li.  Hybrid traffic control and coordination from pixels.  arXiv preprint arXiv:2302.09167, 2023.
  * [8] Dawei Wang, Weizi Li, Lei Zhu, and Jia Pan.  Learning to control and coordinate hybrid traffic through robot vehicles at complex and unsignalized intersections.  arXiv preprint arXiv:2301.05294, 2023.
  * [9] Dawei Wang, Weizi Li, and Jia Pan.  Large-scale mixed traffic control using dynamic vehicle routing and privacy-preserving crowdsourcing.  IEEE Internet of Things Journal, 2023.
  * [10] Kurt Dresner and Peter Stone.  A multiagent approach to autonomous intersection management.  Journal of artificial intelligence research, 31:591–656, 2008.
  * [11] David Fajardo, Tsz-Chiu Au, S Travis Waller, Peter Stone, and David Yang.  Automated intersection control: Performance of future innovation versus current traffic signal control.  Transportation Research Record, 2259(1):223–232, 2011.
  * [12] Guni Sharon and Peter Stone.  A protocol for mixed autonomous and human-operated vehicles at intersections.  In Autonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, Best Papers, São Paulo, Brazil, May 8-12, 2017, Revised Selected Papers 16, pages 151–167. Springer, 2017.
  * [13] Kathy Jang, Eugene Vinitsky, Behdad Chalaki, Ben Remer, Logan Beaver, Andreas A Malikopoulos, and Alexandre Bayen.  Simulation to scaled city: zero-shot policy transfer for traffic control via autonomous vehicles.  In Proceedings of the 10th ACM/IEEE International Conference on Cyber-Physical Systems, pages 291–300, 2019.
  * [14] Hao Yang and Ken Oguchi.  Intelligent vehicle control at signal-free intersection under mixed connected environment.  IET Intelligent Transport Systems, 14(2):82–90, 2020.
  * [15] Songhe Wang, Kangda Wei, Lei Lin, and Weizi Li.  Spatial-temporal analysis of COVID-19’s impact on human mobility: the case of the united states.  In The 20th and 21st Joint COTA International Conference of Transportation Professionals, 2021.
  * [16] Lei Lin, Feng Shi, and Weizi Li.  Assessing inequality, irregularity, and severity regarding road traffic safety during covid-19.  Scientific Reports, 11(13147):1–7, 2021.
  * [17] Jasjeet S. Sekhon, Brenton Kenkel, and Philip VanderWeele.  Dowhy: A python library for causal inference, 2023.
  * [18] Google, Inc.  CausalImpact: A Package for Causal Inference Using Bayesian Structural Time-Series Models, Year of Access.  Accessed on Date.
  * [19] Weizi Li, David Wolinski, and Ming C. Lin.  ADAPS: Autonomous driving via principled simulations.  In IEEE International Conference on Robotics and Automation (ICRA), pages 7625–7631, 2019.
  * [20] Yu Shen, Weizi Li, and Ming C. Lin.  Inverse reinforcement learning with hybrid-weight trust-region optimization and curriculum learning for autonomous maneuvering.  In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7421–7428, 2022.
  * [21] Lei Lin, Weizi Li, Huikun Bi, and Lingqiao Qin.  Vehicle trajectory prediction using LSTMs with spatial-temporal attention mechanisms.  IEEE Intelligent Transportation Systems Magazine, 14(2):197–208, 2022.
  * [22] Bibek Poudel and Weizi Li.  Black-box adversarial attacks on network-wide multi-step traffic state prediction models.  In IEEE International Conference on Intelligent Transportation Systems (ITSC), pages 3652–3658, 2021.
  * [23] Yu Shen, Laura Zheng, Manli Shu, Weizi Li, Tom Goldstein, and Ming C. Lin.  Gradient-free adversarial training against image corruption for learning-based steering.  In Advances in Neural Information Processing Systems (NeurIPS), pages 26250–26263, 2021.
  * [24] Michael Villarreal, Bibek Poudel, Ryan Wickman, Yu Shen, and Weizi Li.  Autojoin: Efficient adversarial training for robust maneuvering via denoising autoencoder and joint learning.  2023\.
  * [25] Bibek Poudel, Thomas Watson, and Weizi Li.  Learning to control dc motor for micromobility in real time with reinforcement learning.  In IEEE International Conference on Intelligent Transportation Systems (ITSC), pages 1248–1254, 2022.
  * [26] David Wilkie, Jason Sewall, Weizi Li, and Ming C. Lin.  Virtualized traffic at metropolitan scales.  Frontiers in Robotics and AI, 2:11, 2015.
  * [27] Weizi Li, David Wolinski, and Ming C. Lin.  City-scale traffic animation using statistical learning and metamodel-based optimization.  ACM Trans. Graph., 36(6):200:1–200:12, 2017.
  * [28] Weizi Li, Dong Nie, David Wilkie, and Ming C. Lin.  Citywide estimation of traffic dynamics via sparse GPS traces.  IEEE Intelligent Transportation Systems Magazine, 9(3):100–113, 2017.
  * [29] Weizi Li, Meilei Jiang, Yaoyu Chen, and Ming C. Lin.  Estimating urban traffic states using iterative refinement and wardrop equilibria.  IET Intelligent Transport Systems, 12(8):875–883, 2018.
  * [30] Lei Lin, Weizi Li, and Srinivas Peeta.  Efficient data collection and accurate travel time estimation in a connected vehicle environment via real-time compressive sensing.  Journal of Big Data Analytics in Transportation, 1(2):95–107, 2019.
  * [31] Lei Lin, Weizi Li, and Srinivas Peeta.  Predicting station-level bike-sharing demands using graph convolutional neural network.  In Transportation Research Board 98th Annual Meeting (TRB), 2019.
  * [32] Qianwen Chao, Huikun Bi, Weizi Li, Tianlu Mao, Zhaoqi Wang, Ming C. Lin, and Zhigang Deng.  A survey on visual traffic simulation: Models, evaluations, and applications in autonomous driving.  Computer Graphics Forum, 39(1):287–308, 2020.
  * [33] Lei Lin, Weizi Li, and Lei Zhu.  Data-driven graph filter based graph convolutional neural network approach for network-level multi-step traffic prediction.  Sustainability, 14(24):16701, 2022.
  * [34] Ryan Wickman, Xiaofei Zhang, and Weizi Li.  A generic graph sparsification framework using deep reinforcement learning.  In IEEE International Conference on Data Mining (ICDM), pages 1221–1226, 2022.
  * [35] Ryan Wickman, Bibek Poudel, Michael Villarreal, Xiaofei Zhang, and Weizi Li.  Efficient quality-diversity optimization through diverse quality species.  In Genetic and Evolutionary Computation Conference (GECCO), 2023.
  * [36] Ryan Wickman, Xiaofei Zhang, and Weizi Li.  Lrn: Limitless routing networks for effective multi-task learning.  2021\.
  * [37] Weizi Li, Tim Balint, and Jan M. Allbeck.  Using a parameterized memory model to modulate npc AI.  In Proceedings of the 13th International Conference on Intelligent Virtual Agents (IVA), pages 1–14, 2013.
  * [38] Funda Durupinar, Nuria Pelechano, Jan Allbeck, Norman Badler, and Weizi Li.  Individual Differences, pages 159–171.  Springer International Publishing, 2016.
  * [39] Weizi Li and Jan M. Allbeck.  Virtual humans: Evolving with common sense.  In Proceedings of the 5th International Conference on Motion in Games (MIG), pages 182–193, 2012.
  * [40] Weizi Li and Jan M. Allbeck.  The virtual apprentice.  In Proceedings of the 12th International Conference on Intelligent Virtual Agents (IVA), pages 15–27, 2012.
  * [41] Weizi Li, Zichao Di, and Jan M. Allbeck.  Crowd distribution and location preference.  Computer Animation and Virtual Worlds, 23(3-4):343–351, 2012.
  * [42] Weizi Li and Jan M. Allbeck.  Populations with purpose.  In Proceedings of the 4th International Conference on Motion in Games (MIG), pages 132–143, 2011.