|  | Accuracy | Precision | Recall | F1-score |  | Accuracy | Precision | Recall | F1-score
Complete | Baichuan | 0.5908 | 0.8721 | 0.3731 | 0.5226 | Baichuan | 0.5004 | 0.5002 | 0.2581 | 0.3405
QA | 0.9332 | 0.9713 | 0.8978 | 0.9331 | 0.6520 | 0.5962 | 0.6457 | 0.6199
Polish | 0.5256 | 0.6538 | 0.3158 | 0.4259 | 0.5024 | 0.5012 | 0.2573 | 0.3400
Expand | 0.5908 | 0.7873 | 0.4070 | 0.5366 | 0.5052 | 0.5026 | 0.2632 | 0.3455
Summary | 0.6384 | 0.8862 | 0.4500 | 0.5969 | 0.6028 | 0.5574 | 0.5036 | 0.5291
Refine | 0.5660 | 0.7603 | 0.3678 | 0.4958 | 0.5024 | 0.5012 | 0.2581 | 0.3407
Rewrite | 0.5840 | 0.8052 | 0.3852 | 0.5211 | 0.5024 | 0.5012 | 0.2557 | 0.3386
Translate | 0.5360 | 0.6286 | 0.3714 | 0.4669 | 0.5348 | 0.5183 | 0.3462 | 0.4151
Operations Average |  | 0.6206 | 0.7956 | 0.4460 | 0.5624 |  | 0.5378 | 0.5223 | 0.3485 | 0.4087
Complete | ChatGLM | 0.6396 | 0.8939 | 0.4489 | 0.5977 | ChatGLM | 0.4956 | 0.4978 | 0.2554 | 0.3376
QA | 0.9668 | 0.9810 | 0.9530 | 0.9668 | 0.6644 | 0.6062 | 0.6714 | 0.6372
Polish | 0.5248 | 0.6535 | 0.3132 | 0.4235 | 0.5008 | 0.5004 | 0.2559 | 0.3386
Expand | 0.5540 | 0.7368 | 0.3515 | 0.4759 | 0.4912 | 0.4955 | 0.2529 | 0.3349
Summary | 0.6328 | 0.8825 | 0.4420 | 0.5890 | 0.5932 | 0.5519 | 0.4856 | 0.5166
Refine | 0.5724 | 0.7910 | 0.3678 | 0.5022 | 0.4992 | 0.4996 | 0.2560 | 0.3386
Rewrite | 0.5736 | 0.7570 | 0.3861 | 0.5114 | 0.4816 | 0.4906 | 0.2506 | 0.3317
Translate | 0.6948 | 0.8175 | 0.5865 | 0.6830 | 0.4672 | 0.4815 | 0.3066 | 0.3746
Operations Average |  | 0.6449 | 0.8141 | 0.4811 | 0.5937 |  | 0.5242 | 0.5154 | 0.3418 | 0.4012
Complete | GPT3.5 | 0.6952 | 0.9251 | 0.5266 | 0.6712 | GPT3.5 | 0.5044 | 0.5022 | 0.2605 | 0.3430
QA | 0.9740 | 0.9714 | 0.9767 | 0.9740 | 0.6736 | 0.6090 | 0.6793 | 0.6423
Polish | 0.5632 | 0.7705 | 0.3571 | 0.4880 | 0.5032 | 0.5016 | 0.2584 | 0.3411
Expand | 0.7296 | 0.9031 | 0.5938 | 0.7165 | 0.5040 | 0.5020 | 0.2595 | 0.3422
Summary | 0.5460 | 0.7833 | 0.3151 | 0.4494 | 0.5956 | 0.5529 | 0.4847 | 0.5165
Refine | 0.6044 | 0.8145 | 0.4206 | 0.5547 | 0.5012 | 0.5006 | 0.2529 | 0.3360
Rewrite | 0.6836 | 0.8768 | 0.5309 | 0.6613 | 0.5024 | 0.5012 | 0.2565 | 0.3393
Translate | 0.5504 | 0.6676 | 0.3843 | 0.4878 | 0.5336 | 0.5176 | 0.3440 | 0.4133
Operations Average |  | 0.6683 | 0.8390 | 0.5131 | 0.6254 |  | 0.5398 | 0.5234 | 0.3495 | 0.4092
Complete | GPT4 | 0.8780 | 0.9646 | 0.8039 | 0.8769 | Llama3 | 0.5048 | 0.5024 | 0.2614 | 0.3439
QA | 0.9832 | 0.9719 | 0.9948 | 0.9832 | 0.6560 | 0.5981 | 0.6504 | 0.6232
Polish | 0.6696 | 0.8759 | 0.5076 | 0.6427 | 0.5032 | 0.5016 | 0.2576 | 0.3404
Expand | 0.7880 | 0.9128 | 0.6856 | 0.7830 | 0.5048 | 0.5024 | 0.2614 | 0.3439
Summary | 0.5976 | 0.8631 | 0.3882 | 0.5355 | 0.5988 | 0.5549 | 0.4932 | 0.5222
Refine | 0.6904 | 0.8802 | 0.5406 | 0.6698 | 0.5040 | 0.5020 | 0.2595 | 0.3422
Rewrite | 0.7736 | 0.9181 | 0.6581 | 0.7666 | 0.5024 | 0.5012 | 0.2565 | 0.3393
Translate | 0.5620 | 0.6886 | 0.4005 | 0.5064 | 0.5352 | 0.5184 | 0.3427 | 0.4126
Operations Average |  | 0.7428 | 0.8844 | 0.6224 | 0.7205 |  | 0.5387 | 0.5226 | 0.3479 | 0.4085
Complete | Qwen | 0.7384 | 0.9529 | 0.5823 | 0.7229 | Llama2 | 0.5044 | 0.5022 | 0.2621 | 0.3444
QA | 0.9780 | 0.9694 | 0.9868 | 0.9780 | 0.6924 | 0.6203 | 0.7099 | 0.6621
Polish | 0.5132 | 0.5922 | 0.3063 | 0.4038 | 0.5020 | 0.5010 | 0.2556 | 0.3385
Expand | 0.6312 | 0.8347 | 0.4606 | 0.5936 | 0.5016 | 0.5008 | 0.2562 | 0.3390
Summary | 0.5500 | 0.7934 | 0.3204 | 0.4565 | 0.6012 | 0.5563 | 0.4984 | 0.5258
Refine | 0.6904 | 0.8802 | 0.5406 | 0.6698 | 0.5036 | 0.5018 | 0.2586 | 0.3413
Rewrite | 0.5452 | 0.7070 | 0.3445 | 0.4632 | 0.5016 | 0.5008 | 0.2538 | 0.3369
Translate | 0.5448 | 0.6436 | 0.3874 | 0.4837 | 0.5392 | 0.5206 | 0.3532 | 0.4209
Operations Average |  | 0.6489 | 0.7967 | 0.4911 | 0.5964 |  | 0.5433 | 0.5255 | 0.3560 | 0.4136
Table 13: The detection results of the XLNet model trained on a single main operation data on each LLM text. Train Data | Test Data | Chinese | Test Data | English
---|---|---|---|---
Accuracy | Precision | Recall | F1-score | Accuracy | Precision | Recall | F1-score
Create | Baichuan | 0.6473 | 0.6392 | 0.5423 | 0.5774 | Baichuan | 0.8283 | 0.8347 | 0.7818 | 0.8036
ChatGLM | 0.6599 | 0.6454 | 0.5512 | 0.5844 | ChatGLM | 0.8547 | 0.8410 | 0.8290 | 0.8303
GPT3.5 | 0.6555 | 0.6417 | 0.5495 | 0.5824 | GPT3.5 | 0.8471 | 0.8306 | 0.8342 | 0.8273
GPT4 | 0.6686 | 0.6497 | 0.5636 | 0.5931 | Llama3 | 0.8621 | 0.8388 | 0.8457 | 0.8374
Qwen | 0.6615 | 0.6452 | 0.5510 | 0.5846 | Llama2 | 0.8619 | 0.8407 | 0.8424 | 0.8368
| Average | 0.6585 | 0.6443 | 0.5515 | 0.5844 | Average | 0.8508 | 0.8372 | 0.8266 | 0.8271
Update | Baichuan | 0.7261 | 0.7622 | 0.6038 | 0.6635 | Baichuan | 0.8445 | 0.8388 | 0.8213 | 0.8296
ChatGLM | 0.7501 | 0.7698 | 0.6413 | 0.6883 | ChatGLM | 0.8587 | 0.8436 | 0.8466 | 0.8434
GPT3.5 | 0.7666 | 0.7698 | 0.6707 | 0.7040 | GPT3.5 | 0.8609 | 0.8381 | 0.8585 | 0.8472
GPT4 | 0.7841 | 0.7708 | 0.7041 | 0.7209 | Llama3 | 0.8670 | 0.8439 | 0.8624 | 0.8523
Qwen | 0.7841 | 0.7706 | 0.7042 | 0.7210 | Llama2 | 0.8706 | 0.8449 | 0.8686 | 0.8556
| Average | 0.7622 | 0.7686 | 0.6648 | 0.6995 | Average | 0.8603 | 0.8419 | 0.8515 | 0.8456
Delete | Baichuan | 0.7502 | 0.8493 | 0.6598 | 0.7297 | Baichuan | 0.8736 | 0.8730 | 0.8647 | 0.8689
ChatGLM | 0.8136 | 0.8901 | 0.7487 | 0.8070 | ChatGLM | 0.9078 | 0.8846 | 0.9246 | 0.9029
GPT3.5 | 0.7715 | 0.8353 | 0.7005 | 0.7487 | GPT3.5 | 0.8908 | 0.8763 | 0.8967 | 0.8854
GPT4 | 0.8073 | 0.8498 | 0.7537 | 0.7904 | Llama3 | 0.8896 | 0.8762 | 0.8947 | 0.8847
Qwen | 0.7711 | 0.8473 | 0.6978 | 0.7488 | Llama2 | 0.8993 | 0.8839 | 0.9072 | 0.8942
| Average | 0.7827 | 0.8544 | 0.7121 | 0.7649 | Average | 0.8922 | 0.8788 | 0.8976 | 0.8872
Rewrite | Baichuan | 0.7116 | 0.7698 | 0.5709 | 0.6473 | Baichuan | 0.8114 | 0.8268 | 0.7413 | 0.7741
ChatGLM | 0.7483 | 0.7845 | 0.6256 | 0.6869 | ChatGLM | 0.8335 | 0.8289 | 0.7814 | 0.7964
GPT3.5 | 0.7515 | 0.7774 | 0.6359 | 0.6888 | GPT3.5 | 0.8424 | 0.8272 | 0.8033 | 0.8069
GPT4 | 0.7926 | 0.7871 | 0.7049 | 0.7303 | Llama3 | 0.8398 | 0.8289 | 0.7925 | 0.8022
Qwen | 0.7954 | 0.7853 | 0.7131 | 0.7333 | Llama2 | 0.8409 | 0.8294 | 0.7956 | 0.8042
| Average | 0.7599 | 0.7808 | 0.6501 | 0.6973 | Average | 0.8336 | 0.8282 | 0.7828 | 0.7968
Translate | Baichuan | 0.5841 | 0.7553 | 0.4341 | 0.5085 | Baichuan | 0.6242 | 0.6086 | 0.4731 | 0.5141
ChatGLM | 0.6016 | 0.7961 | 0.4684 | 0.5338 | ChatGLM | 0.6292 | 0.6122 | 0.4791 | 0.5189
GPT3.5 | 0.5417 | 0.6204 | 0.3735 | 0.4431 | GPT3.5 | 0.6139 | 0.6065 | 0.4528 | 0.5036
GPT4 | 0.5530 | 0.5984 | 0.3939 | 0.4545 | Llama3 | 0.6248 | 0.6096 | 0.4713 | 0.5138
Qwen | 0.5451 | 0.4214 | 0.5905 | 0.4404 | Llama2 | 0.6291 | 0.6129 | 0.4775 | 0.5185
| Average | 0.5651 | 0.6383 | 0.4521 | 0.4761 | Average | 0.6242 | 0.6100 | 0.4708 | 0.5138
Table 14: The detection results of the Roberta model trained on a single LLM data on each LLM text. Train Data | Test Data | Chinese | Train Data | Test Data | English
---|---|---|---|---|---
Accuracy | Precision | Recall | F1-score | Accuracy | Precision | Recall | F1-score
Baichuan | Baichuan | 0.7223 | 0.7007 | 0.6374 | 0.6591 | Baichuan | Baichuan | 0.9465 | 0.9421 | 0.9517 | 0.9464
ChatGLM | 0.7196 | 0.7097 | 0.6251 | 0.6576 | ChatGLM | 0.9588 | 0.9600 | 0.9593 | 0.9586
GPT3.5 | 0.7326 | 0.7145 | 0.6456 | 0.6706 | GPT3.5 | 0.9611 | 0.9596 | 0.9647 | 0.9609
GPT4 | 0.7376 | 0.7179 | 0.6517 | 0.6752 | Llama3 | 0.9688 | 0.9600 | 0.9782 | 0.9687
Qwen | 0.7235 | 0.7140 | 0.6282 | 0.6605 | Llama2 | 0.9706 | 0.9566 | 0.9851 | 0.9705
| Average | 0.7271 | 0.7113 | 0.6376 | 0.6646 |  | Average | 0.9611 | 0.9557 | 0.9678 | 0.9611
ChatGLM | Baichuan | 0.7047 | 0.7049 | 0.5937 | 0.6352 | ChatGLM | Baichuan | 0.8703 | 0.8646 | 0.8847 | 0.8682
ChatGLM | 0.7136 | 0.6950 | 0.6187 | 0.6439 | ChatGLM | 0.9098 | 0.8677 | 0.9572 | 0.9083
GPT3.5 | 0.7106 | 0.7059 | 0.6029 | 0.6400 | GPT3.5 | 0.8813 | 0.8700 | 0.9053 | 0.8782
GPT4 | 0.7097 | 0.7064 | 0.6001 | 0.6375 | Llama3 | 0.8851 | 0.8697 | 0.9120 | 0.8825
Qwen | 0.6942 | 0.7034 | 0.5750 | 0.6233 | Llama2 | 0.9002 | 0.8735 | 0.9359 | 0.8984
| Average | 0.7065 | 0.7031 | 0.5981 | 0.6360 |  | Average | 0.8893 | 0.8691 | 0.9190 | 0.8871
GPT3.5 | Baichuan | 0.7271 | 0.7162 | 0.6399 | 0.6710 | GPT3.5 | Baichuan | 0.9125 | 0.9860 | 0.8519 | 0.9113
ChatGLM | 0.7135 | 0.7113 | 0.6195 | 0.6575 | ChatGLM | 0.9098 | 0.9851 | 0.8477 | 0.9082
GPT3.5 | 0.7368 | 0.7089 | 0.6653 | 0.6796 | GPT3.5 | 0.9797 | 0.9750 | 0.9846 | 0.9797
GPT4 | 0.7430 | 0.7215 | 0.6638 | 0.6848 | Llama3 | 0.9666 | 0.9874 | 0.9479 | 0.9665
Qwen | 0.7316 | 0.7194 | 0.6440 | 0.6730 | Llama2 | 0.9551 | 0.9854 | 0.9282 | 0.9549
| Average | 0.7304 | 0.7155 | 0.6465 | 0.6732 |  | Average | 0.9447 | 0.9838 | 0.9121 | 0.9441
GPT4 | Baichuan | 0.8420 | 0.8680 | 0.8208 | 0.8392 | Llama3 | Baichuan | 0.9327 | 0.9744 | 0.8960 | 0.9324
ChatGLM | 0.8081 | 0.8543 | 0.7673 | 0.8053 | ChatGLM | 0.9535 | 0.9751 | 0.9351 | 0.9531
GPT3.5 | 0.9072 | 0.8891 | 0.9282 | 0.9068 | GPT3.5 | 0.9667 | 0.9773 | 0.9579 | 0.9666
GPT4 | 0.9276 | 0.8826 | 0.9782 | 0.9269 | Llama3 | 0.9749 | 0.9686 | 0.9814 | 0.9748
Qwen | 0.9166 | 0.8885 | 0.9513 | 0.9158 | Llama2 | 0.9765 | 0.9790 | 0.9742 | 0.9764
| Average | 0.8803 | 0.8765 | 0.8891 | 0.8788 |  | Average | 0.9608 | 0.9748 | 0.9489 | 0.9607
Qwen | Baichuan | 0.7758 | 0.9183 | 0.6629 | 0.7568 | Llama2 | Baichuan | 0.9256 | 0.9879 | 0.8717 | 0.9250
ChatGLM | 0.7170 | 0.9011 | 0.5702 | 0.6896 | ChatGLM | 0.9452 | 0.9881 | 0.9072 | 0.9449
GPT3.5 | 0.8695 | 0.9446 | 0.8076 | 0.8658 | GPT3.5 | 0.9685 | 0.9878 | 0.9507 | 0.9684
GPT4 | 0.9589 | 0.9604 | 0.9582 | 0.9588 | Llama3 | 0.9775 | 0.9892 | 0.9666 | 0.9775
Qwen | 0.9662 | 0.9498 | 0.9836 | 0.9662 | Llama2 | 0.9811 | 0.9798 | 0.9826 | 0.9811
| Average | 0.8575 | 0.9348 | 0.7965 | 0.8474 |  | Average | 0.9596 | 0.9865 | 0.9358 | 0.9594