Niansuh commited on
Commit
4ba78a3
Β·
verified Β·
1 Parent(s): 81d22d0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -146
README.md CHANGED
@@ -1,146 +1,8 @@
1
- # ⭐️ A proxy scraper made using Protractor | Proxy list - Updates every 3 hours πŸ”₯
2
-
3
- [![Every 10 Minutes Update](https://github.com/sunny9577/proxy-scraper/actions/workflows/nodejs.yml/badge.svg?branch=master)](https://github.com/sunny9577/proxy-scraper/actions/workflows/nodejs.yml)
4
- ![GitHub](https://img.shields.io/github/license/sunny9577/proxy-scraper)
5
- ![GitHub last commit](https://img.shields.io/github/last-commit/sunny9577/proxy-scraper)
6
- [![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/sunny9577/proxy-scraper/graphs/commit-activity)
7
- [![Website sunny9577.github.io](https://img.shields.io/website-up-down-green-red/http/github.io.svg)](https://sunny9577.github.io/)
8
- [![made-with-nodejs](https://img.shields.io/badge/Made%20with-Nodejs-green.svg)](https://www.nodejs.org/)
9
-
10
- <!-- dynamic-count-start -->
11
- ## Current Proxy Count: 2800+ πŸš€
12
- <!-- dynamic-count-end -->
13
-
14
-
15
- ## Quick Access - By Format
16
-
17
-
18
- TXT
19
-
20
- curl "https://sunny9577.github.io/proxy-scraper/proxies.txt" > proxies.txt
21
-
22
- JSON
23
-
24
- curl "https://sunny9577.github.io/proxy-scraper/proxies.json" > proxies.json
25
-
26
- CSV
27
-
28
- curl "https://sunny9577.github.io/proxy-scraper/proxies.csv" > proxies.csv
29
-
30
- YAML
31
-
32
-
33
- curl "https://sunny9577.github.io/proxy-scraper/proxies.yaml" > proxies.yaml
34
-
35
- XML
36
-
37
-
38
- curl "https://sunny9577.github.io/proxy-scraper/proxies.xml" > proxies.xml
39
-
40
- ## Quick Access - By Type
41
- ###### Supported formats: txt | json | csv | yaml | xml
42
-
43
- SOCKS5
44
-
45
- curl "https://sunny9577.github.io/proxy-scraper/generated/socks5_proxies.txt" > socks5_proxies.txt
46
-
47
-
48
- SOCKS4
49
-
50
- curl "https://sunny9577.github.io/proxy-scraper/generated/socks4_proxies.txt" > socks4_proxies.txt
51
-
52
- HTTP(S)
53
-
54
- curl "https://sunny9577.github.io/proxy-scraper/generated/http_proxies.txt" > http_proxies.txt
55
-
56
-
57
- ## Setup Requirements
58
- 1 - A Windows, Mac or Linux Computer
59
-
60
- 2 - Google Chrome installed
61
-
62
- 3 - Nodejs and NPM installed
63
-
64
- ## Setup
65
- This bot is based on [protractor](https://www.protractortest.org/#/) automated browser testing software.
66
- ### 1 - Open a terminal and make sure you are in the project directory
67
- cd proxy-scraper
68
-
69
- ### 2 - Make sure you have protractor installed
70
- To check if you have protractor installed run:
71
-
72
- protractor --version
73
-
74
- If the output is something like this:
75
-
76
- Version 7.0.0
77
-
78
- This means protractor is already installed, otherwise run the command:
79
-
80
- npm install -g protractor
81
-
82
- ### 3 - Install Selenium Webdriver
83
- This is where your program will run, to install it simply execute the command:
84
-
85
- webdriver-manager
86
-
87
- ### 4 - Database Connectivity ( Only if u want to save output to database)
88
- You can skip this step just set SAVE_TO_DB to false in config.js
89
-
90
- Go to the file config.js and change with your login information
91
-
92
- npm install mysql2
93
-
94
- Table Structure (Table Name: proxies_tb)
95
-
96
- |Name|Type|Collation|Attributes|Null|Default|Comments|Extra|
97
- |----|----|---------|----------|----|-------|--------|-----|
98
- |1 |idPrimary |int(11) | |No |None |AUTO_INCREMENT
99
- |2 |proxyIndex |varchar(50) |latin1_swedish_ci |No |None
100
- |3 |port |int(10) | |No |None
101
- |4 |country |varchar(100) |latin1_swedish_ci |No |None
102
- |5 |type |varchar(100) |latin1_swedish_ci |No |None
103
- |6 |anonymity |varchar(100) |latin1_swedish_ci |No |None
104
-
105
- ## Running the bot
106
-
107
- ### 1 - Update the Selenium webserver and install dependencies
108
- Open a terminal and make sure you are in the project directory:
109
-
110
- cd proxy-scraper
111
-
112
- Update webdrivers:
113
-
114
- webdriver-manager update
115
-
116
- Install dependencies:
117
-
118
- npm install
119
-
120
- ### 2 - Run the bot
121
- Its time to start retrieving the proxies, make sure you are in the project directory:
122
-
123
- cd proxy-scraper
124
-
125
- Run the bot:
126
-
127
- protractor conf.js
128
-
129
- ## Additional Tools
130
-
131
- - #### socks5_test.py
132
-
133
- This python script tests the SOCKS5 proxies against https://icanhazip.com and https://eset.com. Contributed by [@techroy23](https://www.github.com/techroy23)
134
-
135
- Steps to run:
136
- - pip3 install tabulate pysocks
137
- - python3 socks5_test.py
138
-
139
- ## Disclaimer
140
- This repository is intended solely for educational purposes. I neither endorse nor encourage any illegal activities or behavior.
141
-
142
- ## License
143
-
144
- [MIT](LICENSE)
145
-
146
- [![Stargazers over time](https://starchart.cc/sunny9577/proxy-scraper.svg)](https://starchart.cc/Naereen/badges)
 
1
+ ---
2
+ license: mit
3
+ title: Proxy-Scraper
4
+ sdk: static
5
+ emoji: 🌍
6
+ colorFrom: green
7
+ colorTo: yellow
8
+ ---