This guide outlines how to integrate Scrapingbypass API with LocoySpider using a local proxy bridge to handle complex anti-bot protections.
Step 1: Deploy the Proxy Bridge
Since LocoySpider requires a standard proxy format, you must first download the proxy translation tool to bridge the LocoySpider requests to the Scrapingbypass API.
- Download: Obtain the latest release from the Scrapingbypass Github Repository.
Step 2: Initialize the Proxy Service
Start the service and bind it to a local port. You must specify the local address and port to ensure LocoySpider can communicate with it.
- Command: Run the executable with the
-lflag../proxy-bridge -l 127.0.0.1:1087 - Result: Your local proxy endpoint is now established at
http://127.0.0.1:1087.
Step 3: Configure Proxy in LocoySpider
Map your LocoySpider task to the local bridge service.
- Open your task in LocoySpider.
- Navigate to Other Settings > Proxy Settings.
- Enter the server address:
127.0.0.1and Port:1087. - Select HTTP as the proxy type.
Step 4: Configure Target URLs and Endpoint
To leverage the bypass capabilities, you must route your requests through the Scrapingbypass API gateway.
- Go to Content Acquisition Rules.
- Modify the target URL: Replace the original domain with
https://api.scrapingbypass.com.- Example: Change
https://example.com/datatohttps://api.scrapingbypass.com/data.
- Example: Change
- Run a connection test to verify the route is active.
Step 5: Set Required HTTP Headers
You must authenticate your requests and identify the target host using custom headers.
Navigate to HTTP Request Settings and add the following custom headers:
| Header Key | Value | Description |
|---|---|---|
x-cb-apikey | YOUR_API_KEY | Your unique Scrapingbypass API key. |
x-cb-host | target-website.com | The original domain of the site you are scraping. |
Important Note
When using this configuration, ensure the proxy bridge service remains running in the background. If the service is closed, LocoySpider will fail to route traffic through the Scrapingbypass Proxy.