Give Dify workflows verifiable web inputs
Request authorized pages through an HTTP Request node and validate access before cleaning, summarizing or knowledge ingestion. The API returns the response; the Dify workflow handles subsequent content processing.
- Separate HTTP access from content processing
- Use Secret environment variables for credentials
- Verify content and sources even after successful requests
For authorized web access, development, testing and data workflows.
Choose the integration that fits your project
Start with the smallest usable path, then add tools and workflow complexity as needed.
Call the public API
Use your existing HTTP client and control requests, sessions and result handling. Available today.
API documentationOrganize tasks with a Skill
Give the agent access rules, then prepare parameters, check status and handle failures by task.
Cloudbypass SkillConnect through MCP
Existing HTTP tools need custom headers and secure credential injection. The hosted service is still planned.
Hosted MCPSet up, request and validate
Prove the workflow with one inspectable request before adding complex configuration.
Define inputs and allowed targets
Set the target URL and output fields. Validate hosts and paths in a trusted workflow. Do not allow arbitrary internal-network or unauthorized targets.
Configure the HTTP Request node
Use the API URL, GET and x-cb-host. Bind a Secret environment variable to x-cb-apikey. Configure versions, proxies and timeouts using the API documentation.
Branch on the actual response
Inspect status_code, headers and body before passing valid content to cleaning or LLM nodes. Route errors to a failure branch. Knowledge ingestion requires its own configured workflow.
Start by validating one HTTP node
Validate the minimal request in a trusted runtime, then configure the workflow node. The configuration below is a checklist, not an importable workflow.
curl --include --request GET \
"https://api.cloudbypass.com/" \
--header "x-cb-host: example.com" \
--header "x-cb-apikey: ${CLOUDBYPASS_APIKEY:?}" \
--max-time 90
# POSIX shell example; set the key securely first.
# Check x-cb-status, target status and content.
# For V2 / V2s, add version and stable proxy parameters.Method: GET
URL: https://api.cloudbypass.com/
x-cb-host: example.com
x-cb-apikey: <Secret environment variable>
Inspect response headers, HTTP status and body.
Pass only validated content downstream; branch on errors.
Configure timeouts, retries and concurrency in the actual workflow.Dify Secret variables and downstream processing
A successful HTTP request does not mean content has been cleaned or added to a knowledge base. Masked Secret values do not replace log review. Recheck exports, debug output and model context for credentials.
| Check | What to inspect | Next action |
|---|---|---|
| Credentials and scope | Check credentials, balance and authorized access | Fix configuration instead of blindly changing IPs |
| Version and session | Match stable proxies and sessions for V2 / V2s | Rebuild the verification session after a proxy change |
| Two status layers | Check x-cb-status and the target HTTP status | A 200 response can still be a login or challenge page |
| Output and retries | Check content type, expected fields and attempt budget | Stop on configuration errors; limit transient retries |
- Integration guide Shared parameters and access rules
- Official client documentation Configure for your client and version
- Contact support Share a request ID and redacted errors
Put your integration to work
Choose a task guide for access strategies, implementation steps and troubleshooting.
Planning and troubleshooting
- Error diagnostics
Separate configuration, access and target-response failures.
- API and proxy cost calculator
Estimate request credits and proxy bandwidth.
Development and tool integration
- AI agent web access
Prepare web inputs for research assistants and agents.
Business use cases
- Web ingestion for RAG
Pass validated content to cleaning and ingestion pipelines.
- Competitor and price monitoring
Track prices, availability and public page changes.
- OSINT and media monitoring
Keep sources and access records for public-information research.
Frequently asked questions
Which integration should I use today?
Start with the public HTTP API. OpenClaw can use the published Skill. Existing HTTP MCP tools must support custom headers and secure credentials. The hosted MCP service is still planned.
Can every target be accessed?
No universal guarantee is made. Relevant access capabilities include Cloudflare JavaScript Challenge, Turnstile, DataDome and Imperva / Incapsula, subject to target and runtime validation. This integration does not advertise reCAPTCHA or hCaptcha support or replace account permissions.
Where should API keys and proxy passwords go?
Keep them in secure configuration in the actual runtime or a trusted server. Do not put them in browser code, shared repositories, prompts, screenshots or public logs. Remote services have separate environments.
Do MCP or Skills automatically handle pagination and extraction?
Not by default. The access layer handles requests and responses; workflows or site recipes handle pagination, field extraction, deduplication, login state and business-specific signatures. Diagnose failures instead of retrying indefinitely.
Validate one request before scaling up
Confirm target compatibility, then bring the workflow into your project.