smartproxy

Get Started With Web Scraping API

Collect raw HTML data in bulk without worrying about anti-scraping restrictions, CAPTCHAs, or proxy management. Learn how to use this scraping solution with our quick start guide.

Any country
100% success rate
Results in HTML
Headless scraping
Synchronous or asynchronous requests
Real-time integration
Proxy-like integration
Free 7-day trial
14-day money-back option
Get Started With Web Scraping API
smartproxy

What is Web Scraping API?

Our Web Scraping API provides a versatile data collection solution for websites not covered by our other scraping services. It leverages a vast pool of proxies alongside an advanced web scraper to retrieve raw HTML data.

Most popular Web Scraping API use cases

Choose your target and gather data with confidence using our state-of-the-art scraper.

Public data collection

Public data collection

Effortlessly gather real-time public data for analysis or easy integration into your projects.

Travel fare aggregation

Travel fare aggregation

Collect relevant pricing data to get the best deals for hotels, flights, and car rentals.

Fraud protection

Fraud protection

Detect malicious websites and prevent fraud from global locations by scanning scraped HTML data.

How to set up Web Scraping API

Get started with your Web Scraping API configuration by following our guide.

smartproxy

Choose a subscription

After creating your account, find the Web section under the Scraping side menu.

Next, click the Pricing tab and select a subscription plan that suits your needs. Alternatively, start a free trial to get 1K requests for 7 days.

smartproxy

Authenticate with username:password

In the API Authentication tab, click on either the username or password to copy it. Use the buttons on the right to reveal your password or generate a new one.

smartproxy

Create a request command

Go to the Scraper tab, where you can enter the target URL, choose your proxy language (locale), location, device type, and browser. Find more parameter options in our help documentation.

Below, you can send your request directly or select the Request tab and copy the code in cURL, Node.js, or Python format for use in your own environment.

smartproxy

Copy or download the result in HTML

After sending a request via the Scraper features on the dashboard, you’ll receive the raw HTML response presented within a JSON structure. Click on the corresponding buttons located at the top of the Response screen to copy or download this data.

smartproxy

Track scraping usage

To track your scraping API usage, navigate to the Statistics tab. There, you’ll find traffic usage details for each user over a selected timeframe, along with cumulative data on uploads, downloads, and requests.

Easy to integrate scraper

Our Web Scraping API works with all popular programming languages, ensuring a smooth connection to other tools in your business suite.

import requests
url = "https://scraper-api.smartproxy.com/v2/scrape"
payload = {
"target": "universal",
"url": "https://ip.smartproxy.com/",
"headless": "html"
}
headers = {
"accept": "application/json",
"content-type": "application/json",
"authorization": "Basic [your basic authentication token]"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)

Free tools, same great user-friendliness

smartproxy

X Browser

Juggling multiple profiles has never been easier. Get unique fingerprints and use as many browsers as you need, risk-free!

smartproxy

Chrome Browser Extension

Easy-to-use, damn powerful. A proxy wonderland in your browser, accessible in 2 clicks. Free of charge.

smartproxy

Firefox Browser Add-on

Easy to set up, even easier to use. The virtual world at your fingertips in 2 clicks. Free of charge.