Proxies for price monitoring: start with the free options

What price monitoring actually involves

Price monitoring means recording the price of a product at regular intervals, or comparing what different regions are shown. People do it to watch their own market, to check that a regional price on their own store is displaying correctly, or to decide when to buy something.

Before you reach for a proxy, decide which of these you need. Watching a few products you plan to buy is a very different job from collecting thousands of prices every hour, and the right tool is different for each.

Free options first

Try these in order. Each one is cheaper, simpler and more likely to be welcome than automated requests from a proxy.

• Official APIs and feeds. If you sell on a marketplace or run a store, the platform may offer an API or product feed with prices. Check the site's developer or partner pages. An API is the intended way to get the data.
• Price alerts built into stores and browsers. Many retailers offer a price-drop alert or a wishlist notification. Use those for the handful of items you personally want.
• Public price-tracking sites. Some third-party sites publish price history for popular products. Their coverage varies and we have not tested them, so treat what they show as a hint and confirm on the store page.
• A spreadsheet and your own eyes. For ten products, checking once a week and writing the number down is honest, free and entirely within normal use.
• Your browser's location settings for a quick look. In Chrome, DevTools has a Sensors panel that can override your geolocation (per Chrome's documentation). That changes what the page's own location code sees. It does not change your IP address, so a site that decides by IP will not be affected.

Read the rules before you automate anything

Automated checking is where problems start. Do these things first:

• Read the site's terms of service. Many stores prohibit automated access. If they do, use an official feed or do not automate.
• Read the site's robots.txt, which sits at the root of the domain. Google describes robots.txt as a way to manage crawler traffic. It is a request from the site owner, and respecting it is the polite default.
• Collect only product and price data. Do not collect personal information about people, such as reviewer names or seller contact details.
• Do not log in to get around a paywall or a members-only price, and do not try to defeat a CAPTCHA or a block. If a site is turning you away, that is the answer.
• Keep your request rate low: a few requests a minute at most for a small job, with pauses, and run it off-peak. We cannot tell you what any particular site will tolerate, and no tool changes that.

When a datacenter proxy helps

A proxy sends your requests through another computer, so the site sees that computer's IP address instead of yours. For price checks, two situations make this useful:

• You want to see a regional price or currency, for example how your own store looks to a visitor in Germany, and the site decides by IP.
• You are running a small, permitted script and want to spread requests across a few addresses so that one address is not making every request.

A datacenter proxy uses an address that belongs to a hosting company. Websites can usually tell that, and some treat such traffic differently from home traffic: they may show a CAPTCHA, a different page or no page. That is plainly how some sites behave, and you should plan for it. For a longer comparison see residential vs. datacenter proxies, and for the general picture see proxies for web scraping.

A second limit: the country a site assigns to an IP comes from a geolocation database, and databases can disagree. Check what your proxy IP looks like with the IP lookup and what is my IP tools before you trust a regional price.

If you want to try it cheaply, Webshare offers a free plan with 10 proxies, according to its own pricing page. You can look at Webshare's free plan (Affiliate link) and read our Webshare free plan guide for what we could and could not confirm.

A simple, polite setup

• Write down the exact products and pages you need. Fewer is better.
• Check for an official API, feed or alert first.
• Read the terms and robots.txt for each site. Drop any site that forbids automated access.
• Get a proxy and confirm its apparent location with the IP lookup.
• Set up your script or browser. For Python see using a proxy with Python requests; for a browser see setting up a proxy in Chrome.
• Test one page by hand, then compare with what you see in a normal browser.
• Run it slowly: one request at a time, with a delay between them, and a descriptive User-Agent if the site's rules ask for one.
• Save only the price, date and product ID. Stop if you see blocks or errors.

What can go wrong

• Different content. A site may show a CAPTCHA, an error or a simplified page to datacenter addresses. Do not try to work around it; switch to an API or reduce the job.
• Prices that depend on more than location. Currency, tax, shipping destination, logged-in status and cookies can all change what you see. A proxy approximates a visitor from a country. It does not reproduce a real shopper.
• Wrong country. IP databases are not exact, so the location you chose may not match what the store sees.
• Shared addresses. On shared proxies, other users' activity can affect how a site treats that IP.
• Account problems. Provider terms and verification rules apply to you too; read them.