Compliance and ethical web scraping guidelines

Proxies make large-scale data collection possible, which also means it's worth being deliberate about doing it responsibly. None of this is legal advice — it's a practical starting point.

Respect the basics

• Check robots.txt before crawling and honor disallowed paths where practical
• Keep request rates reasonable — hammering a small site's infrastructure helps no one, including you
• Identify your traffic with an honest User-Agent where a project doesn't specifically require anonymity
• Review a site's terms of service for anything specifically prohibiting automated access

Handling personal data

If a project touches personal data — names, emails, profile information — treat it with the same care you'd want applied to your own. Only collect what you actually need, store it securely, and be mindful of regulations like GDPR if any of your subjects are in scope.

How our IPs are sourced

Every residential IP in our network comes from users who opted in through InsideProxy's own SDK — nobody's connection is included without consent. That's a baseline we hold on our side; how you use the network downstream is still on you.