3223 IT & Software Developer jobs in the UK
Python Web Scraping Engineer
£50,000 - 70,000
Guildhall Yard, City of London + Remote
£50,000 - 70,000
Requirements
Must:
- Strong professional experience with Python, including production-grade scraping systems
- Deep, hands-on experience scraping Google or similarly protected platforms
- Solid understanding of HTTP, TLS, cookies, headers, redirects, and browser networking behaviour
- Experience with browser automation frameworks such as Playwright, Selenium, Puppeteer, or equivalents
- Strong knowledge of HTML parsing, DOM traversal, and data extraction techniques
- Proven experience handling rate limiting, CAPTCHAs, IP rotation, and bot detection systems
- Comfortable designing and operating asynchronous and concurrent scraping architectures
- Experience running scraping systems at scale in cloud environments
- Excellent debugging skills and the ability to reason about complex, non-obvious failure modes
- Hands-on experience with headless browser fingerprinting and anti-bot evasion techniques
Responsibilities
- Design, build, and maintain large-scale Python scraping systems that operate reliably against Google and other highly defended websites
- Develop scrapers for dynamic, JavaScript-heavy pages using browser automation and hybrid scraping approaches
- Continuously adapt to changes in page structure, request flows, and anti-automation mechanisms
- Engineer robust extraction pipelines that prioritize data accuracy, consistency, and observability
- Implement and optimize proxy, fingerprinting, and request-routing strategies to maximize stability and throughput
- Monitor scrape health, detect failures early, and debug production issues quickly and methodically
- Optimize performance, cost, and latency across scraping infrastructure
- Collaborate closely with data engineers, data scientists, and product teams to ensure scraped data is usable and trusted
- Document scraping logic, architectural decisions, and operational runbooks for long-term maintainability
Description
We are looking for a Senior Python Scraping Engineer to design, build, and operate large-scale, resilient web scraping systems, focusing primarily on Google and similar search environments. This role sits at the intersection of data engineering and systems reliability, crucial for delivering accurate and trusted data at scale. We are committed to adopting AI-assisted coding workflows and seek someone eager to leverage AI tools to enhance development and further their skills in this area.
Something wrong or incorrect with this job? Tell us in the chat 💬 on the right ➡️
You can find Python Developer salaries in the UK here.
How many Python Developer jobs are in the UK?
Currently, there are 3223 Python openings. Check also: Django jobs, Flask jobs, TensorFlow jobs, Machine-Learning jobs - all with salary brackets.
Is the UK a good place for Python Developers?
The UK is one of the best countries to work as a Python Engineer. It has a vibrant startup community, growing tech hubs and, most important: lots of interesting jobs for people who work in tech.
Which companies are hiring for Python Developer jobs in the UK?
The Scouts Association, Mas & Pas Limited, OpenWorks Engineering, Independent Office for Police Conduct, Sainsbury's, Companies House, CenCom Solutions Ltd. among others, are currently hiring for Python roles in the UK.
The company with most openings is Experis as they are hiring for 91 different Python Developer jobs in the UK. They are probably quite committed to find good Python Developers.
The company with most openings is Experis as they are hiring for 91 different Python Developer jobs in the UK. They are probably quite committed to find good Python Developers.