Browser as API

Imagine being able to scrape without having to worry about infrastructure, but with complete control.

Sounds cool, right? Welcome to Scrapy

What is scrapy?

Scrapy is a service that allows you to manage one or more web browsers in the cloud to do with them what you want.

You can control them easily through our API. From clicking on elements to intercepting network packets, you can access all the options that a local browser could offer you.

Do a lot, with little.

Shall we see some example?

1. Create a new instance of Chrome

const { browserId } = await fetch('/browsers', {
  method: 'POST',
});

2. Open a new tab and go to https://example.com .

const endpoint = '/browsers/${browserId}/pages/';
const { pageId } = await fetch(endpoint, {
  method: 'POST',
  body: {'url': 'https://example.com'}
});

3. Get some data with 'blueprints' .

const endpoint = '/browsers/${browserId}/pages/${pageId}/structured-data';
const blueprint = {
    root: "main article",
    elements: {
      title: "a.item-link",
      price: "span.item-price",
      img: { selector: "picture.item-multimedia", attribute: "src" },
    },
  };
const { results } = await fetch(endpoint, {
  method: 'POST',
  body: { blueprint}
});

/*
[
  {
    title: "element 1",
    price: 13,
    img: "https://..."
  },
  {
    title: "element 2",
    price: 24,
    img: "https://..."
  }
]
*/