Setting up scheduled crawls

FireScraper lets you schedule recurring crawls so your data stays fresh without manual intervention. Schedules run automatically at the frequency you choose and use the same settings as your saved project templates.

Creating a schedule

There are two ways to create a schedule:

From the New Project dialog

When setting up a new project, click Schedule for later instead of Start project. Choose a frequency (daily, weekly, or monthly) and confirm. The schedule is created and your first run will start at the next scheduled time.

From the Schedules page

Go to the Schedules page from the sidebar. Use the "New Schedule" form to select an existing project template and a frequency, then click Create schedule.

Frequency options

| Frequency | Runs at |

|-----------|---------|

| Daily | Once every 24 hours |

| Weekly | Once every 7 days |

| Monthly | Once every 30 days |

Schedules run automatically in the background. Each run creates a new session that appears in your dashboard like any other crawl.

Managing schedules

From the Schedules page you can:

Pause a schedule to temporarily stop it from running

Enable a paused schedule to resume automatic runs

Run now to trigger an immediate crawl outside the normal schedule

View run history to see all sessions created by the schedule

How credits work with schedules

Each scheduled run uses credits just like a manual crawl. The number of credits depends on how many pages are crawled. If your account runs out of credits, scheduled runs will fail until you add more.

Tips

Start with weekly if you're monitoring content that doesn't change daily — it saves credits

Use "Run now" to test a schedule before waiting for the next automatic run

Pause schedules for sites you're no longer monitoring to avoid wasting credits