Build Your Own Web Crawler With Manus (Runs in the Cloud)

In this guide, you will learn how to turn Manus Cloud Computer into a private 24/7 web crawler for any site you want to monitor.

  • Beginner
  • 6 steps
  • By The Rundown University

In this guide, you will learn how to turn Manus (https://manus.im) Cloud Computer into a private 24/7 web crawler for any site you want to monitor. Instead of having Manus burn tokens checking the same site each day, your cloud computer does the checking for cheap.

Step by step 6 steps

  1. Step 1 of 6

    Create the Cloud Computer

    Open the Manus desktop app, go to Settings > My Computer, and click Create now under Cloud Computer.

    create cloud computer card
    Create Cloud Computer entry card

    This gives Manus an always-on server where it can keep scripts, logs, cron jobs, and output files.

  2. Step 2 of 6

    Pick the Smallest Plan

    For web crawlers, API checks, cron jobs, CSVs, and logs, start with the $10 Basic plan.

    cloud computer plan options
    Cloud Computer plan options

    Use bigger plans for local models or video. For a crawler that checks a site twice a day, the Basic plan is the right default.

    Pro tip: Start small. You can always upgrade later if the job actually needs more compute.

  3. Step 3 of 6

    Start a Task on the Cloud Computer

    Start a new Manus task and select your Cloud Computer in the composer.

    cloud computer task composer
    Manus task composer with Cloud Computer selected

    If you skip this, Manus may build in a temporary workspace instead of your new Cloud Computer.

    Pro tip: Any time the job needs to keep running after your laptop closes, select the Cloud Computer before you prompt.

  4. Step 4 of 6

    Paste the Crawler Prompt

    Now give Manus the site, the specific information you care about, and what you want it to do with matching results.

    Set up a web crawler on my Cloud Computer for [website URL].
    First inspect the site formatting and tell me the best way to check it reliably.
    Check for [specific info] twice a day.
    Log it to a csv.
    Create a small script, schedule it with cron, log every run, and only send matching results to Manus for [output].

    The most important line is the inspection step. Before Manus writes code, it should check the site's formatting and decide the best way to read it reliably.

    Pro tip: Do not ask Manus to build first. Ask it to inspect first. That is how you avoid brittle automations.

  5. Step 5 of 6

    Run It Once Before Turning It On

    Have Manus run the script once before turning it on 24/7.

    Use this:

    Run the script once before turning it on 24/7.
    
    Show me:
    - what path you used to read the site
    - what files you created
    - the CSV you wrote
    - the latest run log
    - what Manus would receive if there were a match

    For the recording, we used Grants.gov as the test site:

    grants search csv export
    Grants.gov search results with CSV export

    That page works well because the results are public and there is a CSV export. But the same pattern works for job posts, pricing pages, competitor updates, policy pages, changelogs, and feeds.

    Pro tip: If the test run is messy, tighten the prompt before you schedule it. It is easier to fix the format once than clean up bad logs later.

  6. Step 6 of 6

    Turn It On 24/7

    Once the test run looks right, have Manus schedule the script with cron.

    Use this:

    Schedule this to run twice a day with cron.
    
    After you schedule it, show me:
    - the cron entry
    - the latest run log
    - where matching results are saved
    - where errors are logged

    Check the log after the first scheduled run. A useful log should show the date, the page checked, what it found, and whether Manus received anything.

    Once it is running, you can ask Manus to check the collected data anytime from your phone or laptop. The crawler keeps doing the boring part in the background.

That's the whole workflow

Mark it complete to keep your progress. The background follows.

Keep learning with Rundown Pro

Guides stay free. This is what Rundown Pro adds:

  • Every course and recording
  • The Rundown Certified Operator program
  • Member perks

$29/mo, cancel anytime

Background and context

Who This Is Useful For

  • AI users who want a simple private automation server without touching AWS.
  • Researchers and operators who check job boards, RFP pages, public sites, feeds, or changelogs.
  • Manus users who want to stop spending tokens on “nothing changed” checks.
  • Founders and consultants who want lightweight monitoring without buying another SaaS tool.

What You Will Build

You will set up a Cloud Computer with a web crawler that checks a site twice a day.

The crawler will:

  • check [website URL]
  • look for [specific information]
  • log the results to a CSV
  • run on a schedule with cron

For the demo, we used a Grants.gov search page because it has public data and a CSV export. You can swap in almost any public page, feed, or site you already check manually.

What You Need

  • A Manus (https://manus.im) account
  • The Manus desktop app
  • A site you want to monitor

Do not put API keys or passwords in your prompt. If your script needs credentials later, ask Manus to store them in an .env file on the Cloud Computer.

Going Further

Start with a Manus skill, Claude skill, or any AI workflow you already use.

Then ask Manus:

Look at this workflow and tell me which repetitive parts can run "for free" on my Cloud Computer before the AI agent gets involved.

Separate it into:
- cheap checks the cloud computer can run
- criteria that should wake Manus
- judgment-heavy work the agent should still do
- logs or files that prove it ran

That is the real pattern. Your cloud computer handles the repetitive work. The agent handles the judgment.