Skip to content
NEWMarketplace and the trig8n Agent: sell what you build, and get plans and changes that are tested before you see them. See what is new

Library / Building Blocks & Utilities

Building Blocks & Utilities

Process Bulk URLs from Your Data Source in Batches

When you run it, it pulls URLs from your own data source, splits them into batches and sends HTTP requests with waits between calls.

Import it free Sign in to open it

What it does

# Manualtrigger Workflow ## Overview Automated workflow: Manualtrigger Workflow. This workflow integrates 10 different services: stickyNote, httpRequest, wait, splitOut, set. It contains 19 nodes and follows best practices for error handling and security. ## Workflow Details - Total Nodes: 19 - Node Types: 10 - Error Handling: ✅ Implemented - Security: ✅ Hardened (no sensitive data) - Documentation: ✅ Complete ## Node Breakdown - When clicking ‘Test workflow’: manualTrigger - Wait: wait - Sticky Note36: stickyNote - Sticky Note28: stickyNote - Connect to your own data source: noOp - Get urls from own data source: noOp - Example fields from data source: set - Sticky Note33: stickyNote - Sticky Note34: stickyNote - Sticky Note35: stickyNote - ... and 9 more nodes ## Usage Instructions 1. Configure Credentials: Set up all required API keys and credentials 2. Update Variables: Replace any placeholder values with actual data 3. Test Workflow: Run in test mode to verify functionality 4. Deploy: Activate the workflow for production use ## Security Notes - All sensitive data has been removed or replaced with placeholders - Error handling is implemented for reliability - Follow security best practices when configuring credentials ## Troubleshooting - Check error logs if workflow fails - Verify all credentials are properly configured - Ensure all required services are accessible

The steps

  1. When clicking ‘Test workflow’
  2. Wait
  3. Connect to your own data source
  4. Get urls from own data source
  5. Example fields from data source
  6. 40 items at a time
  7. 10 at a time
  8. Markdown data and Links
  9. Split out page URLs
  10. Retrieve Page Markdown and Links
  11. Error Handler
  12. Error Handler for 71c5a0d4
  13. Error Handler
  14. Error Handler

20 steps in total.

Similar automations

Building Blocks & UtilitiesProcess Bulk URLs from Your Data Source in Batches20 stepsBuilding Blocks & UtilitiesSplit Items into Batches Until None Are Left5 stepsBuilding Blocks & UtilitiesProcess Items in Batches with a Function and Condition6 stepsBuilding Blocks & UtilitiesProcess Customer Data in Batches with Error Handling7 steps