External Integrations Section - FireCrawl

Basics & Overview

1. What is the External Integrations Section - FireCrawl Block used for?

The FireCrawl Block allows your chatbot to interact with the FireCrawl API to scrape, map, crawl, and search websites. It's a powerful tool for gathering data from the web.

2. How does the FireCrawl Block help my business?

This block automates the process of fetching information from websites. Instead of manually visiting pages, your chatbot can retrieve content, which is useful for tasks like competitive analysis, content aggregation, or building internal knowledge bases.

3. What can I do with the FireCrawl Block?

The block offers four main actions, which you select using the "Select Action" field:

• Scrape: Retrieves the content of a specific web page.

• Map: Generates a sitemap of a website.

• Crawl: Follows links on a website to gather content from multiple pages.

• Search: Searches the web for information based on a query.

4. What do I need to get started with the FireCrawl Block?

You need to have a FireCrawl account that has been integrated with your chatbot platform. This gives the bot the necessary permissions to access FireCrawl's features.

5. How do I choose the action I want the bot to perform using the FireCrawl Block?

You use the "Select Action" field to choose from the four available options based on your goal.

Scraping & Crawling

6. What is the difference between "Scrape" and "Crawl" actions in the FireCrawl Block?

Scrape is for getting the content of a single, specific URL. Crawl is for following links from a starting URL to gather content from multiple pages on a website.

7. How do I scrape a web page using the FireCrawl Block?

Choose the "Scrape" action, then enter the URL of the web page you want to get the content from into the "Enter a URL" field.

8. What kind of data does the "Scrape" action from the FireCrawl Block return?

The response typically includes the clean, main content of the web page in a structured format, without unnecessary elements like sidebars or headers.

9. How do I crawl a website using the FireCrawl Block?

Choose the "Crawl" action, then enter the starting URL into the "Enter a URL" field. The block will then follow links to collect content from other pages on that site.

10. What kind of data does the "Crawl" action from the FireCrawl Block return?

The response will include the scraped content from the initial URL and from all the other pages it found while crawling.

11. What is the "Map" action in the FireCrawl Block used for?

The "Map" action generates a sitemap of a website, providing a list of all the URLs on that site that FireCrawl can find.

12. How is the "Map" action different from the "Crawl" action in the FireCrawl Block?

The "Map" action only provides a list of URLs, whereas the "Crawl" action retrieves the actual content from those URLs.

Searching the Web

13. What is the "Search" action in the FireCrawl Block for?

The "Search" action allows your chatbot to perform a search query on the web. It's similar to a search engine but returns the results in a structured format for your chatbot to use.

14. How do I perform a search using the FireCrawl Block?

Choose the "Search" action, then enter your search query into the "Enter a Query" field.

15. What kind of data does the "Search" action from the FireCrawl Block return?

The response includes a list of search results, with each result typically containing a title, a URL, and a short snippet or description of the page's content.

General Configuration & Use Cases

16. What does the "Save Response in Variable" field in the FireCrawl Block do?

This field saves the data retrieved by the block into a variable. This is crucial because it allows you to use the scraped or crawled data in other parts of your chatbot flow, such as in a Text block or an AI block.

17. How can I use a variable for the URL or Query in the FireCrawl Block?

You can use a chatbot flow to ask the user for a URL or a search query. You store their response in a variable (e.g., {{user_url}}) and then use that variable in the "Enter a URL" or "Enter a Query" field.

18. How can the FireCrawl Block help with competitive analysis?

You can use the "Crawl" action to regularly collect data from your competitors' websites to track their products or content changes.

19. How can I use the FireCrawl Block to provide a dynamic knowledge base?

You can use the "Scrape" or "Crawl" actions to fetch information from your company's support pages or documentation, allowing your chatbot to answer customer questions with up-to-date information.

20. Can I use the FireCrawl Block to collect product data?

Yes. You can use the "Scrape" action on product pages to retrieve details like price, description, and specifications, which can then be used in your chatbot to answer customer questions.

Location & Tutorials

21. Where can I find the FireCrawl Block in the builder?

You can find the FireCrawl Block in the External Integrations section of your chatbot builder.

22. Where can I find a video tutorial for the FireCrawl Block?

You can find the video tutorial at:   https://chatmaxima.com/video-tutorials/integrating-the-firecrawl-block-in-chatmaxima/

Troubleshooting & Other Questions

23. What happens if the URL provided to the FireCrawl Block is wrong?

If you enter an incorrect URL, the block will fail to scrape or crawl the website. The response variable will likely contain an error message, which you should plan for in your chatbot flow.

24. What is the difference between the FireCrawl Block and a simple webhook?

This block is specifically pre-configured to handle the FireCrawl API's actions (scrape, crawl, map, search). A generic webhook block would require you to manually set up the entire request, including the headers and body.

25. How quickly does the FireCrawl Block work?

The speed depends on the action you've chosen and the size of the website. Scraping a single page is very fast, while crawling a large website can take more time.