介绍 /map
从单一URL到整个网站的地图的最简单方法。这在以下情况下非常有用：- 当您需要提示最终用户选择要抓取的链接时
- 需要快速了解网站上的链接时
- 需要抓取与特定主题相关的网站页面时（使用
search参数） - 只需要抓取网站的特定页面时
Alpha考虑事项
这个端点优先考虑速度，因此可能无法捕获所有网站链接。我们正在改进中。非常欢迎反馈和建议。映射
/map端点
用于映射URL并获取网站的URL。这将返回网站上存在的大多数链接。安装
pip install firecrawl-py
npm install @mendable/firecrawl-js
go get github.com/mendableai/firecrawl-go
# 将以下内容添加到您的Cargo.toml中
[dependencies]
firecrawl = "^1.0"
tokio = { version = "^1", features = ["full"] }
使用
from firecrawl import FirecrawlApp
app = FirecrawlApp(api_key="fc-YOUR_API_KEY")
# 映射一个网站:
map_result = app.map_url('https://firecrawl.dev')
print(map_result)
import FirecrawlApp, { MapResponse } from '@mendable/firecrawl-js';
const app = new FirecrawlApp({apiKey: "fc-YOUR_API_KEY"});
const mapResult = await app.mapUrl('https://firecrawl.dev') as MapResponse;
if (!mapResult.success) {
throw new Error(`Failed to map: ${mapResult.error}`)
}
console.log(mapResult)
import (
"fmt"
"log"
"github.com/mendableai/firecrawl-go"
)
func main() {
// 使用你的API密钥初始化FirecrawlApp
apiKey := "fc-YOUR_API_KEY"
apiUrl := "https://api.firecrawl.dev"
version := "v1"
app, err := firecrawl.NewFirecrawlApp(apiKey, apiUrl, version)
if err != nil {
log.Fatalf("无法初始化FirecrawlApp: %v", err)
}
// 映射一个网站
mapResult, err := app.MapUrl("https://firecrawl.dev", nil)
if err != nil {
log.Fatalf("无法映射URL: %v", err)
}
fmt.Println(mapResult)
}
use firecrawl::FirecrawlApp;
#[tokio::main]
async fn main() {
// 使用API密钥初始化FirecrawlApp
let app = FirecrawlApp::new("fc-YOUR_API_KEY").expect("Failed to initialize FirecrawlApp");
let map_result = app.map_url("https://firecrawl.dev", None).await;
match map_result {
Ok(data) => println!("Mapped URLs: {:#?}", data),
Err(e) => eprintln!("Map failed: {}", e),
}
}
curl -X POST https://api.firecrawl.dev/v1/map \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer YOUR_API_KEY' \
-d '{
"url": "https://firecrawl.dev"
}'
响应
SDK将直接返回数据对象。cURL将返回如下所示的有效载荷。{
"status": "success",
"links": [
"https://firecrawl.dev",
"https://www.firecrawl.dev/pricing",
"https://www.firecrawl.dev/blog",
"https://www.firecrawl.dev/playground",
"https://www.firecrawl.dev/smart-crawl",
...
]
}
带搜索的映射
带search 参数的映射允许您在网站内搜索特定URL。
cURL
curl -X POST https://api.firecrawl.dev/v1/map \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer YOUR_API_KEY' \
-d '{
"url": "https://firecrawl.dev",
"search": "docs"
}'
{
"status": "success",
"links": [
"https://docs.firecrawl.dev",
"https://docs.firecrawl.dev/sdks/python",
"https://docs.firecrawl.dev/learn/rag-llama3",
]
}