curl --request POST \
--url https://api.open.cx/crawl \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"url": "<string>",
"display_name": "<string>",
"page_limit": 100,
"exclude_paths": [],
"include_paths": [],
"crawl_interval_hours": 168,
"auto_start_crawl": true
}
'{
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"url": "<string>",
"display_name": "<string>",
"status": "<string>",
"page_limit": 123,
"exclude_paths": [
"<string>"
],
"include_paths": [
"<string>"
],
"crawl_interval_hours": 123,
"last_crawl_started_at": "2023-11-07T05:31:56Z",
"last_crawl_completed_at": "2023-11-07T05:31:56Z",
"error_message": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"updated_at": "2023-11-07T05:31:56Z"
}Create a new website datasource and optionally start an initial crawl. Crawled content is automatically indexed into your knowledge base.
curl --request POST \
--url https://api.open.cx/crawl \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"url": "<string>",
"display_name": "<string>",
"page_limit": 100,
"exclude_paths": [],
"include_paths": [],
"crawl_interval_hours": 168,
"auto_start_crawl": true
}
'{
"id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"url": "<string>",
"display_name": "<string>",
"status": "<string>",
"page_limit": 123,
"exclude_paths": [
"<string>"
],
"include_paths": [
"<string>"
],
"crawl_interval_hours": 123,
"last_crawl_started_at": "2023-11-07T05:31:56Z",
"last_crawl_completed_at": "2023-11-07T05:31:56Z",
"error_message": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"updated_at": "2023-11-07T05:31:56Z"
}Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Default Response
^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$Was this page helpful?