List Web Crawler Jobs (POST)
Lists web crawler jobs.
All the DevRev APIs require a token to authenticate the user. Provide Authorization: Bearer <TOKEN> as a header to every API request.
How do I find my token?
In: header
Filters for web crawler jobs created by any of the provided users.
Provides ways to specify date ranges on objects.
Filters for objects created after the provided timestamp (inclusive).
date-timeFilters for objects created before the provided timestamp (inclusive).
date-timeProvides preset types for date filter.
The range starts from the current timestamp and continues for the past n days.
0 <= value <= 4294967295The range starts from the current timestamp and continues for the next n days.
0 <= value <= 4294967295The cursor to resume iteration from. If not provided, then iteration starts from the beginning.
textThe maximum number of web crawler jobs to return. The default is '50'.
int32The iteration mode to use. If "after", then entries after the provided cursor will be returned, or if no cursor is provided, then from the beginning. If "before", then entries before the provided cursor will be returned, or if no cursor is provided, then from the end. Entries will always be returned in the specified sort-by order.
"after" | "before"Provides ways to specify date ranges on objects.
Filters for objects created after the provided timestamp (inclusive).
date-timeFilters for objects created before the provided timestamp (inclusive).
date-timeProvides preset types for date filter.
The range starts from the current timestamp and continues for the past n days.
0 <= value <= 4294967295The range starts from the current timestamp and continues for the next n days.
0 <= value <= 4294967295Response Body
curl -X POST "https://api.devrev.ai/web-crawler-jobs.list" \ -H "Content-Type: application/json" \ -d '{}'{
"next_cursor": "string",
"prev_cursor": "string",
"web_crawler_jobs": [
{
"created_by": {
"type": "dev_user",
"display_id": "string",
"id": "string",
"display_name": "string",
"display_picture": {
"display_id": "string",
"id": "string",
"file": {
"type": "string",
"name": "string",
"size": 0
}
},
"email": "string",
"full_name": "string",
"state": "active"
},
"created_date": "2023-01-01T12:00:00.000Z",
"display_id": "string",
"id": "string",
"modified_by": {
"type": "dev_user",
"display_id": "string",
"id": "string",
"display_name": "string",
"display_picture": {
"display_id": "string",
"id": "string",
"file": {
"type": "string",
"name": "string",
"size": 0
}
},
"email": "string",
"full_name": "string",
"state": "active"
},
"modified_date": "2023-01-01T12:00:00.000Z",
"accept_regexs": [
"string"
],
"description": "string",
"domain_names": [
"string"
],
"frequency": 0,
"max_depth": 0,
"no_parent": true,
"notify_on_complete": true,
"num_bytes": 0,
"num_timeout_urls": 0,
"num_urls_scraped": 0,
"reject_regexs": [
"string"
],
"sitemap_index_urls": [
"string"
],
"sitemap_urls": [
"string"
],
"state": "aborted",
"urls": [
"string"
],
"user_agent": "string"
}
]
}{
"detail": "string",
"message": "string",
"type": "artifact_already_attached_to_a_parent",
"existing_parent": "string",
"is_same": true
}{
"detail": "string",
"message": "string",
"type": "unauthenticated"
}{
"detail": "string",
"message": "string",
"type": "forbidden"
}{
"detail": "string",
"message": "string",
"type": "too_many_requests",
"retry_after": 0
}{
"detail": "string",
"message": "string",
"type": "internal_error",
"reference_id": "string"
}{
"detail": "string",
"message": "string",
"type": "service_unavailable"
}