Getting Started

Authentication

The crawler.dev API uses API key authentication. You'll need to include your API key in the x-api-key header of every request.

Getting Your API Key

1. Sign up for a crawler.dev account at crawler.dev
2. Copy your API key from the API keys page
3. Keep it secure - treat your API key like a password

Using Your API Key

Include your API key in the x-api-key header:

Code
 
x-api-key: YOUR_API_KEY_HERE

Example Request

Code
 
curl -X POST https://api.crawler.dev/v1/extract/file \
  -H "x-api-key: IzyiAzLAKw8EphxGeF9xW9jPbHwNlbqz2skz5Z47" \
  -F "file=@document.pdf"

Environment Variables

Store your API key as an environment variable:

Code
 
# .env file
CRAWLER_API_KEY=your_api_key_here

Then use it in your code:

Code
 
const apiKey = process.env.CRAWLER_API_KEY;
const headers = {
  'x-api-key': apiKey,
  'Content-Type': 'multipart/form-data'
};

Error Responses

Authentication errors return HTTP 401 with details:

Code
 
{
  "error": {
    "code": "unauthorized",
    "message": "Invalid API key provided",
    "type": "authentication_error"
  }
}

Common authentication error codes:

Code	Message	Solution
missing_auth	No Authorization header provided	Add the Authorization header
invalid_key	Invalid API key format	Check your API key format
key_not_found	API key not found	Verify your API key is correct
key_disabled	API key has been disabled	Contact support or generate a new key

Managing API Keys

From your dashboard, you can:

• View all your API keys
• Generate new API keys
• Disable compromised keys
• Monitor usage per key
• Set usage alerts

Need Help?

Having trouble with authentication? Here are some common solutions:

1. Double-check your API key for typos
2. Verify you're using the correct environment (test vs live)
3. Ensure the Authorization header format is correct
4. Check that your API key hasn't expired or been disabled

If you're still having issues, please contact our support team.