This guide demonstrates how to add web content to a Knowledge Graph using web connector URLs. Adding web connector URLs to your Knowledge Graph allows you to automatically extract and index content from websites, making it available for querying in your Knowledge Graph.
You need an API key to access the Writer API. Get an API key by following the steps in the API quickstart.We recommend setting the API key as an environment variable in a .env file with the name WRITER_API_KEY.

​
Overview

Adding URLs to your Knowledge Graph enables you to:
  • Automatically extract content from websites
  • Index web pages for Knowledge Graph queries
  • Process single pages or entire sub-pages
  • Monitor the status of URL processing
  • Exclude specific URLs from processing

​
Add URLs to a Knowledge Graph

To add web connector URLs to your Knowledge Graph, use the PUT endpoint to update the graph with the URLs you want to process. This endpoint allows you to add, update, or remove URLs from your Knowledge Graph.
You can only add URLs to a Knowledge Graph with the type set to web. To create a Knowledge Graph with the web type, you must create the Knowledge Graph in AI Studio and select “Add Website” as the way you want to add content to the Knowledge Graph. See Setting up Knowledge Graph web connectors for more information.You can see the type of a Knowledge Graph in the Knowledge Graph details or in the Knowledge Graph list endpoints.
cURL
curl --location --request PUT 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "urls": [
            {
                "url": "https://example.com/documentation",
                "type": "sub_pages",
                "exclude_urls": [
                    "https://example.com/documentation/admin",
                    "https://example.com/documentation/private"
                ]
            }
        ]
    }'

​
Request structure

The urls array in your request replaces the entire existing list of URLs attached to the Knowledge Graph. To add a new URL, include both the existing URLs and the new one in your request. To remove a URL, send a request with all URLs except the one you want to remove.
When updating a Knowledge Graph with URLs, you need to provide a urls array in your request body. Each URL object has the following parameters:
ParameterTypeDescription
urlstringThe URL you want to add to your Knowledge Graph
typestringHow you want to process the URL: either single_page (just this URL) or sub_pages (this URL and linked pages)
exclude_urlsarrayOptional. Any URLs you want to exclude from processing if type is sub_pages

​
Response structure

After you update the Knowledge Graph, you see the status of each URL in the status object. The status object has the following parameters:
ParameterTypeDescription
statusstringThe current processing status: validating, success, or error
error_typestringIf there was an error, what type of error occurred
{
  "id": "51b058d8",
  "created_at": "2025-07-16T20:19:23.565884Z",
  "name": "Public docs",
  "description": "Public docs",
  "urls": [
    {
      "url": "https://example.com/docs",
      "status": {
        "status": "validating",
        "error_type": null
      },
      "exclude_urls": [
        "https://example.com/docs/admin",
        "https://example.com/docs/private"
      ],
      "type": "sub_pages"
    }
  ]
}

​
Possible error types

If URL processing fails, you may see these error types:
  • invalid_url: The URL format is not valid
  • not_searchable: The content cannot be accessed or processed
  • not_found: The URL returns a 404 error
  • paywall_or_login_page: The content requires authentication
  • unexpected_error: An unspecified error occurred

​
Usage examples

The following examples show how to add a web connector URL to a Knowledge Graph and monitor its processing status.

​
Add a web connector URL

This example adds a web connector URL to a Knowledge Graph. It also excludes two sub-page URLs from processing. 
curl --location --request PUT 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "urls": [
            {
                "url": "https://example.com/documentation",
                "type": "sub_pages",
                "exclude_urls": [
                    "https://example.com/documentation/admin",
                    "https://example.com/documentation/private"
                ]
            }
        ]
    }'

​
Monitor processing status

You can check the status of your web connector URLs by retrieving the Knowledge Graph details. 
curl --location 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header "Authorization: Bearer $WRITER_API_KEY"

​
Query the Knowledge Graph with web content

Once the web connector URLs have been processed successfully, you can query the Knowledge Graph to access the extracted content. 
curl --location 'https://api.writer.com/v1/graphs/{graph_id}/query' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "query": "What are the main features described in the documentation?",
        "subqueries": true
    }'

​
Next steps

By following this guide, you can successfully add web content to your Knowledge Graphs using web connector URLs. Next, learn about other Knowledge Graph capabilities: