Add web content to a Knowledge Graph

This guide demonstrates how to add web content to a Knowledge Graph using web connector URLs. Adding web connector URLs to your Knowledge Graph allows you to automatically extract and index content from websites, making it available for querying in your Knowledge Graph.

You need an API key to access the Writer API. Get an API key by following the steps in the API quickstart.We recommend setting the API key as an environment variable in a .env file with the name WRITER_API_KEY.

Knowledge Graphs can be deployed org-wide (accessible to every team in the organization) or deployed to specific teams. Both are accessible via the API and SDK.

With an org-scoped API key, control team scope from the request body: pass team_ids on create or update, or filter list with the team_ids query parameter. Omit team_ids on list to see only org-wide Knowledge Graphs.
With a team-scoped API key, every request is automatically restricted to the key’s team. Create assigns to that team; list and retrieve return only that team’s Knowledge Graphs; body team_ids is not accepted.

Overview

Adding URLs to your Knowledge Graph enables you to:

Automatically extract content from websites
Index web pages for Knowledge Graph queries
Process single pages or entire sub-pages
Monitor the status of URL processing
Exclude specific URLs from processing

Add URLs to a Knowledge Graph

To add web connector URLs to your Knowledge Graph, use the PUT endpoint to update the graph with the URLs you want to process. This endpoint allows you to add, update, or remove URLs from your Knowledge Graph.

You can only add URLs to a Knowledge Graph with the type set to web. To create a Knowledge Graph with the web type, you must create the Knowledge Graph in AI Studio and select “Add Website” as the way you want to add content to the Knowledge Graph. See Setting up Knowledge Graph web connectors for more information.You can see the type of a Knowledge Graph in the Knowledge Graph details or in the Knowledge Graph list endpoints.

# Note that this operation replaces the entire list of URLs attached to the Knowledge Graph.
curl --location --request PUT 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "urls": [
            {
                "url": "https://example.com/documentation",
                "type": "sub_pages",
                "exclude_urls": [
                    "https://example.com/documentation/admin",
                    "https://example.com/documentation/private"
                ]
            }
        ]
    }'

from writerai import Writer

# Initialize the Writer client. If you don't pass the `api_key` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

# Note that this operation replaces the entire list of URLs attached to the Knowledge Graph with the new list of URLs.
client.graphs.update(
    "GRAPH_ID",
    urls=[{
        "url": "https://example.com/documentation",
        "type": "sub_pages",
        "exclude_urls": [
            "https://example.com/documentation/admin",
            "https://example.com/documentation/private"
        ]
    }]
)

print(client.graphs.retrieve("GRAPH_ID").urls)

import { Writer } from "writer-sdk";

// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();

// Note that this operation replaces the entire list of URLs attached to the Knowledge Graph with the new list of URLs.
const response = await client.graphs.update('GRAPH_ID', {
  urls: [{
    url: 'https://example.com/documentation',
    type: 'sub_pages',
    exclude_urls: [
      'https://example.com/documentation/admin',
      'https://example.com/documentation/private'
    ]
  }]
});

console.log(response.urls);

Request structure

The urls array in your request replaces the entire existing list of URLs attached to the Knowledge Graph. To add a new URL, include both the existing URLs and the new one in your request. To remove a URL, send a request with all URLs except the one you want to remove.

When updating a Knowledge Graph with URLs, you need to provide a urls array in your request body. Each URL object has the following parameters:

Parameter	Type	Description
`url`	string	The URL you want to add to your Knowledge Graph
`type`	string	How you want to process the URL: either `single_page` (just this URL) or `sub_pages` (this URL and linked pages)
`exclude_urls`	array	Optional. Any URLs you want to exclude from processing if `type` is `sub_pages`

Response structure

After you update the Knowledge Graph, you see the status of each URL in the status object. The status object has the following parameters:

Parameter	Type	Description
`status`	string	The current processing status: `validating`, `success`, or `error`
`error_type`	string	If there was an error, what type of error occurred

{
  "id": "51b058d8",
  "created_at": "2025-07-16T20:19:23.565884Z",
  "name": "Public docs",
  "description": "Public docs",
  "urls": [
    {
      "url": "https://example.com/docs",
      "status": {
        "status": "validating",
        "error_type": null
      },
      "exclude_urls": [
        "https://example.com/docs/admin",
        "https://example.com/docs/private"
      ],
      "type": "sub_pages"
    }
  ]
}

Possible error types

If URL processing fails, you may see these error types:

invalid_url: The URL format is not valid
not_searchable: The content cannot be accessed or processed
not_found: The URL returns a 404 error
paywall_or_login_page: The content requires authentication
unexpected_error: An unspecified error occurred

Usage examples

The following examples show how to add a web connector URL to a Knowledge Graph and monitor its processing status.

Add a web connector URL

This example adds a web connector URL to a Knowledge Graph. It first retrieves the Knowledge Graph to see the current URLs and then adds a new URL to the Knowledge Graph. Note that this operation replaces the entire list of URLs attached to the Knowledge Graph, which is why it is important to retrieve the existing list of URLs first.

# 1. First, retrieve the current Knowledge Graph to see existing URLs
curl --location 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header "Authorization: Bearer $WRITER_API_KEY"

# 2. Then, send a new request with all the URLs you want to add. In this example, there were no existing URLs, so it's adding a single URL.
curl --location --request PUT 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "urls": [
            {
                "url": "https://example.com/documentation",
                "type": "sub_pages",
                "exclude_urls": [
                    "https://example.com/documentation/admin",
                    "https://example.com/documentation/private"
                ]
            }
        ]
    }'

from writerai import Writer

# Initialize the Writer client. If you don't pass the `api_key` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

graph_id = "GRAPH_ID"

graph = client.graphs.retrieve(graph_id)
urls = graph.urls

urls.append({
    "url": "https://example.com/documentation",
    "type": "sub_pages",
    "exclude_urls": ["https://example.com/documentation/admin", "https://example.com/documentation/private"]
})

client.graphs.update(graph_id, urls=urls)

print(client.graphs.retrieve(graph_id).urls)

import { Writer } from "writer-sdk";

// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();

const graph_id = "GRAPH_ID";

const graph = await client.graphs.retrieve(graph_id);

const urls = graph.urls;

urls.push({
  url: 'https://example.com/documentation',
  type: 'sub_pages',
  exclude_urls: [
    'https://example.com/documentation/admin',
    'https://example.com/documentation/private'
  ]
});

const response = await client.graphs.update(graph_id, { urls });

const updatedGraph = await client.graphs.retrieve(graph_id);

console.log(updatedGraph.urls);

Delete all URLs from a Knowledge Graph

To delete all URLs from a Knowledge Graph, you can send a request with an empty urls array.

curl --location --request PUT 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "urls": []
    }'

from writerai import Writer

# Initialize the Writer client. If you don't pass the `api_key` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

graph_id = "GRAPH_ID"

client.graphs.update(graph_id, urls=[])

print(client.graphs.retrieve(graph_id).urls)

import { Writer } from "writer-sdk";

// Initialize the Writer client. If you don't pass the `apiKey` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();

const graph_id = "GRAPH_ID";

const response = await client.graphs.update(graph_id, { urls: [] });

console.log(response.urls);

Monitor processing status

You can check the status of your web connector URLs by retrieving the Knowledge Graph details.

curl --location 'https://api.writer.com/v1/graphs/{graph_id}' \
    --header "Authorization: Bearer $WRITER_API_KEY"

from writerai import Writer

# Initialize the Writer client. If you don't pass the `api_key` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

graph = client.graphs.retrieve("GRAPH_ID")
print(graph.urls)

import Writer from 'writer-sdk';

// Initialize the Writer client. If you don't pass the `api_key` parameter, 
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();

async function main() {
  const graph = await client.graphs.retrieve('GRAPH_ID');

  console.log(graph.urls);
}

main();

Query the Knowledge Graph with web content

Once the web connector URLs have been processed successfully, you can query the Knowledge Graph to access the extracted content.

curl --location 'https://api.writer.com/v1/graphs/{graph_id}/query' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $WRITER_API_KEY" \
    --data '{
        "query": "What are the main features described in the documentation?",
        "subqueries": true
    }'

from writerai import Writer

# Initialize the Writer client. If you don't pass the `api_key` parameter,
# the client looks for the `WRITER_API_KEY` environment variable.
client = Writer()

response = client.graphs.question(
  graph_ids=["GRAPH_ID"],
  question="YOUR QUESTION"
)

print(f"Answer: {response.answer}")
print(f"Sources: {response.sources}")

import { Writer } from "writer-sdk";

// Initialize the Writer client. If you don't pass the `api_key` parameter,
// the client looks for the `WRITER_API_KEY` environment variable.
const client = new Writer();

const response = await client.graphs.question({
  graph_ids: ["GRAPH_ID"],
  question: "YOUR QUESTION"
});

console.log(`Answer: ${response.answer}`);
console.log(`Sources: ${response.sources}`);

Next steps

By following this guide, you can successfully add web content to your Knowledge Graphs using web connector URLs. Next, learn about other Knowledge Graph capabilities:

Getting started

Monitor and manage

Core concepts

Models and pricing

Connect to your data

Chat completions

No-code agents

Knowledge Graphs

Tool calling

Additional capabilities

Integrations

Security and compliance

Resources

Add web content to a Knowledge Graph

Overview

Add URLs to a Knowledge Graph

Request structure

Response structure

Possible error types

Usage examples

Add a web connector URL

Delete all URLs from a Knowledge Graph

Monitor processing status

Query the Knowledge Graph with web content

Next steps

​Overview

​Add URLs to a Knowledge Graph

​Request structure

​Response structure

​Possible error types

​Usage examples

​Add a web connector URL

​Delete all URLs from a Knowledge Graph

​Monitor processing status

​Query the Knowledge Graph with web content

​Next steps

Overview

Add URLs to a Knowledge Graph

Request structure

Response structure

Possible error types

Usage examples

Add a web connector URL

Delete all URLs from a Knowledge Graph

Monitor processing status

Query the Knowledge Graph with web content

Next steps