Import website links
The most general type of content you can import is website links. However, due to security and privacy/copyright concerns, not all links are allowed: the base link must match your custom domain. This feature will not work properly if you do not set a custom domain.
We also support links containing wildcards `*.` e.g., https://www.archbee.com/blog/*
This syntax will crawl all the links in https://www.archbee.com/blog/* and crunch the data.
Limits
The system supports a maximum of 500 links. We will take the first 500 links passed, even those containing wildcard syntax.
Sorting links
Links that are fed into the LLM are sorted in the following order:
- Explicit links
- Crawled links from wildcards
Make sure to pass your essential external resources as full links first if you are using wildcard syntax and know you might surpass the limit of 500 links.
