Managing Crawling of the Site Directory

When users add links to sites in the Site Directory, they can choose to include the sites in search results.

Before sites can appear in search results, a member of the Administrator site group must approve them. Sites added by administrators are automatically approved. Rejected sites and sites that are pending approval do not appear in search results.

A site administrator can approve and reject sites for crawling and otherwise manage the crawling of the Site Directory from the Manage Crawls of Sites Directory page. You can reach this page by clicking Manage crawls of Sites Directory in the Search Settings and Indexed Content section of the Site Settings page.

Sites that are approved are crawled when the content index is updated, and appear in search results. Sites that are rejected are not crawled and do not appear in search results. A site administrator can quickly see what sites need to be reviewed, what sites are already approved, and what sites are rejected.

To see one of these views, click the link for the view in the Crawl status of sites list.

From the Manage Crawls of Sites Directory page, there are links to Manage search scopes and Manage content sources. You can also add and delete links to sites by using the toolbar on this page.

Approve and Reject Sites for Crawling

On the title bar, click Site Settings.
On the Site Settings page, in the Search Settings and Indexed Content section, click Manage crawls of Sites Directory.
In the drop-down list next to the site that you want to approve, click Crawl this site.
The site is approved and will be crawled the next time the content index is updated. The site will appear in search results.
To approve multiple sites, select the check box next to each site that you want to approve, and then click Crawl this site in the Multiple Edit drop-down list on the toolbar.

On the title bar, click Site Settings.
On the Site Settings page, in the Search Settings and Indexed Content section, click Manage crawls of Sites Directory.
In the drop-down list next to the site that you want to approve, click Do not crawl this site.
The site is rejected and is not crawled. The site will not appear in search results.
To reject multiple sites, select the check box next to each site that you want to reject, and then click Do not crawl this site in the Multiple Edit drop-down list on the toolbar.

On the title bar, click Site Settings.
On the Site Settings page, in the Search Settings and Indexed Content section, click Manage crawls of Sites Directory.
In the drop-down list next to the site that you want to approve, click Require approval for crawling.
The status of the site is set to pending and the site is not crawled. The site does not appear in search results.
To require approval for crawling for multiple sites, select the check box next to each site that you want to reject, and then click Require approval for crawling in the Multiple Edit drop-down list on the toolbar.

Add and Delete Site Links

You can add and delete site links from the Manage Crawls of Sites Directory page.

On the toolbar of the Manage Crawls of Sites Directory page, click Add Link to Site. For more information about adding a link to a site, see Add a link to a site.

On the Manage Crawls of Sites Directory page, select the check box next to every site link that you want to delete.
In the Multiple Edit drop-down list, click Delete.

Managing Crawling of the Site Directory

Approve and Reject Sites for Crawling

Add and Delete Site Links

Related Topics