Overview
The document explains the visibility of events to search engine web crawlers, also known as robots or bots.
There are three options for crawlability.
All Private means no events are crawlable. This option is the default, so no action is required.
All Public means all events are crawlable. Certain can enable this for your domain.
Some Private means only some events are crawlable. Follow step 2 and then use the Robots META tag described below to make an event private (not crawlable).
The document does not cover how to structure the contents of your site to optimize it for search engines (SEO).
Default Behavior in Certain
By default, events created within Certain are not crawlable.
If the domain wants all events to be public, Certain can enable web crawlers across the domain and block-allow the events to be indexed.
The Customer Success Manager can facilitate this request.
Once this enablement is in place for the domain, there are additional HTML META tags that can be added to the display shell to enable better crawlability; however, that is outside the scope of this document.
META Tags
This section is applicable if web crawlers have been enabled for the domain.
A special HTML <META> tag can tell robots to index or not index the content of a page, and/or not scan it for links to follow.
By adding this extra HTML tag into the head of the event display shell, web crawlers can be instructed to exclude the event’s website(s) and form(s) from being indexed.
Private Events
To exclude Certain events from crawling, add the robots META tag described below to the custom display shell of the events’ display configuration.
How to write a Robots META Tag
When to include it
By default, sites that do not include the robots META tag are found and indexed by web crawlers.
If the goal is to exclude an event from being found by web crawlers, add this extra tag into the event display shell with the following values.
What to put into it
The NAME attribute must be "ROBOTS".
Valid values for the CONTENT attribute are: "INDEX", "NOINDEX", "FOLLOW", and "NOFOLLOW".
Multiple comma-separated values are allowed, but only some combinations make sense.
If there is no robots tag, the default is "INDEX, FOLLOW", so there is no need to spell that out.
<META NAME="ROBOTS" CONTENT="NOINDEX, FOLLOW">
<META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW">
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
Where to put it
Like any <META> tag, the tag should be placed in the HEAD section of an HTML page, as in the example below.
The tag should be put on every page of the site, which can be done by placing it in the advanced display shell with Certain’s display configuration (Plan Configure Display). This enables the HTML “wrapper” to be included in all the websites and forms in the event.
Values for 'Content' Attribute
See the overview for a list of the values for content, and the corresponding behavior that web crawlers will exhibit when they are included as part of the ROBOTS <META> tag.
| Value | Description | Used By | |-------|-------------|---------| | index | Allows the robot to index the page (default). | All | | noindex | Requests the robot to not index the page. | All | | follow | Allows the robot to follow the links on the page (default). | All | | nofollow | Requests the robot to not follow the links on the page. | All | | none | Equivalent to noindex, nofollow | Google | | noodp | Prevents using the Open Directory Project description, if any, as the page description in engine results. | Google, Yahoo, Bing | | noarchive | Requests the engine not to cache the page content. | Google, Yahoo, Bing | | nosnippet | Prevents displaying any description of the page in search engine results. | Google, Bing | | noimageindex | Requests this page not to appear as the referring page of an indexed image. | Google |
List from “<meta>: The Document-level Metadata element,” by Mozilla Contributors: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta, licensed under CC-BY-SA 2.5.
---
Was this article helpful? 0 out of 0 found this helpful
Related articles
- Email Analytics: Email Statuses and What They Mean
- Adding an email address as a hyperlink