Step 11 - Rank Checking
One of the rewards of a well optimized website is the increase in search
engine ranking. Search engine rankings can fluctuate a great deal almost
on a daily basis, and keeping an accurate record of rankings over time
is very important. Webmasters want to ensure they get an accurate picture
of their own website's position in order to react to any changes in search
engine algorithms or competing website strengths.
Websites that rely on visitor traffic from the search engines for leads,
sales, and any other types of search engine marketing want to know how
well their target keywords are performing. Whether you are in the middle
of a campaign targeting new keyword phrases for a well established website,
or your site has just passed the first month of being online, you want
to get an accurate report of your rankings.
In addition to checking on your own rankings, you may also want to check
on your competition from time to time to see if they are gaining or loosing
positions compared to your own rankings.
The manual and automatic rank checking options
We don't think we have to state the obvious, manually checking your
search engine rankings is time consuming and tedious, not to mention the
lack of accuracy and the record keeping aspect. The search engines can
make searching for a specific site a little easier by giving users the
ability to retrieve a larger search result per page, usually up to one
hundred per page. Even with this advanced feature, the ranking check can
take a long time to complete if you have hundreds of keywords in your
list and a handful of websites. So, if you have a site that you are serious
about, you can't pass up the ease and convenience of automated ranking
The search engines' view of automated rank checking tools
There is one main drawback to checking your search engine rankings with
automated tools. The search engines don't like automated tools downloading
their search results pages without an actual human visitor seeing them.
Why, you may wonder? The answer lay in the search engine's revenue model.
Search engine profits are driven by the number of "eyeballs"
seeing not only the natural search results, but the dozens of advertisers'
ads who are bidding on the same keywords the user typed in. It's no wonder,
the search engines have taken steps to limit the automated robots querying
One of the methods used by the search engines to detect automated "robots"
is the frequency of the query requests within a certain period of time.
If the number of requests exceed a specific threshold per minute or second,
the search engines block the users access temporarily based on their PCs
IP address. The ban normally stays in effect until the user types in a
security keyword in response to an image displayed or a specified amount
of time passes without any further suspicious activity.
Please note your own website can never
be penalized for running automated queries on the engines. It's
impossible for the search engines to find out which website URL you are
checking. The only information the search engines can derive from automated
queries is the IP address of the PC running the queries and the keywords
How does SEO Studio address safe rank checking?
In order to avoid getting your PC's IP address banned, the SEO Studio
Ranking tool offers a number of safe methods of automated rank checking.
The SEO Studio
Ranking tool has the ability to use the legal, publicly available application
interface search methods. APIs, short for "application programming
interfaces," give developers a "back door" access to their
search results without "scrapping" the result pages. Not only
these APIs provide safe and legal access for the developers to search
results, but the developers can create online and off-line tools using
these API features and offer them to the public.
The SEO Studio Ranking checker can use your existing Google SOAP API key
or you can sign up for a Yahoo! Developer API key.
Using proxy servers
to avoid detection of automated queries. Public or private proxy
servers hosted on external sites give you the ability to query the search
engines without revealing your own IP address. If a ban occurs, the proxy
servers' IP address used to query the search engines will be banned and
not your own IP address. You can find public proxy servers at Public Proxy Servers or you can sign up for a paid
proxy server service at Find
Not. You can also install a desktop based IP address concealing solution.
One of our favorites is Ghostsurf, a fairly inexpensive
package that does a great job at hiding your PC's identity. With all of
the solutions we have listed, you need to give SEO Studio the IP address
of the proxy server using the Public Proxy server dialog found in the
Settings menu path.
Increasing the time
delay between search queries. The search
engines can easily detect the time passed between search queries, and
an unnaturally high number of queries executed in a short amount of time
suggests there is an automated robot behind the search queries. This is
why you should increase the search query time delay to about 1,000 milliseconds
as a starting point using the Connection setting options.
Running a ranking check using SEO Studio
Once again let's illustrate the SEO Studio's Ranking tool's features
with an example. Prior to opening the Ranking tool, we have already input
our own Yahoo! and Google API keys in the Profile Settings screen. We
have also increased the query delay between each search query to 2,000
milliseconds to avoid our PC's getting banned by the engines.
1. Open the Ranking Tool
we'll name our report for easy retrieval later. Our ranking report name
will be called "Sample Ranking Report.” See the "Report Description"
text box below.
We have selected the "Exact URL Reporting"
option as well. This will enable us to view the full URL found in response
to our search. For example, using the keyword "discount import car
parts,” the engines may return a URL that is not the home page, but a
sub page on a specific domain. We'll want to know which page is ranked
for that specific keyword phrase, and that is why we have checkmarked
the "Exact URL Reporting" option.
We have also chosen to search up to 50
positions for our keyword phrases using the Google and Yahoo! APIs. This
means SEO Studio will retrieve the first 50 search results for each keyword
phrase and will check if the thepartsbin.com site and one of its competitor's
site carpartsimports.com is found in the returned results.
Now, we are ready to hit the Start button
to begin our search.
Checking your website's ranking becomes necessary once your site achieves
a minimum level of popularity and traffic. You need to get a detailed
picture of your site's ranking by keyword and search engines so you can
better target your optimization of specific sub pages of your website
or increase inbound links to pages that needs some more "link juice"
to get higher ranking. We recommend using the Yahoo! API to safely check
your ranking without the risk of getting penalized.
Google doesn't currently have an API interface for rank checking. Since
last year Google switched from the SOAP based API to an AJAX based API
and the new AJAX search API is not suitable for rank checking. The reason
the Google search AJAX API is not suitable is because it only retrieves
the first 9 results. When clicking on the "Next" link to retrieve
additional results pages, the API switches to the normal results pages
making it impossible to retrieve ranking results. Only existing owners
of SOAP API keys can use Google for rank checking since Google refuses
to give out any new SOAP API keys.
The fact remains, you need to run ranking reports at least once a month
with or without APIs, and there are some recommended solutions you can
implement to avoid getting your IP address banned. Some of the alternative
Internet connection methods include web or PC based proxy servers which
can hide your PC's IP address. You should also increase the time delay
between search queries to make your ranking checks search engine friendly
in using the Tools>Connections menu.