Problems with Niche Research Software
|
|
Years ago, before any company had automated the process, Ken Evoy described the idea of keyword supply and demand as a way to research online niche markets in his Affiliate Masters Course. Today, his online Brainstorm It! tool (currently a module available with Site Build It!) is one of the prime examples of effective niche-finding software. Recently I talked to Ken about niche research and problems with the available software.
Terms of Service Violations
One of the chief problems, Evoy contends, is that many research tools violate the search engines' "terms of service" and place undue burdens on search engine resources.
Each search engine has a legal "terms of service" document that outlines how the site may be used legally. Many research programs extract search results and keyword search frequency data from webpages that do not belong to them (called "scraping") and sell it to their own customers without payment or even credit. Some search engines, such as Google and applications such as Alexa.com, provide an API (Application Provider Interface) that allows other software applications access to their database and permission to use it within specified limits.
Some research software vendors -- both desktop and ASP (Application Service Provider) hosted applications -- pay for the data they gather or gather it with permission. Others "scrape" (that is, query and copy) ranking information from search engines, some with permission and some without. The problem is, as Evoy bluntly puts it, "Yahoo! Search doesn't like it when you bang them 300 times in a row." Some also gather search frequency and keyword bid prices -- with permission and without.
Abuse from desktop (client-side) software is harder for the search engines to trace and block, since the originating IP addresses are literally all over the map geographically. Some are search engine friendly, slowly gathering information as might an actual human in a manual browser process. Others are poorly designed, running at top speed, eating up search engine resources. Because of resource abuse by this type of software, search engines have erected barriers making it more difficult to get information, making it harder for everyone.
How does this affect the user? If the research software vendor gets its access to data blocked after you've paid your licensing fee, you're out of luck -- up a creek without a paddle! Your application will just stop working.
ASP hosted services (server-side software) that charge a monthly fee, such as Wordtracker and Brainstorm It!, tend to gather their information honestly and legally. To keep from annoying search engines, vendors are using one or more of the following strategies to lessen the burden on search engines and carefully observe the terms of service:
- Paying for data feeds to develop search databases.
- Spidering the web themselves to develop their own supply database.
- Requiring each user to apply for and submit a Google API Key (www.google.com/apis/), entitling that user to a maximum of 1,000 automated search queries each day.
- Having users do keyword research on the Yahoo! Search Marketing keyword suggestion tool themselves, then saving the page and uploading it to the research tool to scrape offline. This avoids breaking the terms of service.
One not so scrupulous approach is to accumulate data offline in a database of keywords that is updated every couple of weeks so the vendor doesn't have to constantly "scrape" in real time -- though this is strictly illegal, using another company's data without permission.
Evoy observes, "Right now, Wordtracker is the best, ethical, self-contained database for demand data," which is why they are the major source of Brainstorm It's demand data.
It's wise to know where your software vendor gets its data feeds so you can gauge accuracy, bias, morality, and legality of the data -- and not get caught someday with the datafeed being abruptly cut off. Also, if your vendor uses a regional datafeed, such as from a large ISP in Florida, the data may be skewed towards that region.
Supply Inaccuracies
A second problem is that at the very basis of supply-demand ratios is a fundamental inaccuracy. While Google may show 500,000 webpages in a search for "digital cameras," only a small number are actual competitors with finely tuned content webpages and a clear revenue model. But in a supply/demand ratio or KEI, they are all counted as "supply" or "competition," whether they are or not.
To counter the competition inaccuracy, some research software vendors such as Brainstorm It! are beginning to spider the Internet themselves. Instead of looking merely for keywords, they are looking for indicators that represent actual competition, rather than just an occurrence of a keyword, such as the presence of Google or Yahoo text ads, affiliate links, and products for sale.
It's Not Just about the Numbers
A third problem is how use what you learn from research. Ken Evoy reminds his users, "Don't get number bound."
"Success with a website isn't not about the numbers," he says, "but the human visitors and what they're going to value. And it's about the whole competitive universe that the site fits into."
Research tools help you determine whether there's a good mass of content, good demand and not too much supply. But the bigger picture, according to Evoy, is "What do I want to write about on a home page, the tier two sectional pages, and content in tier three pages?"
Develop a plan for traffic-building content first, then find ways to monetize it. If you don't see yourself enjoying writing the mass of content needed to succeed, consider a topic that fits your passion better. Evoy is disgusted with what he calls "trash sites that write any old garbage so long as you put keywords in it." He concludes, "If you keep it real (with enough on-page hooks so the search engines 'get it'), then it's all about pleasing the humans who come to the site."
Content that Sells
Evoy insists that most Internet small businesses don't approach an online business right -- even if they spend time to research. They look for the money, rather than focus on traffic which must come first. His mantra is:
Content > Traffic > PREsell > Monetize
in that order!
- Write great content with both humans and search engines in mind. This generates
- Targeted traffic which is essential for any online success. Then you
- Pre-sell (prepare visitors to make a purchase) by building trust and confidence through over-delivering what they seek -- relevant, original information delivered in your own "voice." Finally,
- Monetize those visitors through whatever revenue method you have chosen, so they're ready to click-through on product sales, affiliate links, or Google AdSense ads.
Niche research must be put into perspective. The research helps you pinpoint niches with possible business opportunities. After the research is done comes the process -- C > T > P > M -- necessary to develop that niche opportunity into a successful online business.
Sample newsletter. We respect your privacy and never sell or rent our subscriber lists. Subscribing will not result in more spam! I guarantee it!
