Categories: Crawling, indexing & ranking :

We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content

Showing 1-326 of 326 messages
We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/2/11 11:16 AM
I have read the FAQs and checked for similar issues: YES
My site's URL (web address) is: www.daniweb.com
Description (including timeline of any changes made):

We are an online discussion forum for people in the IT industry with about 1.5MM posts. Aside from our forums, we also have an editorial side with news, product reviews, tutorials, interviews, etc., which are contributed by a mix of community members and a team of staff writers. Recent interviewees have included Steve Wozniak of Apple, Inc. and Bjarne Stroustrup, the founder of the C++ programming language.

On the forum side, we are very tough on spam. 100% of our content is 100% unique. We do not buy SEO links. We do not sell SEO links. We do absolutely no blackhat -- or greyhat -- SEO, for that matter.

As of Thursday's algorithm update, we suddenly lost half of our US-based Google traffic, from 90K US visitors/day down to 50K US visitors/day. We were immediately thrust into a desperate situation, not able to afford our carrying costs such as next month's rent and web hosting bill. After eight years, we are currently at risk of having to close our doors. I don't know what on earth we would do if the algorithm expands outside the United States.

We made the Sistrix list as one of the top one hundred websites to be most affected by the "Farmer" algorithm update.

One of our staff writers, Davey Winder, has published an article pleading for help here: http://www.daniweb.com/news/story350575.html

We were subsequently featured in an article here: http://blog.searchenginewatch.com/110302-092711

Numerous other publications about innocent sites that were harmed by "Farmer" have mentioned DaniWeb. Numerous people commented throughout the web about DaniWeb being unfairly hit.

We are desperate. Please, please help us.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/2/11 11:22 AM
Hi Dani. I always hit "back" on your site because of the annoying popup you have when I hit your site. Get rid of that. I purposely don't click your links in Google because of it. Maybe you've changed it since then, but that's my #1 reason for not reading your site and hitting "Back" 10 miliseconds after clicking your link.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/2/11 12:34 PM
Hi Lysis, We had already gotten rid of it :) Thanks for the feedback!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content SouledOUT116 3/2/11 1:49 PM
Hi Dani of DaniWeb, I have been trying to find out what the truth of the matter is. I have yet to see anything that would make sense except the fact that tag pages may dilute a website. See: dcreators.com/how-not-to-get-affected-by-farmer-update/

RSS feed generated sites may also cause thin pages as well.

So, IMO weak content pages may be something that is an issue vs. duplicate content pages. For instance ecommerce product data bases are distributed to massive amounts of businesses where abstracts (content) are identical, yet I still see those domains thriving.

Aditionally, I see may domains that serve what I consider duplicate content with the intent of consuming more positions in the SERPs. On page content 95% idential and the only thing that is somewhat tweaked is the mixture or order of words in the page title. From a user perspective the pages are identical. So again this leads me back to thin pages rather than duplicates. If tag pages or RSS consist of a good percentage of total pages than it could be an issue.

For me, I have since removed all tag pages as mine were not as pretty as yours. That was on SAT the 26th. The above article gave me some comfort, PTL! I had more than I care to mention. I have seen some increases across multiple searches today. These types of honest mistakes are hard lessons to learn.

Good luck!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/2/11 4:36 PM

I do not think that there is anything anyone can do about their sites ranking being affected by this update. EXCEPT, all you can do is continue to follow best practice SEO and keep working on the site.

There have been major updates like this one in the past. The complaints now are the same as then. Some sites go up and some sites go down. Google did not change the algorithm then in response to the complaints and are unlikely to do so this time. The test for Google is the global quality of the search results for the searcher and not the individual ranking of individual sites (ie can the searcher find what they looking for?). Google monitor this metric very closely and will probably only change the algorithm if that metric changes.

Google are not going to manually intervene for individual sites:
http://searchengineland.com/google-weve-made-no-significant-changes-to-the-farmer-update-66591

I know a couple of key people in the SEO industry (whose opinions I respect) are still working on a detailed analysis of the winners and losers in this, looking for the factors that Google may have been focusing on. Hopefully they will have blog posts out within a few days.

ALSO, keep in mind, that if you have been in the forum for long enough, there are at least a few, if not a dozen sites complaining every day about a drop in rankings before this update was rolled out, SO, DO NOT automatically blame this 'farmer' algorithm update - look for potential other reasons.

ALSO, this again demonstrates the folly of having a business model that is dependent on a free traffic, from a free listing, in a free search engines. This business model is NOT sustainable long term. This 'farmer' update (like all the previous major ones) show that.

These are Google's search results, they can do whatever they like with them; they have no duty-of-care or any sort of contractual obligation to any single website to even index them let alone rank them, let alone provide traffic to.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/2/11 6:16 PM
We are not an e-commerce site and don't have any collection of pages that are nearly identical with the exception of various snips being pulled from a database somewhere. We are a discussion and editorial site where every page is unique content, either UGC or editorial.

We do have tags, but, as a developer site, we cater to a nearly infinite number of technologies and programming languages (that are constantly being put out), and therefore it's impossible to have a pre-defined list of every potential past, present and future topic. New platforms and architectures are constantly coming out, and sometimes it makes sense to be able to narrowly filter our forum threads like that, at a deeper level than the 100 categories we currently have. Therefore, we use tagging so that community members can find discussion threads and news stories about similar topics and technologies.

We use tags for the users, not for SEO. And it is such an integrated part of the site that it isn't something I am willing to part with right now. If anything, I think it is something community members would miss. Google constantly says that you should design a site for end-users and not for the bots. Well, we have tags for the users, and Google should learn not to penalize us for that. If I do something that negatively impacts the end-users specifically for SEO, then aren't I don't exactly what Google frowns upon? Is that really a valid reason for us to be penalized?

I don't think any of us really know for sure whether Google's algorithm is just getting tweaked or whether a whitelist actually does exist. It's rather suspicious that Cult of Mac made a lot of negative press for Google about how innocent sites were caught in the crossfire of this algorithm. Then, Matt Cutts tweets the owner of the site saying 'the appropriate people at the Googleplex have heard that report'. Then, the next day, suddenly they're back in full force.

Either one of two things happened: Either Cult of Mac was whitelisted, or Google altered the algorithm in such a narrowly specific way as to not trigger Cult of Mac to be hit, but affect almost no other sites. It doesn't really matter which one is the case, as long as the moral of the story is that Google realizes that a lot of innocent sites were damaged, and they are willing to make it right.

Please, Google, make it right for the rest of us. We are losing our businesses, careers, and sole source of income.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/2/11 6:33 PM
 
Every webmaster claims their site is innocent.
 
The facts often lie somewhere else.
 
 
Here are some questions to ask yourself as you analyze your own website....
 
Would you have a website if ads were prohibited?
 
Would this content exist if there wasn't an ad on the page?
 
What product or service are you trying to sell with the page?  Ad space is not a valid answer.
 
What pages have you deleted from your index in the past week, month or year?  Have you ever delete a page of content or combined several pages into a single page?
 
Do you link another website without using an add or affiliate link?
 
What % of your outbound links are nofollow?
 
How many outbound links are follow (without using an affiliate code)?
 
 
These and similar questions are being asked by Google with the Farmer update.  The people who fail my questions are failing the Farmer update.
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content rpray2007 3/2/11 6:34 PM
We too have suffered from this change and while I support Google's pursuit of removing web spam, there is absolutely no reason to bring out the canon when a little mouse trap is enough.  Sites like ours askmehelpdesk.com and daniweb.com are based on user generated content with active moderators and not so-called "reference" type site.  Most of our content is very high quality but it is user generated.  If Google is purposefully going after UGC sites, that's a very different statement and one I believe is unfair in many ways.  Users need access to UGC sites - not just reference information.  The purpose sites like askmehelpdesk.com, answerbag, daniweb etc serve is clearly to provide personalized information and knowledge to users.  This sometimes does result in "shallow" content.  So I encourage Google to reconsider the breadth of their change for the sake of searchers/users.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/2/11 7:14 PM
> Would you have a website if ads were prohibited?
I started the website as a Computer Science student in college. I started it because I was in need of a programming community where students could learn from each other, and didn't like anything that existed at the time. Therefore, I decided to create my own custom solution that fit all of my own needs. I grew it from the ground up with no advertising for the first 3 years. I wasn't even aware that it had the potential to earn money or be a business. Then, I learned that I could use AdSense to break even on web hosting costs. Upon graduation, I focused on building the website full-time, living at my mom's house to save money. I started selling advertising (separate from AdSense) about four years ago, and have since turned this into a career.

> Would this content exist if there wasn't an ad on the page?
Nearly all of the content on the site is user generated. We are a discussion community of just shy of 900,000 members. Community members post on the site to learn from each other, get help with personal and work projects, improve their skill-set, add to their resume, and participate in a community of people who share their interests. Every member who has ever participated on the site would of course have done so if they saw no advertising. We recently expanded into bringing on an editorial team of paid staff writers to supplement our discussion forums with high-quality editorial content such as news stories, interviews, and product reviews. The cost for this editorial comes entirely out of pocket -- We do it to add value to our users, by paying for writers and editors to provide more in-depth, longer articles on the topics that we have found our community is most interested in. The amount of money made off of ads appearing on these articles does not come anywhere close to the cost of paying writers for them. Therefore, we do it entirely as a value add to our community.

> What product or service are you trying to sell with the page?
Our primary goal is to create a great community for IT professionals and developers. Our feature-set is based around what I was initially looking for and couldn't find, and is therefore different than anything else that is currently out there. In addition to editorial, a Q&A discussion forum, and a code snippet library for programmers, we also have a COMPLETELY FREE online marketplace and classifieds section, which we started around the time the economy tanked to help freelancers and out-of-work techies.

> What pages have you deleted from your index in the past week, month or year?  Have you ever delete a page of content or combined several pages into a single page?
We take SPAM *very* seriously. Our moderation team usually deletes spam posts within a half hour of them being posted. We also delete all duplicate content, and strictly forbid any community members from posting articles that exist elsewhere on the Internet. All such content is instantly removed and the poster banned from the site.

> Do you link another website without using an add or affiliate link?
I'm not sure what you mean here, but we do not buy text ads for SEO reasons and we do not sell text ads for SEO reasons.

> What % of your outbound links are nofollow?
Forum signatures are only displayed to logged in community members, to eliminate what our moderation team calls "signature spam", aka when people post on our site just to promote their websites in their signatures. All links posted to our forums use nofollow since it is UGC. Links within editorial content (posted by our staff writers and editors) is not nofollowed, since we believe giving credit to primary sources and references in our editorial. Of course, comments to our editorial content by our community are nofollowed.

> How many outbound links are follow (without using an affiliate code)?
We have hundreds of thousands of pages of forum discussion threads, which is UGC, and therefore nofollowed. We have about 2,500 pages of editorial content which contain links that are not nofollowed.

Now, please ... why do we deserve to have been hit so bad??
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/2/11 7:24 PM
Why does it say this thread has been 'Answered'?! I am in desperate need of a resolution. Nothing has been solved. I need help still.

seo101, our traffic has always steadily increased over time. There was never a drop due to any other algorithm updates. However, suddenly, out of no where, this past Thursday, we lost over half of our US-based traffic, at the flip of a switch.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content TravBuddy 3/2/11 8:44 PM
Hi Dani - I run a large UGC site in the travel industry that was also negatively affected by this update (www.travbuddy.com). We also have millions of pages of unique, user generated content - 60,000 travel reviews, 150,000 travel blogs, 3.5 million travel photos, and an active forum section. We weren't affected as badly as your site, but perhaps our sites share some similarities, as this is the first real hit we've had in 5+ years of running the site.

In a post here: http://searchengineland.com/google-weve-made-no-significant-changes-to-the-farmer-update-66591 a Google employee says "In particular, it’s important to note that low quality pages on one part of a site can impact the overall ranking of that site." I feel that very large UGC sites are going to be unfairly affected by this. The nature of UGC is that some content is going to be good, and other content is not going to be as good - the ideal search engine would be able to answer and filter each query accordingly, rather than hit every page in a domain with the same broad hammer.

Just because one forum thread is of "low" quality, doesn't mean that another post of "high" quality should be penalized, because they are answering two different questions. Again, that's the nature of UGC.

Another example: we have over 3.5 million travel photo pages, which are probably "thin" on textual content, but are still helpful for users planning trips. It sounds like you have a large number of "tag" pages that are also helpful for users, but maybe "thin" on what Google may consider good content. But even if these pages are penalizing our sites somehow, it makes absolutely no sense to remove them unless we are trying to design for search engines and not for users.

Eric Bjorndahl
CEO/Co-Founder, TravBuddy
www.travbuddy.com
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/2/11 9:58 PM
"I am in desperate need of a resolution. Nothing has been solved. I need help still."

how about:
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/2/11 9:59 PM
1)   Then, I learned that I could use AdSense to break even on web hosting costs
 
Bingo.  There be a problem.  AdSense and other ads is the only source of revenue.  You said it.  Google can't find another source of revenue.  You got lots of pages with no buy now or call to make a purchase.  Instead we got pages with ads.  That smells of a Content Farm.  IF the revenue stream from the ads did not exist the site would not/could not exist.  Google says so because there is no proof of otherwise.
 
 
2)   Nearly all of the content on the site is user generated.
 
What isn't user generated.  Where did it come from and why is it on the site? 
 
I consider anything generated inhouse (editorial or visitors) to be of decent quality, even if it isn't because generally it is unique.  The issue is RSS, News Feeds, corporate announcements etc.  Especially if an ad is added to any of those sources of content.  Even if the ad is directly related to the announcement the use of ad indicates the content may have been added just to generate revenue through the ad.
 
 
3)  The amount of money made off of ads appearing on these articles does not come anywhere close to the cost of paying writers for them
 
Google doesn't know that.  Google knows content is added and an ad is generated. 1:1 ratio.  All content has an ad.
 
Try adding content for the sake of content without the ad.  Maybe the trick is to have 1 ad for every three pages of content.  Or one page with 300 ads and the next page with none.  We don't know what the ratio may be or how many ads can be on a page.  But 1:1 in anybody's book is going to be trouble.
 
 
4)   You missed the question.  What pages have you deleted from your index in the past week, month or year?  Have you ever delete a page of content or combined several pages into a single page?
 
Think an ecommerce site.  If a product is discontinued by the manufacturer, does the product remain on the site or does the site owner delete the product page and direct the viewer to the new product?
 
All too often the crappy ecommerce sites leave up the discontinued products as search bait.  Not good for the consumer.
 
You are an IT website.  Should you still have discussions about the merits of Windows95 or something equally stupid and out of date?  What do you do with those old pages?
 
Content Farms, just like crappy ecommerce sites will often leave the old material hanging around.  Not good for the web. Rework the old pages or just delete them.  This is especially true with an ongoing discussion..... let's say a rumour discussion regarding Windows27 to be released in 2041.  Do you need a separate article for each rumour that is produced?  How about combining all the rumours into a single running article and when Windows27 is finally released get rid of the rumour page because it is no longer valid.
 
 
 
5)  Do you link another website without using an add or affiliate link? 
 
What outside links do you generate that are not ad or affiliate based?   Do you link to.... oh I don't know.... UCLA Grad School of Computer Science as a follow link or a scientific whilte paper published by Intel as a follow link?  Or all of your editorial outbound links nofollow?
 
It seems that most sites with an issue with Content Farm do not have follow editorial links.
 
 
6)  You answered well.
 
7)  2500 editorial pages with all links nofollow
 
Stop and think for a moment.  Your ad links are follow.  Your editorial links are nofollow.  As far as Google is concerned the only outbound links from your site are connected to ads.
 
If we take this a couple steps further..... all content that is added is 1:1 with ads.  Lots of content is added, always with an ad.  Content is never deleted.
 
 
It smells like a Content Farm.
 
There are ways to improve the site.  Drop some of the darn ads.  Add follow editorial links.  Delete old content.  Remove the news feeds.  Show the site is to provide quality original information first, revenue maybe somewhere else.
 
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/2/11 10:13 PM
For the record, I've found your user forum helpful, Dani. I really did, and then I got the popup. That popup was there for a while, and I know that I felt shafted when I saw half the answer in the Google snippet and then got the annoying freakin popup. I hit back fast, because I knew I could just find it elsewhere. 

I'll use a Phil phrase that I remember him saying, and that was Google has amazing data mining abilities, and it wouldn't surprise me if Google saw that A) many people were clicking back on your site, and B) people avoided clicking your link because they knew it would just end up with the popup.

I know you said you removed it, but dude, you totally killed lots of traffic and it was there for a longgggg time.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content loki1234 3/2/11 11:00 PM
Checked that list, looks like  sites with user generated content seem to be hit with their new implementation. Looks like google has decided to support webmaster content as opposed to user generated content.

have your posted it on their new sticky post that they have exactly for this?

I'm a user of Daniweb, (It has been very helpful, except for that popup Lysis mentioned and which apparently is no more)
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/2/11 11:14 PM
"looks like  sites with user generated content seem to be hit with their new implementation."

Nope. I have two forums that have had a huge spike in traffic from Google since late last week. Have a couple of other older well extablished sites with no user generated content that lost >50% of traffic.

BTW, I also a user of Daniweb and it is helpful.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/3/11 6:47 AM
Stevie_D,

Thank you for your elaborate response.

Firstly, I have to say that I disagree with you that Google devalues any site that does not have a "buy it now" call to action. Google understands there are eCommerce sites, and then there are content sites that are ad supported. They stated when AdSense was first released that it was specifically geared towards blogs and forums that have no other revenue stream.

Also, I think that you read some of my responses backwards. All paid ads are passed through JavaScript. All links posted by community members use rel=nofollow. All links posted by staff writers within editorial content do not use rel=nofollow (they pass link juice).
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Puntsy 3/3/11 10:29 AM
I'm not an avid user of DaniWeb, but I have had questions answered before and would like to respond to a few comments made in this thread.

First of all, it seems as though some users here are making the mistake of equating UGC with low-value content. Yes, sometimes. But I would like to point out that this forum is UGC and a lot of the content here is valuable. Wikipedia is UGC (and probably the biggest UGC content farm ever) but the content is valuable.

I love Google. Google keeps me employed, but I have a hard time believing they are not making some exceptions to the rules after this update (Cult of Mac). The circumstances are absolutely too coincidental to be actual coincidence. What seems to have been affected most are mid-level sites that don't have magazines like Wired to be their voice.

Secondly, everyone has the right to make money on their content. I don't know what you guys think was happening before the Internet, but content was called BOOKS and NEWSPAPERS and MAGAZINES and most of them weren't free. I think there is some etiquette involved in how ads are displayed and the ad ratio (which 1:1 is nothing), but this assumption that ads automatically devalue content is juvenile. Some people have intense hobbies and are willing to put things out for free. For others, they are creators, writers, developers FOR A LIVING. Google does not care if they sell ads - they just need the content to back it up - which DaniWeb has.

Third - the "old pages" and removing conversations about "Win95" is totally unfair. Many websites have historical content. Just because it's old to someone doesn't mean it's irrelevant. That's akin to saying Wikipedia should remove all articles related to history. That's crap. Win95 is a piece of technology history and has every right to be discussed on DaniWeb.

seo101 - yes, you're right, as individual webmasters all we can do is play by the rules and hope for the best. I think it's good that Google is recognizing that several (many) sites were unfairly hit with this algorithm while sites like eHow still exist (in many of my SERPs where they weren't even at before) with poor poor poor quality content.

Someone has to admit it. Yes, the intentions behind this update were honorable and everyone with any good sense will agree with them. The implementation fell short. It didn't pull out some of the worst sites and some of the best got hit. Go back to MayDay all you want... this is different and a legitimate mistake. I think tweaks will happen in time, but right now you can't fault people with great web content who are absolutely justified in seeking assistance and explanation.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/3/11 11:09 AM
> You are an IT website.  Should you still have discussions about the merits of Windows95 or something equally stupid and out of date?  What do you do with those old pages?

I can understand the merit in removing old product pages from eCommerce sites. As someone who was recently in the market for a new living room furniture set, it would have been frustrating to me to Google for a particular style of coffee table, and find one that I loved, only to learn that it is now discontinued and can no longer be ordered, and was just never removed from the manufacturer's website. VERY frustrating, to say the least.

But I think that a community is different. Old threads in a community still add value. Members have been known to read old posts from a particular user to see how knowledgeable they are about a topic. Additionally, we all aren't technology early adopters. Old technology still frequently gets used -- Enormous companies have been known to not change their architecture for decades. In fact, the whole reason the world ran into the Y2K drama was because the same hardware/software was being used by major corporations, and the government, for decades.

Who is to say that no one could ever use any information about Windows 95 ever again? My grandma might have Windows 95 installed on an old Pentium II hand-me-down because she wants to learn how to ue a Word processor. I might have forgotten how to do something on such an old operating system, and a discussion forum that had been around since back in the day is the only place I was able to find the answer I needed. In fact, a search for live auctions on ebay for Windows 95 discs just returned over 31,000 results. I think it is very short-sighted to say that threads on a forum that revolve around older technology be removed. (That being said, our Google sitemap only lists current articles.)

From the community perspective, members volunteered hours upon hours of their time into helping people and posting content to be of value to our community. If content was deleted simply because newer technology came out making the content they had to say less relevant to people who no longer use the technology, those members would definitely feel like they wasted their time. Additionally, who is to say that 100% of the content in an older article or forum thread still can't be useful? When troubleshooting a network, for example, you apply nearly the same methodologies you did back in the days of Windows 2000 as you do with XP, Vista, and 7.
StevieD, I think that a lot of what you have to say is 100% appropriate to ecommerce sites. When I visit an ecommerce site, I want to see detailed information about the product along with calls-to-action such as "buy it now" and I DON'T want to see ANY advertisements. I think it is very tacky for an ecommerce site to have banner ads or AdSense ads. When I visit an ecommerce site, I expect that the information about every product I am reading about is still currently available for me today.
However, I think that content and editorial sites are an entirely different beast. Even online newspapers such as the New York Times see the value in a searchable, indexable public archive of all their old articles. And look at microforms at brick and mortar libraries. People have been going to the library to look up old newspaper articles and documents for centuries. I think it is absurd to say that the Internet search results are no place to lookup information about older news, information, articles or technologies.

So, when it comes to removing articles about older technologies, I disagree with you. However, as previously mentioned, we take the quality of our content very seriously, and are very strict about quickly and efficiently deleting all content that is spammy or is duplicate content that exists elsewhere on the Internet. 100% of the content on DaniWeb is 100% unique to us.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/3/11 11:17 AM
Puntsy, thank you for being my advocate. I was posting my reply at the same times as you were posting yours, and I think that we made a lot of the same points.

We are doing our very best to follow all SEO rules. We don't do anything blackhat. However, as you pointed out, it is very suspicious that Cult of Mac suddenly regained all of its traffic right after Matt Cutts tweeted to them. What made them so special? It's hard to believe there isn't some sort of whitelist.

We all VERY VERY VERY VERY much appreciate that Google has publically stated that they see problems with this algorithm, and are tweaking it to make it right. But, in the meantime, while we wait for that to happen, we are losing our livelihoods and watching our businesses slip away. My reason for being SO vocal is that if Google understands that there were a lot of innocent sites caught up in the mix -- and I am getting the impression that they do -- they should roll back the algorithm until it is appropriately fixed.

Some of us out there don't have hours, let alone days, for things to be corrected. For us, our 5 year office lease renews tomorrow. We can't afford to renew it with traffic down 50% since Thursday.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/3/11 2:12 PM
StevieD,

> There are ways to improve the site.  Drop some of the darn ads.  Add follow editorial links.  Delete old content.  Remove the news feeds.  Show the site is to provide quality original information first, revenue maybe somewhere else.

> Add follow to editorial links => All links within our editorial content ARE followed. Our ADS are nofollow.
> Delete old content => We delete *useless* content. We do not delete old content. The two are unrelated.
> Remove the news feeds => 100% of content on DaniWeb is unique to DaniWeb (unless people copied us without our permission). We do not pull news feeds from external sources. We do not syndicate anything from anywhere else.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WASDted 3/3/11 2:23 PM
Folks, I am not SEO expert at all but here's the bottom line in my view. Google has been growing at an insane rate and they are basically THE monopoly of search and advertising on the web. What they must realize is that their success depends every bit as much on the content providers who's content Google indexes as vice versa. True, many businesses and especially publishers (like DaniWeb) would not exist or be as sucessful without Google but the same can be said the other way around. Therefore it is important that Google make themselves more available and reachable, especially in times like this. I mean seriously, to roll out such a huge algorithm change and then have your head engineer in charge of spam and such (Matt Cutts) leave and go globe-trotting from event to event is not cool. There are thousands of legitimate businesses, with employees, bills and other expanses who depend on Google that are literally perishing. And all Google can do is to publicly admit they have screwed up a bit and say 'sorry and we're working to fix it' is not enough. WHEN, HOW, WHAT? How long will it be and how many companies must perish before a fix comes? No, they have to be reachable and take calls and/or direct emails or even form requests from concerned content providers and they must field these requests in a responsible and timely manner. They have effectively gained a monopoly position in the search world and they simply don't seem to realize how many businesses are affected and even destroyed by their actions. I've seen someone else say it best (not sure where I read it): "The bigger they have grown, the more untouchable and unaccountable they have become" - GOOGLE, WE LOVE YOU, but seriously you need to be more REACHABLE at times like this.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/3/11 4:22 PM
"There are thousands of legitimate businesses, with employees, bills and other expanses who depend on Google that are literally perishing"
 
They built a business model based on a free traffic from a free listing and a free ranking in a free search engine. Who in their right mind would think that this is sustainable long term? This update (as like all the previous major ones over the years) show that that model is NOT sustainable.
 
"but seriously you need to be more REACHABLE at times like this"
 
If you are unhappy with the service that Google is providing you, then ask them for a refund.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Puntsy 3/3/11 4:40 PM
I'm beginning to wonder if the "building your business based on free traffic" thing is entirely fair. Obviously, it's good to have a combination of online and offline marketing. But the economic ecosystem has changed. People do business online all the time. Several large companies have actually dropped their usage of paper anything (billing, ticketing, etc.) and some have even dropped phone or in-person CS.

Google is now a HUGE part of the economic ecosystem for most businesses (as is Facebook and Twitter to some extent). I think it's unrealistic to NOT base a chunk of your business on Google's traffic. If you don't, you're missing out on a huge portion of the market. Yes, there should be other channels.

I apologize that this is off topic, and in general I agree that it's a shifty business model, but just saying that building a business model based on Google is a bad idea isn't entirely appropriate. In fact, it's hard to find a business that DOES NOT use Google. And, if I can't find that business on Google, I'm likely not to know they exist. So, Google has to be a part. Maybe it shouldn't be all, but considering marketshare and the type of business a person runs, it could definitely be most.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/3/11 5:03 PM
"I think it's unrealistic to NOT base a chunk of your business on Google's traffic"
 
I get the bulk of my NEW visitors from Google, but my business model is NOT dependant on them. You need to put strategies in place (facebook, email newsletters, reasons for visitors to return, RSS feeds, etc etc) so if/when this sort of thing happens, you can survive ... risk management.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/3/11 5:17 PM
When this stuff happens, it's time to go back to basics...old fashioned advertising with a tech twist. Adwords, ads on other sites, social networking, emails to current customers. Some kind of promotion. It's ridiculous to sit on your hands saying "I can't do anything...Google is ruining my business." You enjoyed the ride for a while, and now it's time to put on your business hat and think of how to adjust to the changes. That's what make businesses successful. They change and work with change...they don't just sit there and wait for the business to run into the ground whining that Google ruined their business. Google is a search engine..it's not something you should rely on to support your entire marketing efforts. Come on people.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/3/11 8:19 PM
> If you are unhappy with the service that Google is providing you, then ask them for a refund.
I agree with this, while at the same time I think that Google has a responsibility to understanding how much damage they have the potential to do to small businesses, and they need to respect that. Therefore, when they roll out an algorithm that they themselves have admitted to having SOO much collateral damage, they should roll it back until they fix it.

That being said, we use AdWords VERY heavily. I am also very active on Twitter and we get a lot of traffic that way. We also have a Facebook fan page and a Meetup group that meets in our office the second Wednesday of each month. Based right out of NYC, we also try to attend as many events in the city as we possibly can, and are constantly getting our name out there. We also sponsor industry events, and last summer hosted our own event during Internet Week NYC which resulted in a couple hundred people in attendance (it was our first event).

We also have a large enough community of existing members that the number of new posts contributed daily to the forums has NOT decreased ... yet. The community is still very healthy, for now.

My problem is that we SUDDENLY lost so much of our traffic that it is devastating to our advertising revenue, which is our only source of income. I will admit it ... We were getting the traffic for so many years now, so consistently, that I relied on it, I counted on it, and I sold a certain number of ad impressions away to our advertisers, and now I can't fulfill on my contracts.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/3/11 8:30 PM
Dani, I'm just thinking aloud here, making a fool of myself, because I'm the noob of the TC group. ;) It's late, and I'm tired and drinking wine, so this is just from some quick thoughts.

First, just want to say that I always found your site useful...but oooh the popup! :P

Have you thought about networking with some IT sites that have 0 support. Maybe a developer group (I do a lot of programming searches, that's how I found your site) and asking for a link back for support?

My personal opinion that eHow survived was A) no fluff and B) backlinks.

Have you considered trying to network for some quality backlinks?

Have you considered flat out asking your huge community for a backlink? I'm a bit of a meme/troll geek. For instance, I think it was encyclopedia dramatica that asked for users to buy t-shirts to gain some revenue. If you have a huge support system, why not do something like that for some revenue? You'd be surprised at the support you'll get. 

I am also a part of Bluelight.ru. They run a yearly request from contributors to support. We love the site, so we contribute just because we love the cause of harm reduction. They basically make money from yearly drives for support.

It's a thought. No harm in asking for donations to support the cause.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/3/11 9:56 PM
The only small businesses that google is damaging are those that have been getting a free ride
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/3/11 10:00 PM
My problem is that we SUDDENLY lost so much of our traffic that it is devastating to our advertising revenue, which is our only source of income. I will admit it ... We were getting the traffic for so many years now, so consistently, that I relied on it, I counted on it, and I sold a certain number of ad impressions away to our advertisers, and now I can't fulfill on my contracts.
 
Everything is about the ads.
 
What would you do if your site sold..... oh I don't know.... Popsicles.  Frozen Popsicles.  Packed them in dry ice and shipped them across the country.  Then one day DOT (department of transportation) says that anybody shipping dry ice across state lines must have a permit due to hazards of dry ice and the permitting fee was 10x your annual sales.  You would be devistated, right?
 
But you would also realize that you have based your entire business upon the premise that you could do A, B and C.  Now you can't.  So what do you do?  Personally I would quickly develop a "do it in your own home" popsicle kit that did not require dry ice and continue my business.
 
Today, Google said you can't have a business based solely upon ad revenue.  Their rules.  Their ball field.
 
You can whine.  You can take all your business over to Bing (for the next 3 days until they copy Google).  Or you can play by Google's rules... and alter your business plan.  It took me 3 seconds to alter my popsicle business plan.  I bet if you spent 3 minutes or maybe 3 hours you could come up with a solid business plan for your company.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/3/11 10:14 PM
Amen, StevieD.

Come on people. There are so many ways to adjust. Resilience. Resilience is a part of being an online business. It isn't the end of the world. 

Love the popsicle analogy! 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content The Happy Geek 3/4/11 2:33 AM
To quote WASDted, I am not a SEO expert either. However, a number of questions do remain about the impact of the farmer update that require answering and Google has been pretty silent so far.

1. If the aim of this update was to improve the quality of search by downgrading the content farmer sites, then why is eHow still figuring so highly?

2. If Google is unable to intervene on a site by site basis to either tweak the algo or apply some kind of whitelisting to undo the collateral damage hitting innocent parties such as the DaniWeb community, then how come Cult of Mac went from 50% down on traffic and back up to full strength pretty much overnight after Wired and others profiled their plight and Matt Cutts tweeted them to say that Google would have seen the coverage?

Question number 2 intrigues me the most, as supporters of the update seem to be saying that everyone hit by it was obviously doing something wrong in terms of their content, SEO, business model etc and so there is no reason for Google to have a rethink. Yet it would appear that Google has certainly had a rethink about one such site, Cult of Mac, without asking it to change anything. That, surely, is Google putting its hand up high and admitting it got it wrong - and if it got it so badly wrong on one site you can bet it got just as badly wrong on others, including DaniWeb (I'm the community admin there, just to be clear) and logic dictates the honest thing to do is therefore roll it back for everyone until such times as the algo has been properly tweaked and tested and shown not to hit any innocent sites.

My personal opinion, not as a SEO expert but as an ordinary user of Google, is that the farmer update is broken. It didn't do what it set out to, the eHow positioning is surely proof of that, but it did harm a whole bunch of good sites, with good content and serving good communities of people.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content The Happy Geek 3/4/11 2:43 AM
One other thing, from reading various posts around the place which try to fathom out what is going on with this algo update, it seems that duplicate content is one of the triggers for being flagged as a content farm. The problem there is that the algo is not able to determine which site is the original source of that content, and that appears to be hitting places like DaniWeb hard. We have a team of volunteer moderators, based in various continents so as to cover all time zones, which works hard to spot any content posted to DaniWeb forums which has been cut and pasted from elsewhere. Such posts are deleted immediately and the users responsible for posting are banned.

However, what we cannot stop, what is totally out of our control, are those sites which take our unique content (be that a detailed answer to a coding problem, some sample code or our editorial in the form of product reviews or tutorials) and post it as their own.

We are the victim here, yet are being punished as if we were the perpetrator. In what world is that fair?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 3/4/11 6:58 AM
I've undone the Best Answer as the OP considers the question still unresolved - hope that doesn't offend anyone.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/4/11 7:14 AM
I visited your site a few days ago, but I cant remember whether yours had those horrible contextual ads injected on keywords in the body area?

Did you have any affiliate type content, like buy this now on eBay or Amazon?  That was a common theme on many of the sites hit.

I think your problem stems from the fact that perhaps your ads were more important than your content.  Certainly from the recent Google interviews on the topic it's clear that visitor perception of your site, as formed by the experience they have, is a key to what is tagged as "low quality".  Basically any aggressive advice you got from the Adsense side of Google, or other ad networks for that matter, is now exceptionally dangerous and should be unwound to ensure that content and user experience comes before ad placement and CTR.

I believe you can pull back from this, but you'll have to refocus your content and design to make your site more valuable to consume and visit. Ads are definitely the key to all these changes - not a single site without ads got hit from what I can see. Hope this helps. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Farmer Effects 3/4/11 7:25 AM
The Happy Geek, you are absolutely right.  My site is a blog with a similar footprint as Cult of Mac.  Monetized, popular in its niche, but heavily scraped -- I have run copyscape on my site and discovered literally TONS of thieves and scrapers feeding off my site relentlessly over the years.  Apparently, because I run ads, I am targeted.  The thieves have NO ads nor links anywhere.  I have an enviable back link profile from the top media sites around and my own community.  Yet, this did not matter!  The content scrapers are small and large, varied and come from everywhere, making it impossible to probably figure out who was the original.  Yet if you read the content itself, there are references to my NAME, my family, etc.  So a HUMAN reader can tell it is scraped and that I AM the original.

The irony?   Thieves with cleaner site layouts and no links have feasted on my hard work and are reaping the benefits of this algo.  My posts are nowhere to be found while 5 to 10 scrapers are appearing in page after page above my article in at least on example.  Even aggregate sites with excerpts that point back to me are above my original post.  Plagiarists are ahead of me.  I even discovered an EGREGIOUS example of an eHow plagiarist stealing my content and ranking on the 1st page while I was nowhere to be found.

I am reading how Google is quite pleased about cleaning the web right now, and results are great.  Yes, they are great because thieves with clean and great layouts are ranking on the first page, sporting OUR CONTENT.  That content is OURS on the first page, except they are dressed up in a different site with a less cluttered look and better design perhaps (to a robot).   SO YES, Content is much better now, EXCEPT THAT CONTENT Is OURS.  The true owners are kicked to the curb and victimized while scrapers reign.  

PLEASE FIX THIS.  My site shows similar issues as Cult of Mac.  And I've seen many many examples of this going on.  A car ratings company I stumbled upon had the same story.  I checked their examples and I was aghast by the punishment they've taken.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/4/11 7:41 AM
@Farmer Effects - I suggest you wait a little while to see what's coming.  Google never leaves things that hurt their users, and I'm sure they are working on multiple areas here that will greatly help good sites affected by this and the scrapper issue.

Just hang in there :)
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/4/11 7:42 AM
HappyGeek, eHow wasn't hit b/c their content is so bad that no one syndicates/copies/duplicates it. :)

I think that everybody has been getting the impression that all I have been doing is sitting on my ass and constantly whining in this thread.

Firstly, I know that it is not a good business model to rely solely on Google traffic. As I mentioned in the post above, we do a lot to get traffic from elsewhere, between Twitter, Facebook, sponsoring events, hosting events, etc. We also have enough repeat traffic that the number of new posts and threads being posted has not suffered at all. However, Google just significantly outweighs any of that.

>> For the record, I've found your user forum helpful, Dani. I really did, and then I got the popup. That popup was there for a while, and I know that I felt shafted when I saw half the answer in the Google snippet and then got the annoying freakin popup. I hit back fast, because I knew I could just find it elsewhere.

Lysis, we didn't realize how much the modal window was hurting us. We used to get questions all the time from people who would find us from a Google search, and not fully understand what we were or what we did. Often, they thought they were just looking at a single blog page, instead of realizing we were a full community covering every topic imaginable. People who did register were constantly hijacking threads with their own questions, instead of realizing they needed to start a new thread in the forum.

Direct and referral traffic are constantly making up a larger and larger percentage of our overall traffic. But it is still a minuscule amount compared to Google.

Therefore, we came up with the concept of the modal window which only showed to visitors coming from Google, and only on the first pageview of their visit. It said who we were, what we offered, and gave the option to register for our community. You could click *anywhere* on the page to instantly X out of the window, and not see it again for the duration of your visit. Registrations immediately increased by a factor of TEN and so we thought that it was working. As a side bonus, our ad CTR dramatically increased when we used the modal. The point being, we thought that it was working as intended because we were getting soooo many registrations, and it was just a single click anywhere on the page to exit out of the modal. We had the modal up for about a year and a half, and over that timespan, we ran at least four different tests where we took the modal down for a couple of weekdays in a row. New registrations instantly plummeted, and the absolute biggie, Google Analytics showed absolutely no decrease in bounce rate. We therefore continued to assume it was working as intended since it positively influenced registrations but did not negatively influence bounce rate. We ran yet another one of these tests in early February.

Then, when we were struck with Farmer, the *very* first thing I did was remove the modal window. This was a few days before I started this thread here. My reasoning was that even though every test I ran showed that there was no harm to having it, I thought that maybe Google decided that it was negatively affecting the user experience of people coming in from Google, and therefore they were sending less people from Google as a result. It was the first thing on my mind to change since I had just ran yet another test on removing it about two weeks prior. Your response to this thread a few days later proved that. However, there STILL has been no decrease in bounce rate according to Google Analytics, either globally or when I segment by country and look just at the US. Registrations HAVE significantly decreased (ten-fold). However, even with decreased traffic from Google, we are not receiving any fewer new registrations today than we had been receiving any of the other times we got rid of the modal window.

After the modal window decision, I focused on increasing page load speed, since we rank poorly for that according to Google Webmaster Tools. Step one in accomplishing this was I actually went and changed ad servers. We had been using the AdJuggler ad server for the past six years. After at least two full days of work, we successfully migrated to DoubleClick for Publishers. The idea is that Google's DFP is faster than AdJuggler (Since Google is always better and faster than anything else out there), and perhaps their algorithm will hold less against us for a slow ad server when it is, indeed, their own ad server. Additionally, since so many people across the web use Google's DFP, it's possible the JavaScript for it is already cached in a lot of people's web browsers, further improving page load times.

I also worked with my systems administrator to make sure mod_deflate was correctly set up to GZIP (compress) our pages. For some reason, it had only been enabled on 4 out of our 6 web servers, and I didn't realize that until I asked my systems administrator to double check. We are also currently in the process now of switching from mod_php to FastCGI, because from what I understand it leads to a significant speed improvement on the server side.

After that, I noticed there were a handful of Not found crawl errors according to Webmaster Tools, from some non-informational navigation pages that had moved a little over a year ago but had apparently slipped through the cracks when I was setting up all the 301 redirects. However, I noticed in Webmaster Tools that there were a good number of deep backlinks to these pages that I was now missing out on, so I added the appropriate 301 redirects.

I also discovered that Google Webmaster Tools was indexing a particular page in member profiles that had a different query string for each member, but was essentially the same for each member with just the slightest difference. It was showing up as having duplicate title and meta tags. I therefore added that page to our robots.txt file so that only one URL (the version without the query string) of it would be crawled.

While taking care of all of that, I also made it a point to try to get the word out about our situation. I tweeted to Matt Cutts. I emailed about five Google representatives whom I've worked with in the past (my AdSense rep, my AdWords rep, my Google Custom Search rep, my Google Ad Manager rep back from when I was a GAM beta tester, etc) in the hopes that any of them would be able to forward my email to the appropriate person. No luck there. I also posted comments on some of the earliest breaking news stories about Farmer. I started this thread here, and I also posted within the sticky thread for people who were affected by the latest algorithm change.

From what I had been reading, I got the impression that perhaps Farmer didn't like us because most of our content is UGC as opposed to editorial, although we do have an editorial side. Therefore, I reached out to my editor about making it a priority to get a lot more editorial content out right now. Obviously we are currently strapped for cash, and so it's hard to pay our staff writers for additional stories far over what we've already budgeted, so we started a forum thread asking our members to help us out by contributing editorial to our editor for review, to supplement what our staff puts out.

We actually already do collect donations for a 'Sponsored Community Membership', which entitles members to a handful of benefits including the ability to completely disable advertising and access to a behind-the-scenes forum. It has never been a significant revenue stream for us, though. In any case, one of the things that I went ahead and did was set all of our unsold ad inventory from AdSense instead to a house ad for a Donation Drive leading to the Sponsor DaniWeb page.

Unfortunately, the problem is that since we weren't expecting such a drop in traffic, we had already sold all of our US and UK traffic, and are currently close to not being able to fulfill on these commitments. Therefore, I was only able to advertise the Donation Drive to non-US/UK traffic, and people from India don't typically have a lot of spare cash for things like this.

With the modal window gone, I further wanted to improve the user experience from visitors coming in from Google. Therefore, I made a change such that when someone does a Google search and lands at a DaniWeb forum thread, the keywords they were searching for are automatically highlighted within the thread, and are pre-filled into our in house search box. Furthermore, when someone does an in-house search, and then clicks a search result, it, too, is keyword highlighted. Hopefully this encourages the web visitor to see that the landing page caters to them with the information they are looking for.

I would go on to list all of the other things we have changed over the past six days, but I've been writing this post for so long now, that I really need to move on and do some more productive things today. If nothing else comes from this, DaniWeb is sincerely a much better user experience now than we were only a week ago. Unfortunately, nothing has made even the slightest dent in our Google traffic so far, and I just feel like I'm constantly grasping at straws to find anything I can possibly think of changing.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/4/11 7:57 AM
> I've undone the Best Answer as the OP considers the question still unresolved - hope that doesn't offend anyone.
Thank you. Appreciated. :)

> I visited your site a few days ago, but I cant remember whether yours had those horrible contextual ads injected on keywords in the body area?
No, no, no.

> Did you have any affiliate type content, like buy this now on eBay or Amazon?  That was a common theme on many of the sites hit.
No, no, no. We have a strict policy of not having any affiliate links anywhere on the site, either through our own advertising or within any posts made by community members. The only exception is that we do allow our members to put affiliate links within their forum signatures, but only because you need to be a logged in member to see forum signatures. Therefore, this shouldn't affect Google, or casual web surfers to the site.

> Certainly from the recent Google interviews on the topic it's clear that visitor perception of your site, as formed by the experience they have, is a key to what is tagged as "low quality".
I'm not sure where you got that impression? From what I understand, everyone unanimously agrees that our content is of high quality, but we had that darn modal window popup that appeared for all visitors entering the site from Google SERPS. It was a popup but its purpose was not advertising ... It was a paragraph describing what DaniWeb is and encouraging visitors to register to contribute.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/4/11 8:26 AM
@Dani - here's the problem right now - you're still in denial.

You need to remove yourself from this and ask yourself - if you were a visitor to your site what would you think?  Would you want to see ads shoved in your face?

I'll step away from this for now.  I've tried to give you my opinion on what Google is using to define who to whack, but you're not looking for answers.  You're still stuck in denial mode.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Puntsy 3/4/11 9:19 AM
This thread is so incredibly upsetting and this is probably my last time posting here. I'm not sure how Dani is in denial. They've clearly been working on their site and making improvements. I don't think anyone in this thread is really sitting around complaining and not doing anything about it - and if they are, then they're not too smart and shouldn't be wasting time posting here.

I don't see ads shoved in my face when I visit DaniWeb. The ads that do exist aren't any more obtrusive than ads found on other sites. It seems to me that the majority of posters here consider any and all ads to be bad. If I'm producing unique content, I'm putting ads on it if that is part of how I get paid for that content production. If I'm selling a product (popsicles :)) I'm less likely to have any ads on my site because I make money by selling a product (which by then way is the case of one of my clients. They have zero ads and are still affected by this).

Yes, there are a lot of people complaining and posting in these forums who have huge legitimate issues that are wholly unrelated to this update. However, Dani's site wasn't a piece of garbage before and it isn't now. Find something legitimately wrong other than the ads (OMG an ad! I hate this site! What a poor user experience! - ever bought a magazine?) and the duplicate content that is clearly stemming from the UGC and is being taken care of in any way possible.

I think everyone commends Google for taking a step towards eliminating spam in its index. Unfortunately, they're eliminating the good sites along with some of the spammers. What we're left with is still spam and a bunch of mediocre results.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WASDted 3/4/11 10:33 AM
seo101 - "If you are unhappy with the service that Google is providing you, then ask them for a refund."

My friend, your attitude is funny, almost smelly. No one is asking for a refund, it isn't about that. You (Google) need to realize that Google is also as successful as it is because of all the DaniWebs out there. They built their business on the backs of all the smaller (and larger) companies out there that provide all of the information and content that Google indexes into a search engine. For that reason alone, not a matter of free or paid service, Google has a responsibility. It is not a question of whether or not you provide something for free, and no it's not free, they make billions off of all our (everyone's) content, this is content that we spent time and a ton of money producing. DaniWeb's content costs it a great deal of money to produce and Google gets to make money off of it everyday. They couldn't begin to calculate how much of a refund they would even give anybody for something like that. This is a two way street, our successes depend on each other, it is an ecosystem and as in any ecosystem if you take for granted even the smallest component everything is affected.

DaniWeb's ride on the Google train is by means a "free ride" - it built a business that relies largely on Google but Google's business also relies largely on the idea of the DaniWebs out there. Nobody is asking Google for a refund or any kind of compensation here, the only thing DaniWeb (and others like it) want is access to Google, someone to talk to, to work on this with, someone with answers. It is not much to ask of Google. You don't just make a huge mistake and seemingly shrug your shoulders.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WASDted 3/4/11 10:39 AM
And another thing. It is no secret that Google wants to be in control of all of the world's information. They want to have access to everyone's everything. Medical records, financials, personal and professional data alike, your behavior, your likes, your email, your status, your location, your past, present and future, your TV, your car, your phone, everything, Google aims to index it and store it all. And so far, for the most part the world (myself included) has embraced them and have pretty much played along. A company with those kinds of goals (much of them already achieved) MUST BE a responsible and trustworthy company. They must be accountable and most of all reachable by anyone who depends on them. They cannot make people rely on them and then decide to screw up or pull the plug, or change the game as they please without making themselves available to help people through it. We are ALL in this together.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/4/11 12:07 PM
my AdSense rep, my AdWords rep, my Google Custom Search rep, my Google Ad Manager rep
 
Therefore, I reached out to my editor about making it a priority to get a lot more editorial content out right now.
 
 
Lets see what we got here....
 
Cranking out content for the sake of content.
 
Ads, ads, ads.
 
The very definition of a Content Farm..
 
How about spending some time sanitizing your website of ads.  Start with very old content.  If the content/discussion is appropriate then keep the discussion and remove the ad.  Show Google the priority is the content, not the ad.
 
At some point, if you do enough ad removal on the historical content, your ratio of ad to content will fall inline with the parameters set by the Farmer update.
 
Then, stop using any news feeds for your publications.   Intel got a new chip?  Great.  Write your press release.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/4/11 12:25 PM
> You (Google) need to realize that Google is also as successful as it is because of all the DaniWebs out there

Actually, no, Google is successful because they bring readers what they want. Websites were created before Google came around. It has nothing to do with the webmaster. Webmasters jump when Google changes, because they want the free ride. They want to put 0 money in marketing and cash in on Google's search that is made for readers, not webmasters.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WASDted 3/4/11 1:03 PM
That is only partially true Lysis. Google as only a search engine that is free to use and directs readers to the content they want is not much of a business in itself. There is so much more to how Google works and connects to websites and it is a give and take in many cases. Google gets webmasters to insert their code, they crawl other people's content and intellectual property and they enjoy free access to all of this. Then they get to use this information to sell ads and make all sorts of money in many complex ways. Please, don't kid yourself.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content mayagodi 3/5/11 3:32 AM
I have never commented in these forums but I find myself upset at the uneducated and newbie remarks. so... I will. Dani, love the site. I am a developer and have used your site on many occasions. Too many people who do not have internet businesses believe that content should be free. That myself and every content publisher should spend all day researching topics and creating content for content's sake.Server costs, bandwidth... who cares? right? They just consume and we must deliver regardless. Well thats outrageous.

I manage many sites, some as large as 60 million pages. Those sites take money and resources to keep going. If ads were not a part of the internet we would not have one... just like we would not have network TV if they did not have commercials. Really, we all watch the Super Bowl for free because the advertisers are paying for it. 10 years back, we charged for content, that has changed. Most Internet users will no longer pay for content because they can search and find it for free. So, just like TV, we must adjust our business models to include paid ads. Ads are not the issue. Google has made an empire over that business model. The sites I track all have ads, most are fine, some got a boost, a few went down a bit, but all have ads.

The issue that many are complaining about is the severity of the update. Think about this... I buy a store front on Wilshire Blvd. I work hard to get the signage just right to attract passer by traffic (yes free traffic). I work on my store and invest in making it better for customers. One day, the city comes by and says 'we are closing down Wilshire, sorry about your luck but we noticed some gang activity in the area and to curb that we must also close the street'. That in a nutshell is the issue.

Saying, 'ask Google for a refund' like someone said is naive since we all supplied the content that makes up the SERPS. We are a huge part of their business like they are of ours.  It is a 2 way street, without publishers that give away content there would be no search. Without ads to pay for servers there would be no free content. It is very easy for people who do not pay costs to say remove the ads, for the sites that do get a lot of traffic it is not realistic to foot the bill for 5, 10, 20, 1000+ machines just to supply FREE content for the masses.

In my research, I can't find a difference between sites with and without ads that got hit, therefore, I do not think thats the problem. It looks like certain verticals got the brunt such as IT, home improvement, education, health. Maybe those areas are over saturated, not because of the ads running but because of how much those ads pay causing the over saturation. One of the sites I manage for a client has millions of pages of very thin UGC. It went Unscathed, but, it was not in a high paying niche and more so, it was in a very tight niche.

Removing old content? that ludicrous!!! I do not believe that is an issue or cause. I still have old Win 95 and DOS machines in the basement I turn on now and again as well as look up commands I do not remember. I can't believe anyone would even suggest destroying our history. Why not go burn the books 5 years and older while you are at it? Really?

Dani, please don't remove "old" content. Many of us still look it up. I had to modify some old (10+ years) code recently for someone. I am glad I can still find old reference material for languages now obsolete. The Internet is a digital library, libraries do not destroy their books! Don't listen to the people who have never had a successful site in this venue.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/5/11 8:20 AM
Don't want to destroy old "valueable" content?  Then don't.
 
But you can sanitize the content of ads.  Prove the content exists for the content, not the ads.
 
 
The Wilshire Blvd issue is a great analogy and a complete non issue.  You located your business on Wilshire because of the free traffic. If the traffic moves, costs $ (toll road)  or otherwise disappears, you adjust the business or move the business.  Nobody "owes" you the free traffic.  The same thing happens when the new connector rd doesn't locate where you want it, or the railroad skips your town.  Businesses have had to adjust for 100's of years to changes in their long range business plans.  So why not now as well?
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/5/11 9:59 AM
I'm sorry, Stevie and RedCardinal, but I have to disagree with you about the advertising on the website. I am not in denial. We are not advertising heavy. Sure, we have ads to support the business, but we do not have more ads than any other editorial-based site does. We have one leaderboard ad and one text ad, that we sell primarily directly. Some pages also have one medium rectangle ad. We don't work with any low quality ad networks. We also used to have three AdSense ads, but we reduced that to two on some pages, one on some pages, and none on some pages, when we switched ad servers. We have NEVER received ANY complaints from any site users that our site had too much advertising.

>> I visited your site a few days ago, but I cant remember whether yours had those horrible contextual ads injected on keywords in the body area?

No, we don't have these. It is silly to tell us that we have too much advertising if you can't even remember what we do have, and thought we had something we didn't.

Also, to repeat myself, we do not syndicate content. We do not get our news from other stories that are already out there. All of our editorial content is from first-person interviews, conducting our own product reviews, and insider information. None of our content can be read anywhere else on the Internet. No content is created simply for the sake of creating content.

That being said, advertising-supported content is part of the natural ecosystem of the web. Google understands this. That's why they have AdSense. I don't think that Google's algorithm is penalizing us for having ads.

What I think Google's algorithm is doing is penalizing US because other scraper sites copied our content. Google has admitted that there are many sites that were falsely penalized because other sites scraped them, and not the other way around. My issue is that I sincerely believe that, since Google has admitted there is a problem with their algorithm, they should roll it back, fix it, and then roll it out again.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Ryan Clark 3/5/11 10:56 AM
Some people just don't understand what it costs to operate a quality site such as yours, your ads are nowhere near a problem. A pop-up wouldn't cause this problem either, what a joke response. Tons of quality sites got hit and Google has acknowledged this and its working on a fix, see http://www.wired.com/epicenter/2011/03/google-spam-side-effects/...I'm surprised no one mentioned this yet.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/5/11 11:18 AM
>Some people just don't understand what it costs to operate a quality site such as yours
 
Here, let me fix that for you
 
Some people just don't understand what it costs operate a quality business.
 
 
Online, Catalog, B&M.... it doens't matter what the type, a quality business costs $.
 
Of course a real business doesn't worry about those costs because they have a solid business plan and diverse sources of revenue to back up that business plan.
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/5/11 11:19 AM
>> No, we don't have these. It is silly to tell us that we have too much advertising if you can't even remember what we do have, and thought we had something we didn't.

Not silly, as I visited your site after this first happened, and then again when you posted here.  I cant be sure what changes you may have made in that interval.

What I'm trying to say is this: what you or I think doesn't matter.  The reality is that your site has been affected by this change, so all I've tried to do is suggest that you spend your energy trying to reverse out of this.

The Wired interview gives huge insight into what they are testing to determine "quality".  Go read that, pay for some cheap usability testing (you can do this over webcams and the like) from users who have never visited your site, collect feedback after asking them to be as honest as possible, and then try to conceive some tactics to get your site back on the right side of this.

It's not about having Adsense, it's not about pop-ups, it's about user experience.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/5/11 11:31 AM
> It's not about having Adsense, it's not about pop-ups, it's about user experience.

Exactly. I never said the reason for his issue is the popup, but from a user point of view, it's intrusive and annoying. Hence, why I never went further than the first page on that site. 

As for Adsense, the huge block at the top of the content page is also annoying. I can't skim the page to see if it's right for my query. I have to scroll past ads. Very bad user experience.

No one writes for a reader anymore. It's always for bots.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/5/11 11:36 AM
>Sure, we have ads to support the business, but we do not have more ads than any other editorial-based site does
 
Really?  How many ads do you have (ads per page, ads per word, pages without ads versus pages with ads) on the site versus the other sites?
 
What is allowed?
 
What will be allowed tomorrow?
 
 
You realize if 1 ad per 1000 words is allowed, and you have 1 ad per 999 words then your site is in violation of the ad ratio and will be slapped down.
 
 
So how many ads are allowed?  We don't know.  We never really knew how many words were too many for the keyword stuffing penality either.  People guessed.  Some guessed wrong and paid the price.
 
BUT, and this is a big BUT, everybody whining about the Farmer Update have one thing in common... Ads.  Those without ads are NOT having a problem with the update and most have improved their position.  Others with ads are not having a problem ether. 
 
So what could be the factor?  Java, Flash, CSS, H1, Meta whatever.
 
Hmmm....
 
Maybe just maybe, something simple:  ad versus content ratios.
 
Experimenting with the ratio is simple... drop some darn ads and see what happens.
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content shades1 3/5/11 12:02 PM
I highly, highly doubt the site is affected by the ad to content ratio. People don't think of daniweb as spam at all or a place to just throw ads
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/5/11 3:36 PM
http://searchengineland.com/your-sites-traffic-has-plummeted-since-googles-farmerpanda-update-now-what-66769

There's some of the best advice on this change I've seen to date in this post from Vanessa Fox. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/5/11 5:08 PM
> I never said the reason for his issue is the popup
I'm a she :)

> As for Adsense, the huge block at the top of the content page is also annoying
Huge block at the top of the content? Huh??

> How many ads do you have (ads per page, ads per word, pages without ads versus pages with ads) on the site versus the other sites?
As mentioned, all pages have a leaderboard banner ad and a one-line text ad at the top. In addition to that: Navigational pages have a text ad. Short content pages have an AdSense ad. Long content pages have two AdSense ads.

Short content pages used to have 2 AdSense ads and long content pages had 3. We chopped that down to 1 and 2, respectively. No change.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/5/11 6:07 PM
>Short content pages used to have 2 AdSense ads and long content pages had 3. We chopped that down to 1 and 2, respectively. No change.
 
1)  At this point we don't know and are only guessing, but the response time to changes you make may not be instanteous as there could be a sandbox effect of somesorts.
 
2)  Keep playing with the ratio.   And most importantly try content with no ads.  Remember, you are trying to justify the content as being so important that the lack of an ad is not an issue to your site/viewers.
 
Personally I would sanitize old content of ads.  It is relatively easy to do and could be very effective
 
(besides how much revenue could ever be generated from the old content?)
 
One of my favorite CE forums does exactly that.  No ads in the old content.  At the same time the fancy CSS, avatars, user's signature lines etc are stripped out as well.  It is just the forum discussion... important questions and answers.  I suspect, no proof, that more than vast majority of the site's total content is void of any ad because there are all these old forum discussions running around.  I know Google is still indexing these pages because I did the Google thing to find one of my old posts.
 
 
PS:  if you find the magic ratio.   Keep it to yourself.  Don't want to help the real bad Content Farms to find their way back to Google search results.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/5/11 6:16 PM
> I'm a she :)

Ugh. Sorry. You know, I knew that. Don't know why I said "he." I'm actually a "she" too, and get called a he all the time. I'm used to it.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/5/11 6:18 PM
@StevieD_web

RE: "1)  At this point we don't know and are only guessing,"

There is no point in time when we are not guessing and the point at which we "know" is a self-assessed thing which may or may not be well-founded.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/5/11 8:22 PM
> At this point we don't know and are only guessing, but the response time to changes you make may not be instanteous as there could be a sandbox effect of somesorts.

Google Webmaster Tools still shows the Site performance stats last updated Feb 20th. I'm not sure if they have more updated stats than what they publish in Webmaster Tools, but either way, as you mentioned, they can sandbox the results for awhile as well.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/5/11 9:40 PM
>There is no point in time when we are not guessing and the point at which we "know" is a self-assessed thing which may or may not be well-founded.
 
Oh how true. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Donel 3/5/11 10:03 PM
2)  Keep playing with the ratio.   And most importantly try content with no ads.  Remember, you are trying to justify the content as being so important that the lack of an ad is not an issue to your site/viewers.

I keep seeing you saying that, I am unsure of your intention.

People put websites up to make money, if they didn't the web would not be what it is today.

Most people aren't bloggers on their free time, and do it for a living. You put a website up to compete with the niche.

You don't spend hours working on things for the benefit of your health, and if you did. It wouldn't be competative these days..

Banners can penalize you for the website load time. You should do things to eliminate that.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/5/11 11:58 PM
Yes people put up websites to make $.  The website sells something.  Adobe sells software.  Dell sells computers.  Intel sells computer chips.
 
See a simple pattern?  Selling Stuffl.
 
Content Farms sell ad space.  That is all they have to offer.  So the typical Content Farm has lots of content to sell lots of ad space.  And next week even more content to sell even ad space.
 
The issue is ads = content.  Or content = ads.  How ever you want to phrase it.
 
Google has figured out the blooming Content issue, some of it pretty darn crappy at that AND often highly repetative in nature, is done for one purpose.... selling ad space
 
Since Google can't regulate the content (and besides Google likes content because it helps to expand their search index), the only thing left to regulate is ads.
 
But Google sells ads.
 
Ah yes, the classic Catch-22.
 
Can't regulate content because you like content, can't eliminate ads because you sell those.
 
So what to do?
 
Since others sell ads (besides Google), and going after the ad placement services would be a FTC issue, Google could only do something that might potentially harmed itself as well as their competitors.
 
What is left?  The algorithim is measuring something on the sites.  It surely ain't the size of the CSS file.
 
So what could it be?
 
Looking at what everybody agrees is a Content Farms, it is pretty obvious the pure Content Farm is loaded with ads.  If we take this conclusion one step furthere, it is logical to assume the higher the ad to content ratio the greater chance the site is a Content Farm.
 
So my conclusion is that Google is targeting ad versus content ratio.
 
 
Am I right?  I don't know.  But I do know a bunch of slimmy dirty hat SEO types have come (more or less) to the same conclusions.  There are thoughts that the issue is tabbed pages or paginated pages with different ads showing on the tabs/paginations.  There are a couple blog/ seo website discussions about ad to content ratio issues and one of the slime ball SEO companies issued an email directive instructing his clients to cut back on ad content (hmmm, where have I heard that before) to no more than x ads per shown page and to make sure the ad shown on the paginated pages is identical.  This particular email referrenced another SEO firm doing the same set of instructions to their clients as well.
 
IF I am wrong, then what factor is Google using in the algorithym?
 
PS;  Personally I have an ad free site.  You can load up your site with all the SEO, Viagra and Pepsi Cola ads you want and watch your site tank as we speak, or you can be proactive about the correcting the problem.  Your choice.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Donel 3/6/11 12:35 AM
"Yes people put up websites to make $.  The website sells something.  Adobe sells software.  Dell sells computers.  Intel sells computer chips.
 
See a simple pattern?  Selling Stuffl."

Ya I see the pattern..

If I lookup StevieD is an idiot, I better see adobe or a service selling me something.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/6/11 2:31 AM
>> 2)  Keep playing with the ratio.   And most importantly try content with no ads.  Remember, you are trying to justify the content as being so important that the lack of an ad is not an issue to your site/viewers.

You'll never know the impact.  Google are not going to make this a switch that suddenly turns off when you have the right "mix".  That would be inherently foolish as spammers would take advantage.  At a guess I'd say you're looking at a long multi-month battle to get back what you lost. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/6/11 4:33 PM
I am sorry but I do not agree with a lot that has been said in this thread. If anybody knows anything about anything then Red Cardinal will know so I guess I am wrong but I do not see anything in this update that is affected by advertising nor can I see anything wrong with the results either, from an amateur's viewpoint.

Others have plenty of advice as well but it is easy to give advice when you do not have to open your wallet. If I were Dani, I would ignore the advertising advice and focus on strengthening the "community" aspects of her site. Sorry.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 5:29 PM
SEO101 - and several other of you GOOGLE Answerers.

You are VERY WRONG.
==============================================================================
"ALSO, this again demonstrates the folly of having a business model that is dependent on a free traffic, from a free listing, in a free search engines. This business model is NOT sustainable long term. This 'farmer' update (like all the previous major ones) show that."

These are Google's search results, they can do whatever they like with them; they have no duty-of-care or any sort of contractual obligation to any single website to even index them let alone rank them, let alone provide traffic to."
==============================================================================

See - here is where you forget yourselves. BLATANTLY and disrespectfully to OTHERS.

Google put themselves in the position to MAKE people rely on them for making money online.
Google specifically chose the model - to create a NEED for search engines - to provide better content (or the vision of better content).
GOOGLE - has created what it is - AS A BUSINESS MODEL.
And has become the SLAVEMASTER to most websites.
Because of Google - Search Engine Optimization became a field.
Because of Google - people earn money - because ------ GOOGLE EARNS Money.
Google is a business - and don't forget it.

If you think otherwise - you're an idiot.

I've seen you trash webmasters who come in here looking for help. But because "YOU'VE SEEN IT ALL" and "YOU KNOW IT ALL" - you've snarked them out and have done NOTHING to help them.

You're not an elite class. You're more like bullies.

Attacking and not providing assistance. You're supposed to be the HELP. More often than not, you bite the webmasters who come in here, treat them as if they are stupid - call them out with catty remarks and basically stomp all over them.

Very few people on this forum have been helpful - those that are - I am VERY grateful for - and I hope they know who they are (The few that I can name right off hand are Puntsy, Pelagic, salsurra) who came in - provided assistance without talking down and treating me like a toddler or someone who is not worthy of having an opinion or a thought - let alone having a problem.

Dani's site is decent - and was hit by the Farmer / Panda update.

People rely on the traffic produced by Google - BECAUSE GOOGLE CAUSED THEM TO RELY ON IT.

Google is responsible for the change - Google is responsible - like a doctor is responsible for handing out medications that people get addicted to.

Sorry - I see Google changing the game plan because they feel they need to shake up things and make a difference.

They - like any business - need to keep new and improved items happening - to make it "SUPERIOR" or better.

This economy is bad enough as it is - and even Google is probably affected - although Google is part of the problem (Anyone who has studied advertising trends, etc. knows without a doubt that Google singlehandedly killed many advertising industry leaders.)

So - when you say Google is a FREE search engine that people should not rely on - you know NOT what you speaketh of.

I'm sorry - I'll stay here on this forum as I please - but I will gladly try and help where I can - but I have been snarked at one too many times just simply by asking questions.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/6/11 6:04 PM
Unfortunatly Lycrical, you are the one that is wrong. You rant rant as much as you like about this; you can fantasize as much as you like about what you think Google actually does.
 
At the end of the day, they are their search results and they can do what they like with them. The courts have upheld their right to do that. It does not matter that you think differently.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 6:12 PM
"At the end of the day, they are their search results and they can do what they like with them. The courts have upheld their right to do that. It does not matter that you think differently."

True.

But that is not what you said before.

You said that it was a FREE search engine.

It is TRUE that they can do what they like - they are a business. But for you to be snarky has NOTHING to do with that. You said:

"free traffic, from a free listing, in a free search engines"

and you related that business models should not be maintained on that.


I say utter BS. Google wants EXACTLY that.

Otherwise - why have advertising on Google?

If no one counted on making money - there would be no content sites. Now Google has gone and changed up the model - and both you and I know within a month - blackhat will have crawled it all up - while people like Dani will have to slowly mix and match. So quite literally Google has screwed some innocent bystanders in an update to maintain or upgrade...

Irrelevant to that is the fact that you're still rude to the mass majority of people. You personally. Not Google...

I've read a large percentage of your answers to people and you're not helpful. You're derogatory and talk down to people.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/6/11 7:17 PM
Its not utter bullshit. Google provide you with free traffic - its that simple. If you want to build a business on getting that free traffic, then that your a business choice you make. I just can't figure out why people who made that choice to base a business model on that free traffic complain whn Google take that free traffic away. No one is paying Google for that free traffic, so Google have no contractual obligation what-so-ever to webmasters to provide any traffic at all.

As way of analogy. What if I own a corner bricks-and-mortar store and a big supermaket opens up and brings in a lot of walk in traffic to my business. I pay them nothing for that traffic. What if that supermarket suddenly decides to move acorss town and I loose all that walk in traffic - they have no cotractual obligation to me to provide that traffic that they are now taking away. I might bitch and rant about them, but they are under no obligation to me.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/6/11 7:26 PM
I just saw one of your banner ads Dani! It's a step in the right direction, although I don't know if you've always advertised.

I hope you recover now that you got rid of that popup. :P You always had good info.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 7:43 PM
"No one is paying Google for that free traffic, so Google have no contractual obligation what-so-ever to webmasters to provide any traffic at all."

Hmmm...

OK - I'm going to carefully state something that you may not see, as you are too close to the content.

Google is a MULTI BILLION dollar business.
It created a "WAY" of the internet by "herding" and separating sites into categories, then teaching people they needed those herds in order to find anything.
It taught business owners to use SEO techniques to move to the top of the search results.
It then changes the rules periodically (because of course some people take advantage of such things).
Google's content and search engine are precisely like a doctor who has handed out morphine left and right - then doesn't understand why people are heroin addicts. (my analogy)
Google's actions are VERY precise and very monetary motivated.
You pay an incredible amount for Google's search engine results - if you have ANY website online.

I'm very much in approval of Google's Business ethics - but not for one moment do I think they are doing anyone a favor for free. I think Google is one of the smartest, most competent concepts in the world. Wish I had the brains to think of something so "HUGE" on a worldwide setting. If you have no clue about Google and economics, then you probably should do some research, it is quite fascinating.

They also do need to be held accountable for what they do provide - and what they do create and change.

If you're not aware of it - Google has literally changed the way the world operates in many sectors.

Now... You say FREE traffic.

Not one piece of traffic is free.

Most sites worked hard for their rankings - or to be where they are in order to get that traffic. That's hours of labor - love and devotion.

Google provided the platform. Creating the absolute need.

That's like saying... uhmmmmm ok... here... an analogy for you:

Driving down the interstate - all of a sudden you drive into a huge ditch that just suddenly appeared.

You've been driving down that interstate for years - ditch wasn't there last week.

No signs were put out about the ditch.

State runs the interstate... but you don't live in that state - so you don't pay for the interstate via taxes.

You have the right to drive there - but you didn't pay for it... But ---- somehow there is a level of expectation of safety... Right?

So --- now you're sitting there with a smoking engine in the ditch, crumpled hood and damaged frame - complete wreck of a car that worked fine last week...

Who is to blame for the loss of the car?

YOU??? Because you are driving for free on an interstate?

Google has become a state... in a way. They provide much like the governments provide. What they regulate - how they do things is very precise. If you EVER think that it is for the sole benefit of the people, you need to take off your rose tinted glasses.

Every calculated move is designed and engineered to produce financial results.

Because they have created the interstate - they have drivers running every which way right now trying to figure out which way the new road goes.

Do they have the right to do so? Absolutely - they have absolute domain over their kingdom.

However - every kingdom should be ruled effectively and efficiently. And sometimes changes should take the "little people" into consideration.

Sites have driven the interstate and have put up roadside produce stands because Google wants them to operate them... Because they want people to USE Google - so that Google can sell advertising - etc. Much like billboards behind the fruit stands.

But... whooops... Google just put out a bunch of potholes and ditches in front of perfectly good produce stands --- because there were a few con artists washing windows and selling bags of stolen oranges for a buck.

Should sites use Google to depend on the traffic? Yes - duh... isn't that why advertising is done?
Should sites be upset because they lost traffic? Yes. Absolutely
Should Google really pay attention? Yes - but will they? I don't know it will depend on the revenue increase or decrease from this.

I suppose if every webmaster who was hurt by this decided to stop adwords campaigns - and convinced every friend to do the same - then there might be a change...

But - since such a coup will never happen - it really doesn't matter what I think.

Do I think Google should roll back the change? No.
Do I think Google should fix individual sites that have been inaccurately demoted? Maybe.
Do I think it will be possible to do everyone? No.
Am I realistic about it? Yes.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/6/11 7:47 PM
I rarely see any website that Google has gotten rid of that didn't deserve it. I'd say Google has a 99.9% accuracy when it comes to pulling the plug on crap content.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/6/11 7:49 PM
Your whole argument falls down on the basic conecpet of:
How are you actually paying any money to Google to get that traffic from the organic search results?




Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 7:53 PM
Still SEO -regardless if Google is right (or GOD SUPREME - whatever - ) doesn't matter.

I still stand by my original rant. I think you've been very unkind to people in the webmaster's forum.

You're supposed to be here to help webmasters. That means people from all walks of life... all levels of skill. Every one of them an individual and every one of them thinking they've worked hard.

From the con artist to the common webmaster... They come here to get help. They all think they've done right - but are coming here because it's where they are supposed to learn what they have done wrong or what they could do better.

Most times you jump all over them...

Perhaps you've forgotten what it's like to be new - or to have feelings in a forum.

Maybe you're burnt out by the scammers who have come in and been rude.

But this is supposed to be a community - and instead of inviting people in - and helping them to get better - you're literally scaring them away - and forcing them to go to other places --- or making them dislike Google.

You're the face of Google... Everyone who answers a question or a comment is the face of what Google is supposed to be.

The way you've handled some people makes me feel like you think they are worthless scum sheep that deserve to be stepped on and have no value.

You don't take baby steps - or communicate. That's just not fair to the average new person coming in.

Perhaps you could care less that I'm telling you these things - in this discussion - but I'd like to say that I tried to make a difference. Maybe enlighten you. You seem to have the knowledge of HOW to help people - your method just seems to have gone off somewhere.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 8:05 PM
"I rarely see any website that Google has gotten rid of that didn't deserve it. I'd say Google has a 99.9% accuracy when it comes to pulling the plug on crap content."

OK Lysis - what about DaniWeb? Do you think it deserved to be relegated to the back ranks? I don't - it has a lot of information, good links, it's a really decent site, original content...

I wish that SPAM sites would be gotten rid of.

For example: My site was YANKED out of the search engines - I'm not sure why. But looking at the site - it's got good info on it - it's got good articles and it's brand new. Now it doesn't even rank at all --- but hey - I'm fixing it up... Still it did NOT deserve to be YANKED completely out - we had to have it reindexed. It might have had some keyword stuffing at first - but it doesn't now - yet - Google determines it to be not worthy of being anywhere near the keyword search results... Because it's flash - and it only sees it as ONE page of text...

Which in the industry I'm in - there isn't a lot to say about what we do that doesn't include multiple uses of the keywords... Sorry - just a fact of life... but our site isn't spam.. It's a service site. And it looks pretty decent as a site --- without looking at the cached text version which is skewed... I have to work on that.

But still - it shouldn't have been yanked out of the search engines - there are sites far far far far far BY MILLIONS far - worse than our site that are sitting prima donna.

And Dani Web is a well done site - but it was dropped significantly...

I think that while Google is GENERALLY right - that there might be some issues - and there are some casualties.  

But thank you much for the conversation. I have read a lot of your commentaries - and you're usually pretty good with your responses to people and I think you're a good conversationalist - as well as someone who tries to help others out.


=========================================================        
"seo101
Level 13
7:49 PM
Your whole argument falls down on the basic conecpet of:
How are you actually paying any money to Google to get that traffic from the organic search results?"


OK - first time I've seen you actually try to converse here...

Time is money. The amount of time I spend building a site - learning how to make it work in a search engine is MONEY.

Do you work for free at your job?
Most people get paid for working - some of us volunteer our time for pleasure or some reward... But that pleasure or reward is payment.

Google created the road...
People create the cars to drive down the road...
People create the "produce stands" to sell items or services
Google uses those places to sell advertising

It's kind of like symbiotic relationship - and neither are relativistically free.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/6/11 9:42 PM
DaniWeb? Do you think it deserved to be relegated to the back ranks?
 
 
Yes.  Unfortunately Dani has been pretty clear that if the ads didn't exist, the site would have never evolved to the size that it is.  Since Dani has a tremendous amount of redundant content and all that content contains ads then Dani is going to have problems not fitting the basic definition of a Content Farm.
 
 
Editorial/Commentary/Informational sites are going to have decisions to make.  Where/how is the revenue going to be generated?
 
 
Take a peak at the various CE (consumer electronics), Audio/visual, camera etc forums
 
5 years ago these site were just about 100% advertising revenue.
 
Sure, sure some of the camera fan sites sold T-shirts or odd ball camera accessories.  One of the camera fan sites was purportedly the only source of official replacement lens caps because the manufacturer sold the inventory to the fan site for resale.  But as a group, that was it.
 
Over the past 5 years, especially in the past 1-2 years, the number of CE, AV, Camera forums with their own shopping carts has sky rocketed. 
 
The exception is DPReview... which is owned by Amazon and has buy now from Amazon all over the site.
 
The point is pretty simple, what is your revenue plan if the advertising is insufficient?
 
 
 
 
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 9:56 PM
OK Stevie D - see... that's precisely the issue.

Google took away from advertising companies all over the world. Commercial Television, movies, Phone Books, etc. It literally killed many industries that are just now actually realizing they are dead - and they still haven't woken up.

The mini companies - people - created websites and have used that advertising revenue (just like the other companies did before Google sliced their throats.) DaniWeb is no different than those companies.

It has become a competitive market to be in the advertising business ONLINE now.

So - instead of letting the little guys roll with it - Google has now plummeted the sites that don't have the teams to back up the SEO - and has provided the big companies - and Slurpers with the top spots.. Perhaps not on purpose... But that is what the results are appearing to be.

In an attempt to kill the blackhat - spammers and content thieves - Google has instead killed the originators.

Which is fine if that is how the ball rolls. But to say that Google didn't cause this is ridiculous - and to say that Google didn't create the revenue stream and then just arbitrarily take it away is also ridiculous.

To say that advertising is NOT a revenue plan is silly - because that's what Google itself makes money on.

Which means simply that Google made a pre-emptive strike. It needs advertising funds - and it just knocked out about 11% of the companies who have advertising revenue - which - hmmmmmmmmmm let's guess where that revenue might go... and 11% of revenue is HUGE...

I'd like just .011% of the revenue they're going to get from this move.

I'd be a multi millionaire.

Google - wanna put it in my bank account? *GRIN*

Anyhow - what I'm saying is that DaniWeb and several other sites just plummeted because of a move Google made. I don't think DaniWeb is worse than the sites that are now ranking where they were.

It's a matter of perspective, I'm sure.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/6/11 10:42 PM
> I just saw one of your banner ads Dani! It's a step in the right direction, although I don't know if you've always advertised.
Yes, we have.

> Unfortunately Dani has been pretty clear that if the ads didn't exist, the site would have never evolved to the size that it is.
As I've mentioned before, I ran the site for quite a few years without ads, and it grew to an amazing size that way.

> Since Dani has a tremendous amount of redundant content
Why do you say that? I keep saying that we have NO duplicate content at all.

As I've mentioned, I grew the site from the ground up, while I was in college, and with no advertising on the site. It grew to a tremendous size. There was no revenue model, however. Upon graduation, the website was large enough that I was able to turn it into a fulltime business by using AdSense. It's now been quite a few years later, and I've just recently started selling advertising directly as well.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content omecha 3/6/11 10:49 PM
Check out
http://google-algorithm-ruins-my-business.blogspot.com/

Google's monopoly position should get attention. 15 yrs ago, Microsoft got the monopoly position in the world of PC, they almost get split in 1999. In this new era, we should have law to regulate internet monopoly and unfair competition.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/6/11 10:53 PM
omecha - how did it actually ruin your business? You chose to base a business on getting free traffic from Google. Any business model based on getting that free traffic is never going to work in the long term (this and every other major update Google has done shows that). You got some free traffic, now you complaining that Google no longer gives you that free traffic. You ruined your own business by have such a bad business model.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 11:12 PM
Again - Nothing Google does is free - it uses the content of the sites to place ads and create an ad revenue stream...

Google is not some knight in white shining armor kissing babies and handing out free lunches.

gaaaaaaaaaaaahhhhhhhh.

Can we say brainwashed here?

Again -GOOGLE CREATED THE REVENUE MODEL - GOOGLE PROVIDED THE REVENUE MODEL TO SITES SO THAT GOOGLE COULD MAKE MONEY... GOOGLE ARBITRARILY CHANGED THE REVENUE MODEL OVERNIGHT WITHOUT WARNING... GOOGLE IS GOING TO USE THE 11% BASE OF SITES THAT FAILED ON THIS TO GAIN MORE ADVERTISING REVENUE... Even if only 1 percent of those sites seeks adwords --- what do you think the revenue stream will be?

Come on - do the math please...

how many sites does Google have?

How many of those = 11%?

How much is one percent of that?

Sites will be desperate to pay to get to the top. 5% will give up - 5% will hire people or work hard - 1% will pay for adwords to get top position.

Hmmmm let's think - how much money is that?

phew...

did I scream that loud enough?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content omecha 3/6/11 11:21 PM
seo101,  Let's make an analogy with Microsoft 15 years ago. ref http://en.wikipedia.org/wiki/United_States_v._Microsoft  The plaintiffs alleged that Microsoft abused monopoly power on Intel-based personal computers in its handling of operating system sales and web browser sales. That time Netscape almost went bankrupt. Why other company get "free" business by install their software on DOS/Windows,and they can still complain Microsoft? Does all of those businesses are bad model? Why the plaintiffs can allege Microsoft abused monopoly power? Why EU can allege Microsoft for monopoly and unfair competition. Is this exactly the same for Google over internet today?
This is why we need law to regulate monopoly over internet.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content seo101 3/6/11 11:23 PM
Google are not remotely close to being a monoply, so your argument carries no weight.

@lyrical - all my sites (except 2) are up since the algorithm update. For every site that goes down another goes up.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 11:28 PM
SEO - that's great.

I hold no ill will - I just wish you'd treat people nicer.

I'm cool that you have blind loyalty  - that's da bomb. I could use people like that in my field.

And I am cool that you and I don't agree on things - that makes good conversation.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/6/11 11:34 PM
No, it doesn't
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 11:47 PM
Well since opinions are all relative anyway - nice to meetcha cuz.

Are you a kool aid drinker?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/6/11 11:52 PM
Actually you don't drink the kool aid - you're one of the nice people in the forum. Please remain that way. I've read your comments to people and I like the way you handle them.

It's very professional and polite - with kindness and honesty behind the answers.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/7/11 12:51 AM
Thank you but it adds nothing to Dani's thread.

They let anyone push their own barrow here. There is no need to commandeer someone else's thread.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Immys 3/7/11 2:47 AM
The more changes Google makes the more SEO-driven sites - maybe sites like those of "seo101" - come into the Top 10, as we´ve seen in the last months, maybe not directly after an allgo-change, but short time after that ... as those who are not perfect in SEO - but often useful - fall back.... in the end we have only SEO-Projects ranking.

Too many SEO-people are having an eye on every move and every patent of google..  so all the changes google makes are finally resulting in "no change" or are making the results even worse, costs only negative PR and trouble for google.

And till now, I must say that this thread is "crappy content" .. nobody helped Dani... I thought this is an expert-forum?!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 10:14 AM
I did some more tweaking yesterday to further improve page speeds.

If I haven't mentioned it before, we used to receive about 300,000 pageviews daily. Before the algorithm change, about 90,000 pageviews were US based. Since the algorithm change, about 50,000 pageviews are US based. Therefore, our US-based traffic has been cut in half, and we now receive about 250,000 global pageviews per day instead of 300,000. (As of right now, the algorithm change has not been rolled out to the rest of the world YET).

Nearly all of our traffic is long tail -- Google search results to individual forum threads and articles based on niche keywords. We rank for hundreds of thousands of different keywords. I do not keep track of what they are because I focus on having quality content for our audience, based on what our audiences is interested in. I do not base what topics to write about on how much keyword traffic different topics can pull in. Therefore, while I never paid attention to what keywords we ranked for, or what keywords had the potential of ranking us better or worse, I have recently started looking at this stuff in Google Webmaster Tools.

And what I found made me angry. Here is the BIG problem that I have with Google's algorithm, in a nutshell.

Before the algorithm change, we used to rank #1 in Google's search results for "android tablet". The article which ranked #1 was this one: http://www.daniweb.com/reviews/review327951.html ... It is a very complete, comprehensive round-up of all different makes and models of android tablets, extensively tested by our in-house staff writer. It consists of opinionated product reviews about various android tablets on the market. There are no "manufacturer specs" that have been stolen from the manufacturer's website, etc. 100% of the content is unique, and our own staff writer played and tested each one.

We have since fallen to page two of the search results for this query. Some of the results that have delisted us make sense: Blogs devoted exclusively to Android tablets. That makes sense considering Google is putting more weight on niche sites that are an authority on the topic. For example, I can understand where Google is coming from when they say that an article on a site exclusively dedicated to Android tablets is probably more of an authority on the subject than a single article on a less-niche technology website.

However, then let's examine the other sites that delisted us. Our article was replaced in the results by items such as this: http://www.engadget.com/tag/android,tablet ... There is no usable content there. It's a mashup.

And then let's look at the new #3 spot in the SERPS: http://androidtabletcentral.com/ ... C'mon, really?! Google considers THIS a trustworthy, reputable, authority site?

And now let's look at the new #2 spot in the SERPS: http://tech.fortune.cnn.com/2010/08/06/attention-kmart-shoppers-149-android-tablet-on-aisle-5/ ... Google considers an article that does nothing but promote tablets for sale at Kmart on CNN Money.com is more of an authority on Androids than a technology publication?!

It just goes downhill from there, so I'm going to spare the details, because I'm getting angry just writing about it.

- - - - - -

Now let me present you with another equally pittiful example. As I've mentioned, we still get a decent amount of traffic from Google. Therefore, for all of the keywords that we no longer rank for, there are a handful that we actually started to rank BETTER for since the algorithm update.

An example is the keyword "rectangle". Yes, 'rectangle'. If you do a Google search for 'rectangle', then this forum thread made it to about #3 in the SERPS as a result of the Google algorithm update: http://www.daniweb.com/forums/thread91156.html

Now let me ask you this ... WHO IN THEIR RIGHT MIND would be Googling for rectangles and actually be looking for a complex C++ algorithm? A kindergarden student?? Thanks Google, but I could do without this useless traffic that is completely untargeted to my audience.

What's even worse is that we just recently started ranking for this keyword, so it is a result of the algorithm update.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/7/11 10:36 AM
 
doesn't have an "advertise with us" tab on their website.
 
And androidtabletcentral is selling the products they list.
 
 
Personally, I would never buy anything from a crap site like androidtabletcentral, but the simple fact remains.... they are selling something other than ad space.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Puntsy 3/7/11 10:44 AM
Um... Android Tablet Central is an affiliate site. They don't even sell anything on their own site. This whole ad theory is garbage. Their entire site is basically one big ad/funnel for another site... this one: http://www.merimobiles*com/ProductDetails.asp?ProductCode=MERI0541&Click=3102

Maybe click around before saying the site has no ads. They don't need an "advertise with us" sign... they've already got enough ads. And their "live chat" is fake. They're probably another goober who thinks having a live chat is some kind of trust signal. Seriously... they're selling something other than ad space - and that's affiliate links and rev share. Not content and not products.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Puntsy 3/7/11 10:47 AM
Okay wait... finally someone came on their live chat. It works, but I dare you to ask them a technical question...
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Farmer Effects 3/7/11 10:59 AM
Are you serious?  http:// androidtabletcentral.com/ Is so spammy.  If I encountered it, I would run away from it as fast as I can, just by the looks of it.  Now to me, this just looks bad. DaniWeb has a true community, looks well designed, looks trustworthy.  Any HUMAN can see that!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 12:07 PM
Android Tablet Central is still the only site in the list that sells something other than ad space. That still doesn't explain why a fluff article on CNN Money.com is considered more of an authority on Androids than DaniWeb, a respected **technology** publication. Or any of the other publications that sell ad space pushed DaniWeb from the #1 spot all the way to the second page of the SERPS. (I think we've climbed a few spots up since the algorithm update last week).

And then ... c'mon, ranking a C++ algorithm for the keyword 'rectangle'? Sorry, but DaniWeb doesn't target kindergarden students.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content squibble 3/7/11 12:52 PM

Please excuse, I have not read all this thread. I think that it would be helpful to focus on your site and not of others - that will be more productive use of time.

Though looking through your site, I wonder whether you will find a smoking gun.

I would look at everything. Like >

http://www.google.co.uk/search?&q=site%3Adaniweb.com%2Fnews


Is this really good content that your users know you for ?  Does it bring you in traffic ? My questions are rhetorical..


Consider blocking google from crawling search results>

http://www.google.co.uk/search?q=site:daniweb.com/forums+inurl:search.php%3F&filter=0

--

I searched for your post content eg > Php and mysql creating a table and PHP Multiple row edit and your site returned on the first page. It did not look filtered to extents like can be seen at times of trouble.

When I search for your content like this >

http://www.google.co.uk/search?&q=site%3adaniweb.com%2fforums+asp

http://www.google.co.uk/search?&q=site%3adaniweb.com%2fforums+php

I see older and tag pages at the top of the results, I might not expect that. Is this normal for your site ? It might be, but also it might be indicative of something - but the way that your site is structured - it is hard to drill down further to see deeper content - but you would know more yourself the content which is there .. or better ways to find it.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 1:25 PM
squibble, thank you for your productive contribution to this thread :)

>> http://www.google.co.uk/search?&q=site%3Adaniweb.com%2Fnews
>> Is this really good content that your users know you for ?  Does it bring you in traffic ? My questions are rhetorical..

Well, it is not very useful, considering all of the results at the top of the SERPS seem to be for very old news stories that are no longer relevant. However, if you do a Google News search with the same query string, the results are much more timely.

I am VERY surprised that all of those search.php?searchid= results came up. There are no links to any of them on the website, so I'm confused as to how Google found them in the first place. In fact, we haven't even used that URL structure for about a year and a half. Our current URL structure is www.daniweb.com/forums/search12345.html ... What is the proper way to exclude files like search123.html and search 246.html from my robots.txt file? Can I just do www.daniweb.com/forums/search as a partial URL?? Would that work for all search engines (Google, Yahoo, Bing, etc)?

>> I see older and tag pages at the top of the results, I might not expect that. Is this normal for your site ? It might be, but also it might be indicative of something - but the way that your site is structured - it is hard to drill down further to see deeper content - but you would know more yourself the content which is there .. or better ways to find it.

I am confused by what you mean here. When you search for 'ASP', navigational pages for threads tagged with ASP seem to dominate the search results. As mentioned, I have never really paid attention to what keywords we rank for in the search engines, or actively pursued SEO anytime recently (Aside from ensuring we are following all of the webmaster guidelines and best practices). However, from what I can remember, navigational pages showing up like this was not always the case. Actual forum threads that were relevant to ASP (or the topic at hand) used to rank highest. Not the navigational pages.

I like the fact that Google crawls the navigational (tag) pages, because it is a good way for Google to find all of our content that revolves around a particular topic. Therefore, I do not want to block these tag pages in the robots.txt file. However, how can I tell Google to not weigh them as highly as actual thread pages? I am already using a sitemap listing all the forum threads, and setting a higher priority to threads with a lot of comments, editorial articles, etc.

It was mentioned earlier in this thread that I do not wish to remove our tag cloud functionality, or tagging feature, as it is widely used by our community. Therefore, removing it would be doing EXACTLY what Google advocates against, which is designing a website for the search engines instead of for human beings.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 1:33 PM
I've just gone ahead and blocked http://www.daniweb.com/forums/search.php?searchid= in the robots.txt file. How did you find that all of those bad URLs were indexed anyways? As I've pointed out, I haven't used that URL structure in forever!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content squibble 3/7/11 1:36 PM

I would just put a noindex tag on the search page and forget about it.

The reason I mentioned the searches for your content, it is not relating to the actual existence of your tag cloud - more as to why they appear highly, though.... matters are of opinion in this precise instance.. but mine is that I might expect for your tag pages/older not to be high. But I cant see what is at the bottom because there is not a good way of filtering through your urls. (I know this is a duff point.. but it would have helped to have split the directories up a bit to be able to drill-down in times like this. (eg forums/php/thread))  

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 1:59 PM
None of these navigational URLs at all are in our sitemap. We set a priority for all pages that ARE in our sitemap.

That being said, our current URL structure was created about five or six years ago. I have been wanting, for at least the past two years, to update our navigational structure into www.daniweb.com/php/forums/123 and www.daniweb.com/java/tutorials/456.

However, I never did it because I've always been scared to lose our high rankings by redoing the entire navigational structure on a site that performs awesome in the SERPS. Don't fix what ain't broke. However, from a usability perspective, I can DEFINITELY see the benefits of being able to tell, from a URL, where a page fits in the overall site structure / hierarchy.

I am thinking that now is the time to just bite the bullet and do it. Things couldn't get much worse right now. And, with all of the other changes I've been making lately, this would be yet another one which would put us in a better position in the long run.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content squibble 3/7/11 2:05 PM
Well.. I would only do it if you were going to do it anyway. I would not do it just for this purpose as it would take weeks/months to get any real info. There must be other ways of drilling down into content. Look into your databases perhaps - or users who know your site well (like yourself) could probably find areas which, on reflection could do with review ?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 2:08 PM
> Well.. I would only do it if you were going to do it anyway.
It has been something I have wanted to do for a VERY long time because, from a usability perspective, there is a huge benefit to urls such as daniweb.com/software-development/java/threads/123 and daniweb.com/software-development/c/tutorials/456. I've just been scared to do it, because I always had too much to lose.

What is currently great is that Google currently understands our navigational structure, even though the URLs don't express what it is. If a forum thread appears in Google search results, a correct breadcrumb appears by the URL.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content squibble 3/7/11 2:11 PM
I would probably do now then, whilst you are redecorating anyway.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 3/7/11 2:12 PM
Hi Dani, I would slow down with the changes, keep analysing your site and compile an action plan, just don't implement it too quickly out of desparation.
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 2:30 PM
While I would be the first to admit that I am desperate, I don't think that ANY of the changes I've implemented have been out of desperation. Every single change that I have made up until this point --- and there have been A LOT --- have been for the greater good of the site, as a whole, in the long term. There's nothing that I've changed that I would at all consider rolling back if it didn't improve the current situation. Additionally, just about everything that I'm doing has been for the benefit of usability with the end-user in mind ... not just tweaking stuff around for spiders.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/7/11 2:59 PM
If I go to the front page of your site and click on a "news" story, I am taken to the story and I then see a breadcrumb that leads me back to....a forum.  THAT's kind of odd.

If I trim the URL of the "news" story down to just "daniweb . com/news" I find a news page that links to all the news stories, but there is no way to get to that news page.  THAT's kind of odd.

If I browse one of your forums, I see thread...thread...thread..."news" story...thread...thread.  THAT's kind of odd.

The fact your old news stories are showing up as the most important in a site search implies to me that you're confusing the search engine.  I suggest you consider breaking your news stories out from the forum listings, give them their own URL tree, and maybe consider embedding a news widget in each forum that recaps its its associated news in a sidebar.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 3:29 PM
Hi Michael,

I think the single most important thing that sets DaniWeb apart from all other IT sites out there is that our editorial content is embedded within the same timeline as our forum threads. From our mission statement: " ... we break content up by topic instead of type so you don't have to care whether something is classified as a tutorial or a white paper ... like you do in most other online publications. Just choose the topic you're interested in and everything we have available about that topic is displayed in one single timeline, be it a Q&A forum thread, news story, product review, code snippet, tutorial or whatever."

The idea is that I HATE sites that break content up by type, such as news, reviews, forums, etc. If I am interested in PHP, I want to find everything the website has to offer about PHP, and could care less whether this particular website classifies it as a tutorial, white paper, or news story. I'd end up finding myself having to check ten different places in the website in order to find everything the site offers about a particular topic.

What DaniWeb does differently -- and what sets us apart -- is that the editorial is meshed in with the UGC.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/7/11 6:35 PM
What DaniWeb does differently -- and what sets us apart -- is that the editorial is meshed in with the UGC.

I'm in no better position to tell you what is holding you down than the next commenter, but at some point if you change everything else and nothing improves, you may have to consider that being that far outside the normal expectations could be a problem.

You're short-circuiting your Website's opportunity for crawl.  Maybe that's part of the problem, maybe it's just another problem, maybe it's not a problem at all.

it's something to keep in mind should nothing else help you out of your present situation.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 8:00 PM
About a half hour ago, I finished redoing our new URL structures for the forum listings, forum thread and editorial pages. I did not 301 redirect the old URLs (I'm afraid to, so as of right now the pages can be accessed from both old and new URLs) but I updated the link rel="canonical" meta tag to reflect the new URL.

I am going to do member profiles and tag pages tomorrow.

Is it okay that I'm not 301 redirecting the old URLs? Is the canonical meta tag good enough to specify which URL I want to be used?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/7/11 8:17 PM
Update: I just added 301 redirects
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content phpld 3/7/11 8:47 PM
I don't want to scare you but redoing a lot of urls and overall making a lot of changes on a site has a history or losing rankings. I personally did it anyway because I was concerned about the quality of the site for the long run.  We went through a stretch last year where google had dropped our rankings and we were seeing some declining sales. I hated it at the time, but we thought about what we could do to be successful despite google.

You've 900,000 users. It's time to call on them for support. Have some contents. Send them an email. Expand your advertising program. Roll out that new section of the site. You say you are not an ecommerce site. With 900,000 for goodness sake, roll out a couple of products!

We did something like this, and we are now doing BETTER even through our traffic did not recover. We even have a number of areas of our site that google can't spider that are really busy. In our case we are the #1 script in our niche, but google seems to think we are about 14th. That's too bad, but we've moved on. Maybe google will change their mind, but we have stuck with quality as our goal. Sometimes google fails to see quality. We are still getting customers that choose a higher ranked script, get bad support, and then come to us second.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content phpld 3/7/11 8:53 PM
Had a look around. Would "Link to this page with a member certificate" be possible Reciprocal Linking Scheme? I don't the scale of it, but I could imagine at least the possibility of a penalty.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Celebird 3/7/11 9:05 PM
the website's content seems tailored to a community
of users rather than search-engines or traffic per se --
but there has been a noted traffic drop and the
analysis seems geared toward getting back
the traffic and correcting the unfair hit.

if so, doesn't that necessarily mean refocusing the
original goal of the community and tailoring the site
for a specific search-engine and its generated traffic?

how are the effects of any new changes to the site being measured?
what result would constitute a satisfactory improvement?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/7/11 10:13 PM
CELEBIRD --- hits it on the nose in 3 paragraphs or less.

Ba da Bing --- Ba da Boom...

Google forces good sites to devolve into SEO schemers in order to maintain ranks and beat out the current top position holding scum.

Ding din ding ding --- refocusing of the sites to meet the needs of Google - not the people... Ding ding ding.

Dani - good work - I see that you're doing everything in your power to do the best that you can to regain your ranking. I truly hope it works - your site is pretty decent.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/8/11 9:02 AM
phpld,

I know that I am most likely going to lose even more rankings by changing the URL structure. However, it's something that I've been wanting to do for a very long time now, from a usability perspective. I was just afraid to do it b/c we were ranking so well. I finally felt like right now is a good time b/c there's barely anything to lose. It really couldn't get much worse! And, in the long run, the site will be so much better for it.

Everything that I am doing, I am doing for the users and for the community and good of the site. I have changed a lot -- but everything has usability improvements associated with it. I haven't changed anything that I did solely to hopefully get my rankings back, or anything I would roll back if it didn't work.

- - - - -

>> how are the effects of any new changes to the site being measured?
>> what result would constitute a satisfactory improvement?

I am desperate to get our old rankings back from before the "Farmer" update, when we lost about 50% of our US-based traffic. That is the primary end goal, in order to keep the business afloat right now. However, everything I am doing is not just for SEO, but also for usability enhancements. I am also considering a "satisfactory improvement" any decrease to bounce rate, increase to pages per session or time on site, increase in signup conversions, more posts contributed, etc.

- - - - -

LyricalQuestion,

I am REALLY trying very hard to stay focused and true to my beliefs to not do anything solely to please Google. Everything that I've done has had the members' well-being first and foremost.

There was, however, one thing that I did do exclusively for Google. From what I understand, a big problem with why innocent sites are getting caught up in "Farmer" is because Google cannot differentiate between a site that copied original content, and the source of the original content. Therefore, if bad sites scrape our content, they could rank higher for OUR original content than us!!

I realized that one of the problems we have is that our RSS feeds do allow you to pull entire articles, instead of just abstracts. Therefore, a scraper site or mashup site could pull DaniWeb's RSS feed, and republish our articles on their site. Google might rank THEM higher than US! I went ahead and changed our RSS feed to only allow abstracts of articles, and not the full-lenth articles to appear in the feeds. I did this yesterday.

Unfortunately, after 24 hours, I decided to roll it back. It ended up being that it did negatively affect our userbase. A lot of our members read our articles in RSS readers instead of on the actual website, and they were no longer able to do this. Additionally, moderators use RSS feeds to skim the latest posts contributed to help weed out spam. This was a case where I originally thought that I coudl help Google without harming our audience, but this simply wasn't the case. Therefore, you can now once again see our complete articles in RSS feeds.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 3/8/11 1:58 PM
FYI, I did a search last night and DaniWeb came up #2
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/8/11 6:42 PM
Was the result helpful?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content O-Dog 3/8/11 7:47 PM
My cents and all, I see that my {} want stuff that they can download, mod, plug and play... not someone telling them where they might download stuff to plug and play. I dunno.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/8/11 9:29 PM
HUH??
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/8/11 9:44 PM
It made sense to me Dani but checking it raises another question. 

Is the link below the type of article that you should be running? I do not know all that much about it, so I am probably wrong and it is probably not contrary to guidelines but I think publishing articles like this one is like walking along a cliff-edge.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/8/11 11:32 PM
"I realized that one of the problems we have is that our RSS feeds do allow you to pull entire articles, instead of just abstracts. Therefore, a scraper site or mashup site could pull DaniWeb's RSS feed, and republish our articles on their site. Google might rank THEM higher than US! I went ahead and changed our RSS feed to only allow abstracts of articles, and not the full-lenth articles to appear in the feeds. I did this yesterday."


Daaaaaaaaaaaang....

I was applauding that move - and then realized what a change that might have been.

That was a brilliant thought tho...

Hmmmmm... Do you, by chance, have a Copyright posted with every article? As in Article reprint with permission, ask the author, etc?

I know that sounds like a strange thing - but you could put out that only 30% of the content can be copied without giving linkage back to your site... That might help you - I'm not sure... but it was a significant thought. Because if every site is going to copy you at least you can gain the linkage. (which more often than not is good - based on the quality of the content - at least in my experience... I think there have been bad cases - but if people are going to scrape your content - and it affects you negatively - DMCA them.)
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/8/11 11:34 PM
Shop - not sure what you mean - it's a very good article review put in - and it has valid points in it... Don't see the cliff edge you're talking about.

From a content point of view - it's good.

If you're looking at the backlinking to the other site - I don't know - not going to go figure that out - but I'm not sure what you're looking at?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/9/11 1:08 AM
   
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/9/11 1:24 AM
OH... Gotcha - as in cliff for is this ok in the legal world sense kinda thing...

Uhm - in the tech world there are a lot of things that are like that - uhmmmm downloading and file share --- but their still listed in google - etc.

So - as far as an article about them - well - they can be used for things that are NOT illegal (although we know better - *GRIN*) --- but the article is a good one.

So - content would be ok - at least I think it would be. As far as editorial wise - the article is well written, very spunky - and not super over loaded with keywords.


As far as whether the item itself is ok - dunno..
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/9/11 2:10 AM
There is no potentially illegal content on the site but potentially illegal content is linked to. I have no idea if that means anything or not.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/9/11 12:35 PM
>ShopSafe
 
Yes, linking to bad neighborhoods or certain "bad" sites is suppose to be a major no-no with a significant PR hit.
 
What is a bad neighborhood?  Who knows for sure, 4chan maybe?
(unknown) 3/13/11 6:56 AM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/13/11 10:44 PM
Hi everyone,

My apologies for not posting to this thread for a few days. I have been busy working mornings and nights on the site, mostly finishing up restructuring all the URLs, fixing some bugs that had previously stumped me, and trying my best to improve the user experience overall. Unfortunately, there has been absolutely no increase in traffic as of yet. I currently have high hopes for the new URL structure though, as I really think it goes a long way to understanding our site navigation hierarchy.

Unfortunately, to add insult to injury, Google AdSense closed my account last week. I am just being hit by Google from all angles!! I had previously been selling all my US advertising space in house, and pretty much selling out. International traffic is notoriously hard to monetize, so I had been relying on AdSense for that. When we lost our US traffic, nearly all the revenue from all of our in house ad campaigns went right down the drain. We were then relying exclusively on AdSense for revenue, which previously only accounted for about 20% of our overall gross revenue. Suddenly, that 20% was the only monies we were bringing in. Now, our AdSense account has been suspended, and we are bringing in absolutely no revenue.

It actually is a rather long and silly story why our AdSense account was abruptly terminated. To make a long story short, it happened on Thursday and I was able to reach out to my rep, and we are getting the situation resolved, but unfortunately due to communication between multiple time zones, the problem wasn't able to get resolved in time on Friday, and now the weekend got in the way. I am hoping to have at least a little bit of money coming in again either Monday or Tuesday.

In the meantime, it doesn't help much with our $8000/month server bills, not to mention office carrying expenses.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/13/11 10:47 PM
HOLY TOLEDO... That's a high set of bills...

Good Luck Dani.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/13/11 11:12 PM
You don't know the half of it. I also have an office lease, a salary for a full time employee, two systems administrators on monthly retainer, and a new mortgage on my hands. And, as of right now, absolutely no source of business or household income whatsoever. All I've been doing is working 24/7 trying to improve the site. I can't eat. I can't sleep.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Farmer Effects 3/13/11 11:20 PM
Let's hang in there Dani.  Been going through the same process here, fixing what I can.  Hoping things will make a difference.  I'm also on the Google Panda diet, if you know what I mean....
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/13/11 11:27 PM
I'm trying. The hardest part is waiting anxiously for Google to recognize all the changes I've made. It's been a few days already and older versions of the pages are still prevalent in Google. There are loads of pages still in the index that have been added to my robots.txt disallow list for days. Additionally, I'm seeing a lot of duplicate pages in the SERPS. For example, when I google my username 'cscgal' I see two versions of my member profile in the SERPS, with the old URL and then with the new URL, even though I am doing a 301 redirect!

In fact, to give you an example of just how bad I'm doing in the SERPS, I don't even rank #1 for my username! My profile on a forum that I am not active in and in which I have not even filled out my member profile ranks higher!!!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content phpld 3/13/11 11:36 PM
Your website is beautiful in both look and usability. I don't understand the farmer update issue. I don't what has been going on with google lately, but small businesses are taking a hit, just like in the overall economy. I don't know if maybe google corporate is pushing for tweaks to help the big guys or what, but squeezing out the little guys who make up the majority of web developers is not going to fare well in the long run. I would like to believe that this is not their intention and they will be making some corrections in the very near future, so that people like you can be validated. If you did anything wrong, it would not be anywhere near the scale of what much larger companies did wrong in terms of SEO or similar.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/13/11 11:42 PM
I feel for you Dani. Try to interrupt the cycle with clear thinking time in a park somewhere, some things can't be found with a keyboard. (Don't fall asleep or you'll lose your wallet as well :) )

If it is related to the main issue, the AdSense thing might turn out to be a godsend. Here's hoping for you.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/14/11 10:49 AM
I'm trying. The hardest part is waiting anxiously for Google to recognize all the changes I've made. It's been a few days already and older versions of the pages are still prevalent in Google.

For what it's worth, it came out at the SMX West conference last week that Google is apparently giving less crawl priority to sites affected by the Big Panda update.  While that's not very helpful for sites that are aggressively seeking to get back into Google's good graces, it does make a certain sense.  After all, why should Google keep crawling what it deems to be low quality content as frequently as what it deems to be high quality content.

Hopefully the reduced crawl priority will go away once a site's revisions have been taken into consideration by the algorithm.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/14/11 12:21 PM
Google really just is coming at me from all angles, aren't they?

As a side note, I want to re-explore the tag clouds that we have. I am not concerned with them being spidered or not (I'm fine with rel="nofollow", robots.txt to disallow the entire /tags/ folder, etc) My concern is they really are valuable for our end-users and I am worried about it being considered keyword stuffing.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/14/11 2:10 PM
As a side note, I want to re-explore the tag clouds that we have. I am not concerned with them being spidered or not (I'm fine with rel="nofollow", robots.txt to disallow the entire /tags/ folder, etc) My concern is they really are valuable for our end-users and I am worried about it being considered keyword stuffing.

I don't have time to go back through the whole discussion to see if someone posted this.  Last August Matt Cutts did a video for Google Webmaster Tools where he warns against allowing tag clouds to become filled with too many words.  They can look like spam to Google.


Using "rel='nofollow'" won't take care of that -- in fact, it's a bad idea to use "nofollow" on your internal tags.  You want to let the PageRank flow.  Tag PAGES are okay as long as they are not duplicating other content.  Dropping the tag clouds or isolating them (such as in iframes) would be a better solution.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/14/11 6:19 PM
>> it's a bad idea to use "nofollow" on your internal tags
I am not. I am using <meta robots="noindex" /> on the tag pages. I don't want Google to index the tag pages themselves, but I do want them to follow all the threads on the tag pages to help them find new articles.

>> I want to re-explore the tag clouds that we have ...
It was quickly mentioned earlier in this thread that tag clouds are bad and to drop them b/c they can be considered keyword stuffing. But I really don't want to drop them. I find them extraordinarily useful. I was wondering if there are alternatives such as loading them dynamically via an AJAX call or something. Is this worth it? Would it be considered cloaking b/c I am injecting content into the page dynamically, therefore hurting the usability factor due to an extra server call and javascript execution, for the sole reason so that googlebot doesn't see it? I'm not happy with that solution either. AFAIK, Googlebot can follow IFRAMES.

All this being said, I like my tag clouds. I find them useful. I want to keep them for the users, I don't care if google doesn't see them. I don't want to have to jump through hoops because Google has given me the ultimatum that either I get rid of my tag clouds or I get no traffic.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/14/11 6:54 PM
Oh ... I see what you mean. Load the tag in an iframe, and add the path to all the iframes into my robots.txt disallow list. Doable ... but still would be an extra http request to the end-user for no reason. Silly google. I think I'm most likely going to go down that route. Just want to explore all my options first.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/14/11 7:11 PM
I think Michael has given you good advice (and he did not mention robots.txt). 

I have never seen Googlebot follow the path to the iframes, there might be no need to block it in robots.txt. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/14/11 7:20 PM
Stop and think for a moment about all these changes you are making for a moment.
 
How do you know what is correct info?  We don't.  It is all a guess.  One reason it is all a guess is because the line of demarkation always moves.
 
Keyword stuffing.  What ratio was allowed in 1999 is a whole lot different than 2004 or 2009 or Today.
 
And even so, we really never knew what the absolute top # of words was bad and what was allowable.
 
Why?  Because if we knew that 25 keywords to 100 words of text was allowable there would be 1 Million websites with exactly 24.99999 keywords per 100 words of text.
 
Scummy webmasters are always going to push the envelope.
 
So Google hides the ratios, the rules.  And the rules change over time.
 
 
On way Google can hide the rules is to install a sandbox for new websites and formerly penalized websites.
 
In this manner you can't cook up a website with 24.9999 keywords per 1000 words of text to discover the magic keyword stuffing ratio.
 
 
You are making changes to the site and expect instant results.  What if you did x or y or z  or Z+ABC and your site was magically restored in the index?    Would you keep the info to yourself, or would you broadcast the secret formula to the entire world to copy?  See Google's dilemma... let you discover the trick and tell others.... or to keep the tricks behind a sandbox and let you make enough changes that you will never know the trick when you come out of the sandbox?
 
Is Google being bad?  No. 
 
1) Their search engine is their sandbox.  We can play by their rules or take our toys over to Bing.
 
2)  Google publishes general webmaster guidelines that direct us to write sites for the viewer.  That is all we are going to get from them, simplly because they have never told us the magic ratios, tricks or whatever for any other website practice.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/14/11 7:33 PM
Peer-review. 

It is why this forum is so useful. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 3/15/11 3:17 AM
>> There are loads of pages still in the index that have been added to my robots.txt disallow list for days. 

I haven't followed this thread completely, but I don't think you should use robots.txt to remove any content.  It's doesn't remove content, it simply blocks access.  If you add Robots META NoIndex, and block access nothing will happen.

I'd strongly suggest you remove bad content ASAP, and this means NOT using robots.txt for this.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/15/11 11:41 AM
Yes.  Either use meta NOINDEX (which seems to be what Google is advising people to do at conferences) or use robots.txt to prevent crawling.

With respect to protecting iframed content, you DO want to block it in robots.txt because you never know when some idiot will link to the iframe page.

With respect to getting stuff out of the index, it's better to just go with the meta directive and a handful of patience because it will take Google a while to recrawl and assess all those pages.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 3/15/11 12:44 PM
Dani - I'm holding my breath for you - patience is what you need now... I know it doesn't help with the bills... But I think more changes could cause the pendulum to swing too swiftly - without knowing the exact results.

I don't know how often the bot crawls your site to make a difference in the ranking - and how fast you're seeing results.

Good news is - that you're documenting everything that you've changed. You're really working hard on your site.

You'll be one of the few people that understand completely what to do and how to do it - off of this algo change.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/15/11 5:24 PM
@LQ It's not a reality show, it's real life. It is obvious that you mean well but consider that chatting about it might not be helping Dani. 

Re: Robots.txt. Dani already has 49 Disallows. I think it is the wrong time to tell Googlebot to come back later.

I think this is the best advice and should also be applied to the iframe pages: "With respect to getting stuff out of the index, it's better to just go with the meta directive and a handful of patience because it will take Google a while to recrawl and assess all those pages." 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content SouledOUT116 3/15/11 6:01 PM
For what it is worth, a heavy qty of tag generated pages, with heavy pagination were removed today via the Crawler Access, area in WMTs. I have also retooled my XML sitemap as it was to large b4 and I had not noticed. New URLs were submitted and the directories that to me (potentially) caused the dilution of my site were removed and could not be found in the SERPs. I agreed with Google, the pages removed were of little value. I have tightened up my ship and I will post if I see benefits.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/15/11 9:05 PM
> Re: Robots.txt. Dani already has 49 Disallows. I think it is the wrong time to tell Googlebot to come back later.
Just about all of these lines have existed in robots.txt for years. I only recently added 2 or 3 lines to the list, all three of which are either behind a login wall or consist of millions of pages of member statistics with no valuable content. (aka it was an oversight that they weren't there to begin with)

I have been using the meta directive for the tag pages, ever since I stated that I was going to be doing that a couple of days ago. (Or so I think I mentioned it in this discussion thread.)

SouledOUT ... huh??
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content SouledOUT116 3/15/11 10:24 PM
Sorry Dani, to confuse you or be off topic.

I see redundant pages in my site as being tag pages. Yours are much better for users. But to give you an idea of what might matter or be seen and systematic content. Take a look at your page titles in your tag cloud pages as well as the headers and meta descriptions.

Thread Tagged with...
Tag Cloud for...
Thread Tagged with...
Tags related to...

Count them up, and you may agree they bend the efforts of having unique pages. Your PHP tag cloud page is one page that has 11 pages (pagination) of the same title, although with page numbers. I am of the opinion that these sorts of page may cause the dilution "effect" and potentially be seen and synthetic or the farmer type. Basic site health is to strive to have unique elements within a site. I have seen this in other sites that were hit with these characteristics.

Hope that makes some sense.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content drsmoothie 3/16/11 2:35 PM
Dani I see Google ads still be displayed on the site. I do wonder though the way they are being injected into the forum post leads me to believe they might violate TOS. Some are at the top some are in the mix of the post. Has the adsense account been reinstated? I am just now looking at the site to begin my study of the update as we came through this without any negative effects. The sites our company owns will be what I will use for my Analysis. I will be posting back after I am able to dig into this, but can you please let me know on the adsense.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/16/11 8:35 PM
Our AdSense account was finally reinstated yesterday, and we went live after final approval today. We have always had custom AdSense units, which lets us design them to our own taste, blended in with the site.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/16/11 8:39 PM
SouledOUT, our tag pages have the meta directive to noindex.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content drsmoothie 3/17/11 11:56 AM
Hmm I wonder then could the manual review of the site due to something with the adsense be a cause?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/17/11 1:53 PM
No, because the AdSense thing happened just a few days ago, I got in touch with my AdSense rep, and the problem was fixed. AdSense is back to loving me.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WendyPiersall 3/17/11 4:39 PM
Dani - I feel for you, girl. I'm not in IT, but you would have to be brain dead not to be able to tell your forum is an extremely valuable resource with an obviously vibrant community behind it My own craft blog has been hit - 150+ hand written original tutorials, I take my own photos, draw my own printables... It even just won a top craft blog award from one of the biggest parenting sites on the web. At least humans can tell the quality of the content is good.

But I'm not even close to being in the horrible boat you are in. :( I know you have made a ton of changes, but I wanted to ask you if you could list out the things you did that you feel made your site better for your own users. Also, did you uncover any true errors that needed fixing? I have made some very small changes to my site, but since I am in the midst of another (unrelated) VERY large project, my time to devote to recovering my site is limited (and I can only watch the income wash away until said project is complete).

So, what are the things you have done that you think have helped your site in the near term?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content TomislavG 3/17/11 4:51 PM
Just a quick note that might be usefull, one of our client sites gots hit by similar problems last summer. We endured months and months of agonizing scrutiny over the website and just what might have been wrong. Google had dropped the site to unbelievably low serp positions, indexing of the site seemed awkward and bot-sites that copied parts of the content through RSS would be dozens of positions ahead of the original site.

At around 40.000 news articles googlebot seems to have had severe problems with crawling and indexing and there wasn't a single warning, notice or anything that would tell us that. Just wild quirkiness.
We managed to solve the problem by splitting the sitemap in smaller parts. Very soon, all the traffic came back.

We might have found the solution earlier, but the problem started at the time of significant algorythm changes and for months we thought that was the reason.
It is frightening and sad how problems caused by google, not yourself, can trump your project, your job or career. And to top it all, you cant even reach them to ask a question or get any help.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content aaron_baugher 3/18/11 7:44 AM
> What would you do if your site sold..... oh I don't know.... Popsicles.  Frozen Popsicles.  Packed them in dry ice and shipped them across the country.  Then one day DOT (department of transportation) says that anybody shipping dry ice across state lines must have a permit due to hazards of dry ice and the permitting fee was 10x your annual sales.

You forgot to mention the part where first the DOT sold you the shipping software and hooked you up with a dry ice supplier and sent you emails every week recommending new faraway places to ship your product.

We've all seen the diagram showing all the places Google recommends you put ads on your pages.  Forget that low-clickthrough spot on the lower-right; stick them right up top or in the middle.  The more obtrusive the better!  We've all gotten the Webmaster Tools messages saying, "Hey, we just noticed you only have two Adsense units on 10% of your pages; here are some tips about how to shoehorn some more in.  Make sure every page has the maximum of 3 for maximum revenue!  And don't forget you can add one link unit too!"  

It'd be pretty ridiculous at this point for Google to admonish people for monetizing their informational content sites around Google's own advice (*if* that's indeed what they're saying with this update, and with obvious content scrapers sans ads jumping ahead of the originals, it makes one wonder).  That's like the cashier at McDonald's slapping you for ordering a large Coke and giving you a two-minute lecture about eating too much sugar.  They SOLD us the ad-monetized business model.  They even sent us credits for PPC in the mail -- reminiscent of AOL coasters -- to encourage us to put that part of our marketing egg in their basket too.  And now it's our fault that we signed up?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/18/11 8:39 AM
>aaron
 
It is one thing to monetize content on a site and another thing to produce content just so the content can be monetized.
 
Where is the point of demarcation?  Nobody has been able to find a specific "line in the sand", but
 
(to borrow a phrase from Potter Stewart)
 
 it is more of "I know it when I see it".
 
 
If we jump over to the big semi-official Google thread there are dozens upon dozens of pretty obvious (at least to me) sites that qualify for the "I know it when I see it" problem.  A few border line sites.  A few sites that should be safe but maybe I missed something.  And a couple that, like Dani, should be fixable with a little soft focusing, cropping or whatever (borrowing a solution used by more than one movie production house to satisfy Potter Stewart) to meet the moving Google target.  And a few that just got caught in the algo change.
 
Unfortunately the Google target will never be known because the slime of world would write content=ads right up to the decimal point of allowable.  And Adsense is only going to encourage more & more ads, just like a porno producer will keep trying to use less soft focusing and tighter shots until one day somebody screams "bad".
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/18/11 1:34 PM
www.daniweb.com went from 90K US visitors/day down to 50K US visitors/day in one day. It's a good-looking site that seems to well-supported and well-liked.

It's a simple question that deserves simple and specific analysis.

  

 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content aaron_baugher 3/20/11 4:35 PM
Stevie D,

Certainly there are cases of "I know it when I see it," but I don't think you can call DaniWeb one of those.  Sure, there are going to be pages of worthless content there -- like on any forum or other UGC site.  Does that mean UGC-site owners have to go through their pages and decide which ones are worthy of carrying ads -- not in their own eyes, but in Google's?  Pretty tall task.  How about when I do a search for something obscure on Google, and the only results I get are crappy spam sites and pages that don't even have my search terms -- yet there are Google ads right on the side.  Aren't they violating what you're claiming is their own policy in that case?

You seem to have a pet peeve against ad-supported content sites, but I don't see the striking difference between that and other models that you seem to.  Someone creates a web site and offers something in exchange for something.  In the case of a free forum, the owner provides the site and services like moderation in exchange for traffic and content.  If I start a forum, I could charge users a few bucks to belong to the forum, or I can make it free and let people choose to click on ads now and then.  Why is one okay and one a sign that I'm trying to pull something shady?  Years ago, people tried both models, and at times there would be scary stories in the press about how the big players like the TV networks and newspapers were going to insist on a subscription model, and free stuff on the Internet was going to be a thing of the past.  But they were wrong:  typically the subscription sites got beaten out by the ad-supported ones -- largely because Google pushed AdSense so hard for exactly that use.

I do understand why Google wants to get rid of all the thin content farms and link farms.  There are so many of those crap sites that, even if they get little traffic each, they must take a significant amount of Adsense revenue from Google, without contributing back anything in quality content or user experience.  Makes perfect sense to try to kill those sites.  *But they are a monster of Google's creation.*  Now Google seems ready to burn down the whole village to kill the monster it created, and that's not right.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/20/11 10:23 PM
>aaron
 
>Does that mean UGC-site owners have to go through their pages and decide which ones are worthy of carrying ads
 
Yes.  Exactly.  The question every site owner should ask is:
 
Does the page exist just to run an ad or does the page exist because it is currently relevant or historically important?
 
More importantly, could or should the page be offered without ads?
 
 
PS:  Like a lot of people I read a lot of blogs.  The number of blog owners publically whining about the decrease in traffic due to Panda is pretty significant.  Yet there are information/forum sites that have very little advertisements... and some that have found other means of funding the site and have no ads.  It can be done.  For some strange reason those sites are not publically whining about their free traffice decling.... because it most likely has increased as the result of Panda.
 
 
 
 
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/21/11 9:23 AM
>> Yet there are information/forum sites that have very little advertisements ... For some strange reason those sites are not publically whining about their free traffice decling

Most likely because these site owners do not run their information sites as fulltime businesses with offices, employees and $50,000 per month carrying costs, and a little less traffic from Google does not thrust them into bankruptcy.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/21/11 9:24 AM
... Not because their traffic increased as the result of Panda.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/22/11 8:27 AM
> Most likely because these site owners do not run their information sites as fulltime businesses with offices, employees and $50,000 per month carrying
costs, and a little less traffic from Google does not thrust them into bankruptcy
 
Try again.  On of the site I refer to rejected an 8 digit buyout offer by one of the content farmers.  Another site their big push from 3M in venture capital financing.
 
There is $ invested in a lot of forums, blogs and baby content farms.  Obviously people are investing $ because they think there is a rainbow of hope for a great return down the road.  Which is why a lot a bedroom blogs et al also exist.... because they think they might become viable enough to get a buyout offer.
 
That said, read the IPO docs of the one content farm.... I am paraphrasing..... Revenue projections may be compromised if Google changes their algo.
 
Oh shucks.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/22/11 8:37 AM
@StevieD_Web

I don't think you will find any statement anywhere from a Google spokesperson mentioning "advertising" and "Panda update" in the same sentence. Same for "content-farms".

If you have a link, I would like to read it. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/22/11 11:12 AM
I would like to read such an article too.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lazarus1 3/22/11 2:07 PM
The fact that you are a major player in your field doesn't mean you are innocent in your field and the "100% of our content is 100% unique" in your post even confirms that.What people forget is that, google had no particular website in mind. Your website got affected because you were not playing by the rules. Most of these so-called do-gooders are the same people who go about scraping contents from other people's websites without giving them(the original sources) any credit at all. I have seen your forum before and It looked good to me at first but that doesn't mean you are innocent. There is no way "100% of our content is 100% unique" can be true. I may be sounding as if I don't feel your pain but you must remember 'NO PAIN, NO GAIN'. Maybe it is time you begin paying for traffic as most others do these days. You also mentioned your server fee. Of course the server fee was that high when you were receiving that bunch of traffic but now that the traffic has gone down, I don't think you will need that much bandwidth,etc., which means your server fee will definitely come down a litte bit for you. Now you've cleaned the forum up a little bit so it is looking 'normal' but I remember a few weeks ago I was searching on google and landed on your website by accident and boy I was shocked to see your website completely filled with ads including annoying pop-ups. There is nothing wrong in putting ads on your website to help pay some of the bills but please don't overdo it. Don't just create thousands of pages just so you can make more money from adsense. Delete some of the ad-filled meaningless pages and start adding fresh and meaningful pages. Once again, being a major player doesn't mean you are an innocent player. I pray you get back your traffic but I don't think God will answer that prayer unless you start working seriously on your website instead of wasting precious time blaming google.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/22/11 4:41 PM
>Shop
 
You are right, Google spokesdudes are never going to address the ad versus Panda update
 
Actually nobody from Google is every going to talk specifics about Panda or any other algo change.
 
So what is the Panda algo all about?
 
 
It is left to us to figure out.
 
So I thought about it, how could an algo distinguish this site from that site?
 
We could figure out the key word spam issues (word density calcs and the like)
 
Hidden text?  Yep another easy algo calc.
 
Like to bad neighborhoods, age of site, speed  etc etc. are easy enough calcs.
 
(not that the calcs are easy, but to the laymen we can understand the basic idea of the calc)
 
SOOOOOOO....
 
what parameters define a content farm versus a good site?
 
Think about it for a moment... 
 
Microsoft, Adobe, Dell etc etc have huge page counts around a seemingly wide range of subjects (to the algo).  But each of these pages will have few if any outbound links to sites other itself and when there are true outbound links there is a heavy use of nofollow.
 
So is a high outbound link content bad?
 
No.  Princeton, Harvard etc also have a high number of content pages with high link counts (as academic papers reference other papers quite often) and we know the academic sites are favs of Google.
 
 
SOOOOO....
 
what is a content farm?  My answer (right or wrong) is a high number of pages, a variety of topics and a specific type of links and/or density of those links.
 
What could those links be?  Ah yes, ads.
 
I cant think of any other way (in terms of an algo) to distinguish Adobe and Princeton from ehow, can you?
 
 
PS:  The White/Black and Grey SEOs seem to have come to the same type of conclusions.... ad density, different ads being shown on paginated pages, type of ad (java etc), type/ocation of the server of the ads, location (above the fold, below the fold) of the ads, different ads shown or not shown to different classes of viewers (registered versus nonregistered), banner ads versus static ads, graphic versus text.
 
 
Of course everybody could be wrong and the real answer is the size of the main css file.  Yea, right.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 3/22/11 5:00 PM
PS:  The White/Black and Grey SEOs seem to have come to the same type of conclusions.... ad density, different ads being shown on paginated pages, type of ad (java etc), type/ocation of the server of the ads, location (above the fold, below the fold) of the ads, different ads shown or not shown to different classes of viewers (registered versus nonregistered), banner ads versus static ads, graphic versus text.

And if they had it right, we could expect to see a whole lot of Websites fixing these "problems" and coming back to dominate the search results.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/22/11 5:57 PM
>> Most of these so-called do-gooders are the same people who go about scraping contents from other people's websites without giving them(the original sources) any credit at all.
We do not do that. Every editorial article on our site was written exclusively for us. If we catch a forum member posting something that exists elsewhere, we delete it immediately. We are very strict about it.

>> Maybe it is time you begin paying for traffic as most others do these days.
We have been using Google AdWords for many years. In recent months, we average about a $15,000 per month spend with AdWords. We also tinker with other forms of online advertising. We also do real-world advertising in the form of event sponsorships, etc. I went into detail in an earlier post in this thread about all the types of advertising we run.

>> Of course the server fee was that high when you were receiving that bunch of traffic but now that the traffic has gone down, I don't think you will need that much bandwidth,etc., which means your server fee will definitely come down a litte bit for you.
Unfortunately this is not the case. The problem is that I have been doing so much SEO work trying to fix this, that traffic has INCREASED! Therefore, we actually are spending more money than ever on hosting. Remember, the Farmer/Panda update has only been rolled out to the US so far. Therefore, our US traffic tanked. However, all of our international traffic (non-US) has been increasing dramatically the past few weeks. The problem is that we are really only able to monetize US traffic. Sure, we have some global banner campaigns and we have AdSense for all that international traffic, but it is very very very difficult to make money off of millions of programmers in India.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/22/11 7:00 PM
@StevieD_Web

RE: "It is left to us to figure out."

Full credit to you, mate, you are more ambitious than I am but consider opening another thread to tell us how you did it.

I think I count 17/171 posts in this one thread. I think the payload was effectively delivered on the first attempt.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/22/11 7:42 PM
Micheal
 
>And if they had it right, we could expect to see a whole lot of Websites fixing these "problems" and coming back to dominate the search results.
 
 
Yep, it is going to happen.  Some of those Content Farms have more than a few $$ laying around to invest in trial projects to determine the "fix".

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WendyPiersall 3/22/11 8:15 PM
@Dani - Don't feed the trolls :)
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content aaron_baugher 3/23/11 5:18 PM
In the thread where a Google employee asked people to report legitimate sites having problems with Panda, sites with no ads at all have been reported.  So this theory that sites are being punished for using AdSense the way it was designed and marketed to be used may hold no water at all.  (And think about it: why would a site that sells products directly have AdSense?  Your product is certain to have a higher profit margin than you can make from someone clicking an ad, so any click that takes them off-site, even if it pays you a few cents, is a loss.  If you're selling an e-book for $14.95, you want people staying there and reading your sales pitch for the e-book, not clicking an ad that takes them away and pays you a few cents.  The only sites where AdSense really makes sense are the content-driven sites that Stevie D. despises.)

One thing I find interesting is that I never see this with the other major search engines like Yahoo and Bing.  My clients' sites get a decent amount of traffic from those too, and we've never seen that traffic take a nosedive.  Yet, I use those search engines (and Google) myself, and they seem to give comparable results.  (If anything, they do a better job of filtering out the junk.  If I do a very specific and obscure search, Yahoo/Bing may only give me two results, both relevant, but Google will give me those two plus a half-dozen irrelevant ones.)  Presumably those sites have algorithm updates, and somehow they manage to do a reasonably good job of keeping up with Google -- without trashing the rankings of numerous legitimate sites every time they change something.  Wonder how they manage that?

The lesson of Panda (and previous updates) for site owners would appear to be this:  Any individual site can get killed by a Google update.  There are no guarantees: no matter how unique your content, how relevant your links, how careful your SEO, any site can get hammered.  Therefore, don't pour your efforts into making one site the best it can be.  Instead, spread your effort out over many (dare I say "thin"?) sites.  Don't invest too much content or time in any one site.  That way, when an update kills a site here or there, your others will sustain you.  Having a single site is feast or famine -- just too risky.  Don't get attached to your sites, so it'll be easy to abandon them if they get hammered and create new ones.  Keep your sites thin, keep them moving ahead of whatever the latest guesses are about Google's intentions, and you'll be okay.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Immys 3/24/11 2:07 AM
@aaron_baugher: "The lesson of Panda (and previous updates) for site owners would appear to be this:  Any individual site can get killed by a Google update.  There are no guarantees: no matter how unique your content, how relevant your links, how careful your SEO, any site can get hammered.  Therefore, don't pour your efforts into making one site the best it can be.  Instead, spread your effort out over many (dare I say "thin"?) sites.  Don't invest too much content or time in any one site.  That way, when an update kills a site here or there, your others will sustain you.  Having a single site is feast or famine -- just too risky."

I think you´re absolutely right!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 3/24/11 6:10 PM
There are a lot of very successful companies that are exclusively based around one domain name. Amazon.com. eBay.com. Facebook.com. Wikipedia.org. IMHO, a company built around a single domain name is perfectly fine, provided that the one website has a solid business model. However, what is important is that you diversify your traffic, so if Google suddenly hates you, your business still has customers and can pull out of it.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content WendyPiersall 3/24/11 8:02 PM
I'd argue that all of those site would go under if Google decided to hate them, too, with the possible exception of Facebook. A site that big couldn't sustain the damage that we are sustaining, either.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 3/24/11 10:45 PM
Search engines don't care who provides the result, they care only about the quality of the result as they perceive it. Call that an emotion if you like, but it won't help figure out how to re-earn the position.

As long as Danny knows there will be no surprises, like cloaking or buying links, then I think that she should relax. 

One way forward was to accept and adjust to remain viable first, before working through the issues. I think she is doing that already.

It has to be smarter than pointing fingers.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content aaron_baugher 3/25/11 8:32 AM
Yeah, I don't know if Google could *kill* Wikipedia the way they can a mom-and-pop site, but they could sure do it some damage if they decided to.  Most people don't search Wikipedia; they search Google and get a Wiki link as one of the first results.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 3/25/11 5:26 PM
> aaron
 
You are absolutely correct about searching/using Wiki.   I use Wikipedia as a source of a lot of obscure info, but I do a G/B search for the topic and then click over to Wiki (who is usually the first SER).   I never search Wikipedia, mainly because their search is pretty crappy AND if I am searching for "blue widgets" I usually desire something more than the Wiki answer so G/B, with multiple sources of results, is much more productive of my time.
 
PS:  I don't even have Wikipedia saved in my favorites.
(unknown) 4/13/11 8:42 AM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content organic-12 4/13/11 9:17 AM
I agree that it is important that you diversify your traffic.  Has anyone seen the Panda algorithm update impact traffic from google.ca as of yet?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/18/11 12:50 PM
Yes, the second iteration of Panda has killed all of our International traffic -- UK, Canada, India, etc. We went from 350,000 daily pageviews down to about 75,000 daily pageviews, globally.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/18/11 3:56 PM
Dani, here is a suggestion for you to consider:

When whatever you're doing doesn't work, try something else.

You have a very non-standard design for your site.  You insisted before that you wanted to stick to it and do things your way.  Your way may appeal to you but apparently it doesn't appeal enough to all your visitors to keep them coming back despite a Google algorithm update.

Google is not going to change its algorithm to restore one Website's search referrals.  Since other sites are doing well, I suggest you consider following their example (in design) while you still have the opportunity to make some radical changes.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content pandahatesme 4/18/11 4:46 PM
@Michael, here's where it seems tricky to me.  So now we are to follow a "standard" way of doing things in order to get traffic.  We have to be unique and original yet we aren't really unique and original if we are to follow "standards" for design/layout, etc.  We need to please users yet we are spending more cycles today to please the bot than users.  I wish that people did not speak out of both sides of their mouths.  Can we make up our mind already?  I came in not knowing SEO (just a smidgen) but now I am afraid we need to know more SEO to be successful.  The unique content drivers that were hit hard are mostly people who are not very tech savvy or as internet savvy as some.  Sort of hurts to see these people suffering and probably ultimately wiped out.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 4/18/11 5:04 PM
You people think that google should adhere to your standards. It's the other way around. You want a free service, you need to adhere to theirs. If your local grocery store wants to rearrange its fruit section because customers like to see the pretty strawberries first, it's not the cantaloupe vendor's say on whether or not his produce gets shelved in the back.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 4/18/11 5:42 PM
RE: "We need to please users yet we are spending more cycles today to please the bot than users.  I wish that people did not speak out of both sides of their mouths."

It is only complex if you insist on making it that way. The system records user behaviour with every search. Think of of it as "pleasing the bot" if that is what suits you but the reality is, it is what users want that counts.  
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/18/11 5:50 PM
@Michael, here's where it seems tricky to me.  So now we are to follow a "standard" way of doing things in order to get traffic.

No, I think there is plenty of room for creativity.  The problem is that it looks to me like Google has chosen to restrict certain types of creativity.  I don't believe people should allow themselves to become so emotionally invested in the precise implementations of their passions that have been found not to work that they refuse to try something else.

If DaniWeb.com is really all about that particular design (and not the community that has used the site for years, as well as the many random passersby who have found help there), then frankly -- in my opinion -- it has outlived its usefulness.

A Website is more than just a set of HTML instructions and words and pictures.  You need that to have a Website.  But what a Website is -- what DaniWeb is -- is a dream, a vision, an expression of passion.

Pardon my paraphrasing Captain Jack Sparrow, but he had a point.  It's not the structure that makes the site.  It's the people and what they do with everything else.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/18/11 11:46 PM
> Your way may appeal to you but apparently it doesn't appeal enough to all your visitors to keep them coming back despite a Google algorithm update.
Our community members keep coming back. New posts and activity on the site has not taken even the slightest hit, despite such a steep traffic drop.

> Since other sites are doing well, I suggest you consider following their example ...
I do not know of any sites that have been confirmed to have been hit by Panda and have since recovered. I have made MANY changes post-Panda. It is very possible that I actually fixed the problem, but that we are all in some type of post-penalty sandbox.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 4/19/11 8:34 AM
It is very possible that I actually fixed the problem, but that we are all in some type of post-penalty sandbox.
 
 
Speaking of a penalty sandbox.....
 
You may have fixed the isssue(s), waited for reinstatement (for lack of a better word), didn't see one (because nobody knows how big/durationg the sandbox may be) and did not something with another change to actually cause the site to regress.
 
Bottom line, take your time.  30-60 days between major changes would be my suggestion. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/19/11 12:12 PM
As I've mentioned multiple times in this thread, there is nothing that I did specifically and only to help the Panda situation alone. Everything that I've changed over the past month and a half has had some type of usability benefit, performance improvement, etc. There's nothing that I've done that I would consider going back on if it doesn't help my current situation. I am confident that everything that I've done has made some type of positive improvement -- if not with Google then definitely along the lines of usability, navigation, page speed, etc.

If there's something that I did recently that caused the site to regress, then I'd be at the point where I'd just say screw Google, because everything I've done has had a positive improvement for the user experience. Therefore, if other people start recovering from Panda and we still aren't, I will continue to try to improve the site, but I can't think of a single thing that I've done that I would have second thoughts about putting back to the old way.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/19/11 12:13 PM
I do not know of any sites that have been confirmed to have been hit by Panda and have since recovered. I have made MANY changes post-Panda. It is very possible that I actually fixed the problem, but that we are all in some type of post-penalty sandbox.

Google says it's not a penalty, it's a change in ranking methods/priorities.  Waiting for the clock to tick on some penalty is probably not going to work.

As for recoveries or partial recoveries, I have read about a few.  People are questioning the claims of success.

Depending on the size of your site, however, it could take Google months to recrawl and re-evaluate every page.  Sites that delete pages or internal links (or add "rel='nofollow'" to internal links -- which they should really not do) or which "NoIndex" pages should NOT expect to see a return to pre-Panda traffic levels until they grow their (Panda-acceptable) content back to a reasonable number of indexed pages.

Our community members keep coming back. New posts and activity on the site has not taken even the slightest hit, despite such a steep traffic drop.

Okay, I see where you mentioned the lost revenue in a previous comment: "My problem is that we SUDDENLY lost so much of our traffic that it is devastating to our advertising revenue, which is our only source of income. I will admit it ... We were getting the traffic for so many years now, so consistently, that I relied on it, I counted on it, and I sold a certain number of ad impressions away to our advertisers, and now I can't fulfill on my contracts."

On March 7 you wrote the following: "Nearly all of our traffic is long tail -- Google search results to individual forum threads and articles based on niche keywords. We rank for hundreds of thousands of different keywords. I do not keep track of what they are because I focus on having quality content for our audience, based on what our audiences is interested in. I do not base what topics to write about on how much keyword traffic different topics can pull in. Therefore, while I never paid attention to what keywords we ranked for, or what keywords had the potential of ranking us better or worse, I have recently started looking at this stuff in Google Webmaster Tools."

I understand how difficult it is to analyze traffic patterns across 10s of thousands of pages.  However, did you notice any common factors in the lost traffic?  Were there certain sections of the site that suddenly received less traffic (more so than other sections -- Google indicates there is a cascade effect, where one part of the site affects the rest of it)?

One more point.  In addressing suggested changes in structure, you wrote: "That being said, our current URL structure was created about five or six years ago. I have been wanting, for at least the past two years, to update our navigational structure into www.daniweb.com/php/forums/123 and www.daniweb.com/java/tutorials/456.

However, I never did it because I've always been scared to lose our high rankings by redoing the entire navigational structure on a site that performs awesome in the SERPS. Don't fix what ain't broke. However, from a usability perspective, I can DEFINITELY see the benefits of being able to tell, from a URL, where a page fits in the overall site structure / hierarchy."

I see that you subsequently rebranded your forum URLs and also implemented 301-redirects.

I can tell you from multiple URL-rebranding experiences on large Websites that there is virtually little to no long-term adverse affects from doing this.

It's possible to screw it up but if you implement good redirect logic you should be okay in a few months.  So, I'm curious.  How do you feel about your changes in URL structures now that more than a month has passed?

You were reluctant to dispense with your tag cloud.  Am I correct in deducing that you have now iframed it?  I see an "iframecloud" element in your front page.

Your root URL ranks first (for me) for "Daniweb", "IT discussion", and "IT discussion community".  You're also 4th for "IT community".  How are you doing for other terms that the root URL previously ranked well for?


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/20/11 4:16 PM
> Depending on the size of your site, however, it could take Google months to recrawl and re-evaluate every page.

What is strange is that our crawl rate has not noticeably decreased -- Google is still crawling about 250,000 pages per day on our site. There was an uptick right after we changed our URL structure around up to 500,000 pages per day for a couple of days. It's strange that they would devote so many of their resources towards crawling a site that supposedly has only thin content and no valuable, unique content. At this point, they're crawling about four times as many pages as the number of visitors they're sending!

> I understand how difficult it is to analyze traffic patterns across 10s of thousands of pages.  However, did you notice any common factors in the lost traffic?

No. I spent hours and hours analyzing Google Webmaster Tools and Google Analytics to death, and the penalty appears to be sitewide for us. Just about every keyword we have went down evenly across the board. Our rank for the keyword "daniweb" even went down slightly! It appears to also be across all sections of the site evenly. I can't find any patterns. If there is a cascade effect, it starts at the homepage (site root) and trickles down evenly everywhere.

> How do you feel about your changes in URL structures now that more than a month has passed?

I don't know what I was so scared of. Google has recrawled about 90% of our new URLs so far, and more of the older versions of the URLs are being removed from the index every day. Our users greatly appreciate the better navigational structure as well, which is a biggie!! My biggest fear is that now all of our millions of backlinks are pointing to our old URLs, and from what I understand, not 100% of the link juice is passed through 301 redirects. It would have been interesting to see if our PageRank was affected without Panda in the equation. However, due to Panda, I have no way of knowing whether or not we lost a good amount of our link juice from changing the URLs.

> You were reluctant to dispense with your tag cloud.  Am I correct in deducing that you have now iframed it?
The original plan was to iframe it, but I ended up using JavaScript instead. The tag clouds are being created on their own page, and are being injected into the page using JavaScript, for a seamless feel to the end user. The actual tag cloud pages themselves are in the robots.txt disallow list, so Googlebot can't see them at all.

> How are you doing for other terms that the root URL previously ranked well for?
Terribly. We used to rank #1 for 'computer help', 'computer support', 'programming help', etc ...
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/20/11 5:23 PM
> How are you doing for other terms that the root URL previously ranked well for?
Terribly. We used to rank #1 for 'computer help', 'computer support', 'programming help', etc ...

Again, I don't think you're penalized.  The Panda algorithm seems to have tightened enforcement of Google's Webmaster guidelines.  That's just my opinion or interpretation of things.

If I search your home page for "computer help", I don't find the expression there.  So how relevant should a random visitor think your page is to "computer help"?

You proudly articulate the spirit and purpose of your community on your front page.  You just don't mention "computer help".

I see a headline (as I write this) with "C++ value initialization" as the text and three pages from your site appear in the top ten results for that query for me.  I think that's relevant.

Google and other search engines are not so good at the conceptual bridging that I think you're expecting of them.  Maybe in the past there were links pointing to your home page with anchor text of "computer help" -- and if so, perhaps those links are no longer assisting the home page.

I think you need to look at the pages on your site and determine how closely they match the queries they receive traffic from, versus the way things were in the past.  I think you'll find that there is a much stronger correlation for those queries from your content.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/20/11 6:13 PM
Our page title used to have the words 'computer help' in them, and therefore we built up a large backlink profile (that still exists) with the keyphrase 'computer help' as part of the anchor text ... Even long after those words were removed from the page title. Additionally, that phrase still exists in our meta keywords and meta description tags. Additionally, while not together as a phrase necessarily, the word 'computer' is repeated billions of times throughout the site, and the word 'help' is also repeated billions of times. Coupled with the fact that they exist as a cohesive phrase in our meta tags and in the anchor text for a lot of backlinks, is what got our ranking for that phrase.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/20/11 6:17 PM
Follow Up:

I just went into Google Webmaster Tools and clicked on the section 'Links to your site' => 'Anchor text'. Apparently one of the top phrases used as anchor text to link to DaniWeb is "daniweb computer help forums". Coupled with everything above and our formerly amazing pagerank, and we ranked #1 for 'computer help'.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/21/11 12:26 PM
Coupled with everything above and our formerly amazing pagerank, and we ranked #1 for 'computer help'.

But not any more.  Whatever you think worked for you in the past doesn't seem to be working for you in the present, and probably won't in the future.

In my opinion, if you want to rank your home page for "Computer Help", you need to spell it out in plain English text that the site offers "computer help".  2 or 3 uses of the expression would probably be better than none or 1.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/21/11 1:12 PM
Oh, I don't care if I rank for 'computer help' or not. I was just giving you an example of something we ranked #1 on for many, many years and then abruptly stopped the day Panda hit, since you asked for examples of what we used to rank for on our homepage.

What I'm concerned with is that we lost 20% of our SERPs traffic for the keyword "daniweb", at least according to Google Webmaster Tools.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/21/11 4:56 PM
Oh, I don't care if I rank for 'computer help' or not. I was just giving you an example of something we ranked #1 on for many, many years and then abruptly stopped the day Panda hit, since you asked for examples of what we used to rank for on our homepage.

I understand but I'm also speaking abstractly.  I doubt there is any one reason why you lost so much traffic.  I am sure there are several factors working together to lead Google to downgrade your site.

Using the terms you think you should receive traffic for in your own content is probably one of the things you will have to do from now on in order to receive more traffic for those terms.

What I'm concerned with is that we lost 20% of our SERPs traffic for the keyword "daniweb", at least according to Google Webmaster Tools.

Don't look at the default date range in that kind of report.  Try time-slicing it by week and see if that pattern has held true for very long.  Also, keep in mind that they are rounding their numbers (I think to 3 positions now -- I don't have time to look for the original announcement on their blog).

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 4/21/11 9:56 PM
>> What I'm concerned with is that we lost 20% of our SERPs traffic for the keyword "daniweb", at least according to Google Webmaster Tools.

I'd also suggest ignoring the data in there.  Sadly it's so terribly bad that it border's on useless.  I'd only use it for medium to long term trending, and potentially some ranking info, but as Michael Martinez points out you need to filter it a lot in order to get any value from ranking data.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/22/11 9:34 AM
I'm confused how you can use it for medium-to-long term trending, because the problem that I keep running into is that it doesn't let me change the dates back enough to pre-Panda, so I can't even compare pre-and-post Panda. I have been finding that it's been useful in giving me a bit of an idea of how far along Google is at recrawling our entire site since the URL overhaul.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 4/22/11 9:57 AM
I agree with RedCardinal regarding the ranking info, it is often skewed by I assume Google Places listings and/or Personalised results? if only those could be filtered also ;)
 
The date limitation is also a big frustration, however the data can be downloaded :)
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 4/22/11 10:21 AM
I don't pay attention to any of the webmaster tools stuff for stats. I check Google analytics though every day.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 4/22/11 10:55 AM
There is some great useful data provided in WMT, however without the ability to filter Places and/or Personalised results from the web search stats its of little interest to me ;(
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/22/11 2:48 PM
People who want to track trends in Google Webmaster Tools data have to download the data every 1-2 weeks and combine older downloads with newer data.  I would look more at the page performance data than the query performance data but these are not numbers I would try to live or die by.  As others have pointed out above, there are a lot of possible search conditions that could affect listing placements.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/23/11 9:32 AM
I don't know how accurate their stats are, but at least it's some data to use for comparison

Alexa's data is based in part on ISP traffic sampling.  For sites in the top 10,000 the estimates are probably rather good but I would not go so far as to say they are as good as site-level statistics.  All services like Quantcast, Alexa, and Compete can do (when their code is not installed on a Website) is estimate traffic patterns.

The Alexa estimates for search visits is actually promising, but may not agree with data that Daniweb collects more directly.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/24/11 12:23 PM
Alexa is accurate in the sense that traffic was steadily climbing up through February. When Panda hit, it took a dive. A few weeks ago, when Panda went global, it took an absolute huge nosedive. So, in that regard, the trend is accurate. Quantcast is more accurate because we do use their analytics beacon.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/24/11 1:38 PM
A *huge* issue that I've been having post-Panda is that scraper sites are ranking above our unique content (Been the trend across the web lately, hasn't it been?) However, it's just getting worse and worse ...

Here is an example:

http://www.daniweb.com/software-development/python/threads/80581

http://www.airfreshenerspray.net/how-do-you-do-a-combobox-in-python-glade.html

An air freshener scraper site actually copied our content b/c they detected the keyword 'Glade' !! Meanwhile, it's the wrong kind of glade.

But this is just the very tip of the iceberg for us. Scraper sites are ranking above us more and more every day!
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 4/24/11 2:50 PM
There has ALWAYS has been a POTENTIAL for scraper sites to rank higher than the original content
 
Always.  Potential.
 
That said, usually the scrapper site shows up in the index for an hour/day/week/month, then the algo kicks in and the scrapper site disappears off the face of the earth or down to page 19000
 
Since Panda kicked down some sites, it would stand to reason that some scrapper site would see a bounce in the index.  TEMPORARY bounce.  Given enough time the scrapper will also get knock down/off the index.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/24/11 3:01 PM
It was always the case for DaniWeb that scraper sites would rank high for our articles. However, since Panda, it's been just ridiculous. Every single one of our articles without fail has scraper sites ranking above the original. Not only that, but scraper sites have been holding steady in the index for weeks now. It's just getting worse by the day.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/24/11 7:16 PM
For those of you who have been following this thread, our AdSense account was temporarily disabled for about 3 days last month due to something very silly ;) Regardless, I just logged into Google AdSense and saw a red message at the top saying "You Have an Urgent Message". I clicked on it and saw the following:

At least 62940 of the pages on daniweb.com are running less than three ad units. Adding any of our top performing units (336x280, 300x250, 728x90 or 160x600) to prominent sections of your pages will substantially increase your ad impressions and overall revenue.

         
[huge blue button]Add more units![/huge blue button]
 
Across the pages we analyzed, you could potentially add at least 109633 units on daniweb.com.


** Super sigh. **
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 4/24/11 11:57 PM
Above the fold /  below the fold /  no-pagination verus with pagination.
 
More ads are not automatically bad, it where/how the ads are handled & located that might be the factor in demoting a page.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/25/11 12:48 AM
The AdSense Team doesn't really coordinate well with the Web Quality Team, although they did just revise their guidelines (perhaps after meeting to compare notes with the WQ folks).

It's a common thread among Pandalyzed sites that many of them see scrapers outperforming them.

There are a few theories floating around about why this happens so much but I think it doesn't really matter.  You need to focus on what is happening with YOUR site.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Celebird 4/25/11 12:51 AM
such opposing forces do not seem all that unusual.

literary publishing has had similar seemingly constant tussles
among ad revenue, ad space, circulation, and quality content.

push one and the others expand or contract --
like a bad eddington space-time balloon analogy.

i believe there will always be such balances
weighted by both editorial and fiduciary goals.

the new england journal of medicine v.
reader's digest v. people; in bits and clouds.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/27/11 4:04 PM
I guess so -- It's just incredibly frustrating from this side of things because first Google takes away half your traffic, then they start consistently ranking scraper sites that copied your content above yours, then you read that too many ads above the fold contributed to why you lost your traffic in the first place, then you get a message from Google saying you don't have enough ads on each page. It's just very frustrating.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/27/11 4:40 PM
It's still happening -- EVERY single article on our site is being beat out by a scraper site. Fro very recent articles too:

Just check out this link ...

http://www.google.com/search?q=Don%27t+panic+but+Apple+knows+exactly+where+you+are%2Fbeen&sourceid=ie7&rls=com.microsoft:en-us:IE-Address&ie=&oe=
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/27/11 5:45 PM
I see your site listed above all other results when I use that query.  I have my location set to "United States" and I am logged out of Google in one browser and have Web History disabled in the browser I'm using to type in this reply.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content pandahatesme 4/27/11 6:02 PM
My hardest hit pages are those with a lot of scrapers.  I've fixed so much already yet no real changes here.  I suspect it may be because of these scrapers still in the index?
(unknown) 4/28/11 6:33 AM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/28/11 10:56 AM
> I see your site listed above all other results when I use that query
> Your article shows on Number 1 for me where I am

Unfortunately it shows #2 for me, with one of the scrapers as #1, and additional scrapers in the 3rd, 4th, and 5th positions. Not logged in, and in New York City.

> Have you ever looked at your site when JavaScript has been disabled ?
Yes, a very light homepage was done deliberately.

None of the dropdown menus throughout the site are accessible without JS because I thought that Google might consider 200 links at the top of every page as keyword stuffing / too many links. Only the top level menu items (Software Dev, Web Dev, etc) are visible without JS.

The tag cloud cannot be seen from googlebot based on a recommendation that was made in this thread, to combat Panda. The idea was that Matt Cutts has made the statement that tag clouds can be considered keyword stuffing by Google.

Most of the actual links on the page for our forum threads and content are customized based on cookies and the user-selectable options. Therefore, I decided to give it a try (to help combat Panda) to only display our top five most recent high-quality editorial pieces on the homepage.

This way it keeps the homepage nice 'n' light, with only high quality links -- Links to our top level categories, and links to our latest five editorial pieces.

All other pages should display similarly with or without JavaScript -- with the exception of the navigational drop down menus and tag clouds, for the reasons cited above.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/28/11 10:58 AM
Putting all those tag cloud links where a search engine will find them on the front page may not be the best solution.  Google is advising people that large tag clouds may look like link spam to their algorithm, so Dani is doing the right thing with the tag cloud.

Dani, maybe you should look at random pages throughout the site with this text-only viewer:


That may give you a feel for what Google is seeing or trying to see.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/28/11 11:16 AM
> Well. Then you are serving one website to users and one website to the google-bot.
I am NOT cloaking. I am creating the tag clouds on an external page and using JavaScript to inject them into the current page, similar to an iframe. The advantage is that the tag clouds can be cached for the end-user from page to page. I am not hiding this from Google at all. In fact, Google is smart enough to be able to interpret JavaScript like this, and understand that an external page is being injected. Googlebot has no objection to this. The SEO advantage is that the tag clouds are being generated on an external page, and I am using robots.txt to disallow googlebot from crawling that page. Googlebot can't see my tag clouds because, while it knows the page exists, an entry in my robots.txt is disallowing googlebot from crawling it.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/28/11 11:17 AM
And, yes, we do get a lot of use from our tag clouds, which is why getting rid of them is not an option for us. ;) Went into detail earlier in this thread.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/28/11 1:16 PM
All of the same content on the homepage used to be visible without JavaScript. I removed it post-Panda to only feature our [higher-quality] editorial content as opposed to our user-generated content, and use JavaScript to populate the UGC stuff, because I had read that Panda doesn't like UGC.

Also, my last post was not meant to sound unappreciative. I was simply trying to explain the technique that I use, because I think it's a good technique if you don't want Google to crawl part of your page. In my case, there was no advantage to Google crawling a million tag clouds that looked keyword stuffed, and there was no advantage to end-users having to constantly download these huge tag clouds on every single page. Having them load from an external page that can be cached and that I can put into the robots.txt file solved both issues.

I would never serve different content based on user agent. Also, I was not banking on Google being able to or not being able to understand my JavaScript. It was all about the ability to put the page containing the content into the robots.txt file -- which is what Google recommends.

This was also done post-Panda.

We don't have a lot of mobile users at all.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/28/11 2:00 PM
Well. Then you are serving one website to users and one website to the google-bot. Google could be capable of understanding, through the fetching of your site content ( scripting included ), those cloud tags are there, and thus not hiding the links at all.

You don't understand what cloaking is.  Cloaking means you show different content to the search engine in place of what you show to the visitor.  It doesn't mean you cannot show the user more content than you're showing to the search engine.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/28/11 2:14 PM
The HTML code does not change depending on the user agent. I'm simply using some JavaScript that google most likely understands anyways. I'm not using any cloaking or grey-hat SEO techniques on the webpage itself. All the SEO magic happens in the robots.txt file.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/28/11 3:08 PM
Firstly, I'm a she :)

Secondly ... You still don't understand that Google will never have any way of seeing or spidering the cloud links in any way whatsoever. Cutts specifically suggested that tag clouds are seen as keyword stuffing -- so that works.

Thirdly ... You are right in that it *IS* possible that DaniWeb lost its traffic because all our pagerank went away because the sites that link to us were devalued from Panda, and not us directly. I've spent a lot of time working on our social media campaign post-Panda, so I will continue to focus more on that and other forms of off-page seo.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/28/11 4:24 PM
 I was referring to the practice of not displaying all of his content to an SE, and that not using a noscript tag was/is having a negative impact on his rank.

And you were wrong.

What I was implying, is google can read and maybe even determine those cloud links are still being displayed

Google has to be able to see the content and, quite frankly, they are pretty clear about encouraging Webmasters to block content that the Webmasters don't want Google to see.

The only place where Google is really saying "Let us in" is in duplicate content.

You have thoroughly derailed an already long discussion with this nonsense.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/28/11 6:46 PM
There is documentation by Google that recommends what I suggested ( noscript )
Only for content that the Webmaster wants to ensure is indexed.

That's not a matter of opinion or interpretation.  It's a matter of respecting Webmaster's wishes and the guidelines are quite clear on the matter.

You're really wasting your time with these bizarre theories and your cheap shots.  I stand by what I wrote: What you posted above is nonsense.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 4/28/11 6:58 PM
You are usually spot-on, Mike, but I think you may have missed a point with this one.

"spam" can only be "spam" if it appears in the index.

If you view the source of one of the Tag Cloud pages you will find <meta name="robots" content="noindex" /> 

I am no expert but it looks like good coding to me. I don't think it means Daniweb.com is "serving one website to users and one website to the google-bot."


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 4/29/11 8:53 AM
> It's obvious after looking at this site, it was designed for an SE, and not primarily visitors
Why do you say that? If you read other pages of this thread, you will see that EVERYTHING that is done is done with the end users in mind -- there's nothing that I've done for the search engines and the search engines alone. Even using JS for the tag clouds was partially done to increase loading time on each page, as the tag clouds can be cached by the user's browser and reused on multiple pages of the site.

> So, why not take Google's advice and prune the forum - just get rid of forum threads that don't offer much to visitors
I've done this. About 20-25% of our forum threads now contain the "noindex" meta tag. Additionally, our entire Business Exchange section (Over 30,000 pages which IMO has the lowest quality of content on the site) is now behind a login wall, although this was actually done about two weeks before Panda hit.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/29/11 10:59 AM
Dani, I've looked at thousands of Websites that were made for search engines through the past 12 years and yours doesn't even come close to looking like it was made for search.  You provide a fundamentally unique approach to Website design and presentation.

Google Panda is not nearly as simplistic as that.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Puntsy 4/29/11 1:43 PM
SEO is an insult to the intelligence of the google-bot, which is sophisticated enough to weed out repetition or spammy content - result = no appreciable difference. SEO is more myth then anything else.

Why then, good sir, do you choose to offer "EXPERT SEO ADVICE" on your site? Perhaps you should stick to offering your advice there, because the advice you've given in this thread is pretty ill-informed: http://www.mfwebservices.com/search-engine-optimization-advice.php
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Jarrod1937 4/29/11 2:17 PM
I would argue there are two definitions of "SEO". To me it is just designing your site and page structure in a way that makes the site easier for a bot to crawl your site and figure out what your content is about, this is classified as "Whitehat SEO". On the otherhand, you have "SEO" in terms of spammy-ness and SEO "Techniques", which more so is "Grey" or "Blackhat SEO". Obviously whitehate seo is the only one you should be involved with.
Though to say all SEO is a myth, for any version of it, is a bit of a laughable statement in my opinion. Though maybe he just misspoke.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content mraccardi 4/29/11 2:52 PM
Mike and Dani why bother even posting here ? This forum is fulled of the same people making stupid remarks all day.  You would never take advice from crazy people in real life why do it in a forum  ? Unfortunately, the same people post the same junk over and over again without making any sense to what they are saying. . Its unfortunate really that the same people always have to spoil every thread on here. The problem is the lack of real moderation. My advice if you have a question go else where or else the 10 or so people who comment on every post is going to show up spewing the same stupidity over and over again.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Jarrod1937 4/29/11 2:58 PM
Well, technically items you can do that act as whitehat SEO may also help with the content on the site. Providing breadcrumb type page titles helps the bots and the users understand the context of a page better. Alt tags for images help those who have screen readers as well as those who have slow internet connections. In general, SEO in the context of markup can improve usability and accessibility, which in my book enhances/advances the content value of a page.
And I barely know you, as i haven't been on too active on these forums, but i find this to be an odd statement, "Everyone here is just so interested in being validated or being right. Whatever. I'm done with this discussion. I've offered my advice and feedback to Dani - she's free to ignore it or use it.". Why do i get the feeling you're taking comments personally? No sense in that, i simply disagreed with your statement (in the context of this topic), nothing more. Differing opinions does not equate to no one listening to your advice.
But hey, whatever.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/29/11 3:58 PM
Mike, if you knew what you were talking about in the first place, I never would have responded by pointing out the nonsense in your posts.  If you're going to continue to derail the thread then hopefully Dani will just ignore you.  Your advice and feedback have been less than helpful.  Your complaining is even worse.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Jarrod1937 4/29/11 4:01 PM
Defending is not being defensive, at least in the sense you mean. You offered your opinion, i disagreed, you stated that i may have not understood the context, i simply stated that i did not misunderstand and explained why. So, no, that is not taking something personally, but "Whatever. I'm done with this discussion." signifies at the very least a personal frustration, hence my original statement. But really, nothing more needs to be said, i was just merely voicing an observation, and would suggest not letting a conversation with random people on the web get to you. If they don't, then fine, perhaps i observed incorrectly.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 4/29/11 4:42 PM
Nobody can be proven right or wrong here so defending an opinion is futile.

Consider opening another thread to debate this thread, Mike. I can see at least five statements where I think your opinions are unfounded but I won't mention them here. This is Dani's thread, not yours. 

 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 4/30/11 12:26 AM
when you noindex a page ( 30,000+ ) you're playing with fire..

You're only embarrassing yourself, as NoIndexing content is precisely what Google has been telling people to do both in these forums and in their Webmaster Guidelines.

Just stop posting.  At this point, all you've done is flood this discussion with absolute nonsense and ridiculous misinterpretations of Google's statements.  No one can possibly benefit from the advice you're handing out.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 4/30/11 9:30 AM
Mike, quick point; meta-noindex does not prevent pr flowing through any given page
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 4/30/11 9:54 AM
Mike I think you are maybe confusing robots noindex and meta noindex? With meta-noindex the page is crawled and exists within G's internal index, it is of course not shown in the public index.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 4/30/11 10:28 AM
Mike, read Vanessa's article http://googlewebmastercentral.blogspot.com/2007/03/using-robots-meta-tag.html, then re-read Matt's on the url that you provided earlier http://www.mattcutts.com/blog/google-noindex-behavior/  I'll quote him to save you time; "In that case, currently Google wouldn’t show the page but would follow the outgoing links" 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Farmer Effects 4/30/11 11:46 AM
My understanding is that meta noindex, follow means that the page is not shown in the index but link equity still flows via follow.  Why distinguish noindex, nofollow and noindex, follow then?   Follow means you pass PR right?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 4/30/11 1:26 PM
"Follow means you pass PR right?"

Yes, for a page that is in the index.

NO!! A page isn't required to be indexed to flow PageRank. Only the nofollow directive stops the flow of PR.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 4/30/11 5:19 PM
When so many are disagreeing with you, do you not consider that you might be wrong? Noindex instructs the search engines not to index the page - it doesn't tell the bots not to crawl the page. Crawling and indexing aren't simultaneous, the indexing follows crawling, they are entirely separate processes.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 5/1/11 3:04 PM
Hi Mike

It's not a popularity poll - it's a matter of understanding how things work. I make no claims for myself, but Michael, for example, has been around a good few years and is highly regarded by his peers. His opinion is well worth considering.

As to Panda. One of my sites got hit, just fractionally, but result was I came in below Places instead of above them - the sum total is that for those queries it doesn't exist any longer. OK, I've had a free ride for 7 years and now I have to try a bit harder to dislodge a site which has nudged me off the top spot. I think I've figured out the problem and all that's left is a stack of hard work.

It's quite true that Google values content above anything else. It's also true that AdSense has preached the monetizing sermon of placing as much as you can above the fold. For those of us who have information sites and tried these placements and found it worked extremely well - well enough to go part of the way to funding our work - who can blame us? The advertising model was created by Google and publishers were encouraged to use it to the nth degree.

Having said which, it's up to the publisher to implement or disregard that guidance.

The main point, though, is confusion over noindex and nofollow. If a page is "noindex" the bot is free to crawl it and follow the links - it doesn't become invisible to the bots, it is simply prohibited from appearing in the index. As I said earlier, crawling and indexing are two entirely different procedures. The bot gets the information and the indexing processes it. Whilst a noindexed page is hidden from users, it, and its content is not hidden from bots. The instruction is to prevent the page from appearing in the SERPS - crawling goes ahead as with a normal page.

Noindexed pages exist, just as a standard indexed page does - the difference is the page doesn't appear in the results generated from a search. Therefore, any such page will flow PR in precisely the same way as any other unrestricted webpage does.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content PeggyK 5/1/11 5:34 PM
I'm not an expert, but I've been assuming that what Matt Cutts said in this 2007 interview with Eric Enge still holds true:

http://www.stonetemple.com/articles/interview-matt-cutts.shtml

Eric Enge: Can a NoIndex page accumulate PageRank?

Matt Cutts: A NoIndex page can accumulate PageRank, because the links are still followed outwards from a NoIndex page.

Eric Enge: So, it can accumulate and pass PageRank.

Matt Cutts: Right, and it will still accumulate PageRank, but it won't be showing in our Index. So, I wouldn't make a NoIndex page that itself is a dead end. You can make a NoIndex page that has links to lots of other pages. For example you might want to have a master Sitemap page and for whatever reason NoIndex that, but then have links to all your sub Sitemaps.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pro Timewaster 5/5/11 9:02 AM
let me throw the facts back at you for a second there.

You were in business for 8 years!

You were getting free traffic from google all this time.. Now you've lost 1/3 of your traffic* so you cannot pay your staff, employees, rent etc..

So after 8 years in business and getting tonnes of traffic.. You were just scraping by just about meeting your overheads and earning a tiny profit.

Something isn't right here, it's not traffic, it's your business acumen.



*1/3 of traffic drop calculated based on the fact that Google US traffic dropped from 90K to 50K, assuming non-US google search, bing, yahoo, direct traffic, linking traffic all stay constant.. 1/3 porbably on the high side in terms of loss of traffic. I would even say maybe you just dropped 1/4 of your total traffic.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 5/9/11 1:52 PM
Hi,

Sorry for not posting in awhile. I was getting a little underwhelmed that this thread was turning into a flame war so I decided to take a step back.

We are forums + editorial for everything IT related, including Internet marketing. In those forums especially, we have a problem with spamming, people plugging their businesses, etc. Aside from snipping self-promotional plugs from within posts, one of the things we do is make forum signatures only visible to logged in members of the community.

Our Business Exchange section is essentially our online marketplace and classifieds section. It's completely free for people to post web hosting deals, advertising space for sale, job offers, etc. A few weeks before Panda hit, I decided that it was silly that we were essentially allowing an entire section of "legal spam" to be guest-viewable, but members were being scrutinized for even linking to their website in one of the other sections of the site. I felt like this was encouraging spammers to come to DaniWeb. Therefore, I decided to put the Business Exchange section behind a login wall, so that it essentially was a free online marketplace and classifieds between registered community members, keeping in line with promotional forum signatures which have been behind a login wall for about a year now. Doing this has definitely helped the quality of content within our community as a whole, and it is not something that I would go back on. We essentially did lose an entire section of content as far as Google is concerned, and it is VERY possible that we also lost a lot of backlinks to the section that passed PageRank to the site. However, it's an acceptable trade-off for the betterment of the community, as far as I am concerned. Because every single page of the Business Exchange section is now a 'Permission Denied' error for everyone who is not logged in, I put the entire directory into the robots.txt disallow list, because there is no reason for googlebot to waste its time crawling tens of thousands of error pages.

Something that was done post-Panda was to add the noindex meta tag to Q&A forum threads that have received zero replies. Once they receive replies, the noindex tag is removed. As I understand it, they are still being crawled and they still pass internal pagerank. They simply don't show up in Google's index. I did this to decrease our bounce rate from google searches because, let's face it, every single time you go from a google search to a forum thread that just restates what you're looking for without providing any answers or solutions is just incredibly frustrating and never at all beneficial.

Pro Timewaster, keep in mind that Panda *has* affected non-US google searches. Globally, pageviews decreased from 350,000 per weekday down to 180,000 per weekday, and 200,000/day on weekends down to 110,000 on weekends. While we are eight years old, we started as a hobby and didn't start monetizing our audience until about 4 years ago. We have been putting some $ away into savings over the past few years. However, we currently have lost so much of our traffic that we are now in the red on a month-by-month basis (unable to pay our web hosting bills, rent, and employee salaries), and are pulling out of savings to pay the bills ever since February. It is not sustainable for very much longer.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 5/9/11 4:13 PM
So Dani, what are you willing to try next at this point?  And have you read Google's Friday Webmaster Central blog post?

More guidance on building high-quality sites

What I have noticed among reactions to the article is a lot of denial from Webmasters who feel like Google is asking the wrong questions.  I don't think people should focus on whether these are the right questions.  They appear to be the questions (or a subset of them) that Google asked their quality raters to use last year to divide sites into "high quality" and "low quality" groups last year.

What happened afterward is guesswork, but let's say you can find ways to answer those questions about your site such that you would (if you were in a quality rater's shoes) put your site into the "low quality" group.   I'm not suggesting that you doubt the sincerity of your efforts.  Rather, I'm suggesting you try to put yourself in someone else's shoes -- keeping in mind that they may never have included your site in the quality review.

Doing that, can you see ways to interpret the questions in ways that cast the least favorable light on your site?

If so, you may be able to make some business decisions about how much you want to try to comply with Google's expectations.  You don't have to think in terms of "This is how the algorithm must work".  Rather, just assume that the algorithm is trying to help Google evaluate sites such that those questions would be answered one way for more of what Google deems "high quality sites" and another way for more of what Google deems "low quality sites".

You might also want to look at the revised AdSense placement guidelines.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 5/10/11 10:21 AM
"   Something that was done post-Panda was to add the noindex meta tag to Q&A forum threads that have received zero replies. Once they receive replies, the noindex tag is removed. As I understand it, they are still being crawled and they still pass internal pagerank. They simply don't show up in Google's index. I did this to decrease our bounce rate from google searches because, let's face it, every single time you go from a google search to a forum thread that just restates what you're looking for without providing any answers or solutions is just incredibly frustrating and never at all beneficial.  "
 
 
Bingo.  Major step in the right direction.
 
I would take it one step further and nonindex the page until the responses have been hand reviewed OR automate the system to index the Q&A when a high valued poster responds.  This way a goofy Q with a poor A doesn't get added to the index.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 5/10/11 11:05 AM
> I would take it one step further and nonindex the page until the responses have been hand reviewed
Not possible with thousands of posts being added daily.

> automate the system to index the Q&A when a high valued poster responds
In our community, the value of a post cannot be determined by who posted. Often times, our most valuable members are the first to jump at a new thread just asking for more detailed information (because they want to "train" posters as to the best way to ask questions) while the less active posters will try to help people right away even if the question is incorrectly phrased or doesn't provide all the information needed.

I could come up with some elaborate algorithm to try to determine the quality/usefulness of a forum thread, and try to determine whether or not the responses accurately answer the question being posed. But ... isn't that Google's job? And isn't that what they, with all their resources, currently fail at (trying to determine the actual subjective quality of the written word)??
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 5/10/11 11:06 AM
> Bingo.  Major step in the right direction.

It was done weeks ago, and then written in this thread that I did it.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 5/10/11 11:17 AM
I could come up with some elaborate algorithm to try to determine the quality/usefulness of a forum thread, and try to determine whether or not the responses accurately answer the question being posed. But ... isn't that Google's job? And isn't that what they, with all their resources, currently fail at (trying to determine the actual subjective quality of the written word)?

Unless you were to use the criteria that Google uses, it would be pointless for you to try to develop such an algorithm.

I think you're missing the key point here: it's Google's index and they get to write the rules for how that index is managed and by their interpretation of those rules they are doing a very good job.

If you want to participate in the index you MUST comply with those rules.

This technology isn't just looking at words on the page (but it isn't fussing around -- much if at all -- with dirty signals like "bounce rate").  Read the Webmaster Quality Guidelines again and ignore anything that mentions "content".
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 5/10/11 12:30 PM
I am not criticizing Google's algorithm. I am simply saying that it is futile for me to come up with my own algorithm to determine the importance/quality of a page and therefore automate when to noindex the page. Google has some of the smartest people in the world working on determining just that, and they have yet to perfect it. So it's certainly not feasible for me to come up with my own algorithm to attempt to do it.

You had suggested that I either manually approve the quality of a page, or automate it. I was simply explaining to you why neither is realistic.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 5/10/11 1:17 PM
You had suggested that I either manually approve the quality of a page, or automate it. I was simply explaining to you why neither is realistic.

That was not me.  I would never advise anyone to try to replicate Google's algorithm.  In fact, I've written about why that is impossible on SEO Theory.

What I do believe is possible, however, is to rethink the design of any Website that has been downgraded by Panda and implement changes that are more compliant than the previous design.

Google's Webmaster Guidelines look at both content and site structure.  Too many people are ignoring what the guidelines suggest and instead are trying to second-guess Google's collection of signals with no real understanding of what kind of data Google collects.

The simpler approach is to simplify the site design and build out from that.  People are speculating, on the basis of a remark from Matt Cutts (and another unconfirmed source of information) that there may be windows between evaluations of Websites.  If that is the case, then the sooner you update your design the better (assuming site design is contributing to your poor Panda performance).
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content fijinet 5/11/11 12:36 AM
@StevieD_Web
 
Every webmaster claims their site is innocent. The facts often lie somewhere else. Here are some questions to ask yourself as you analyze your own website....
Would you have a website if ads were prohibited?
----

That just makes no sense at all. Almost every site on the net needs to sell something. A product or a specific service.
If that's not the case, the sites provide some sort of information by people who are passionate about some topic like forums and blogs,etc

Millions of people start a site or blog based on passion for a specific topic. People DO need to cover running costs and overheads. Google adsense and other advertising options is always is a good way to keep websites up and running and pay the expenses ( even if that means the webmasters make a profit ).

Just because people use advertising on the site does not mean they are a farmer site or have bad intentions. That's a bit over the top.

Yeah I can understand if the case is when you enter some website, the information you seek is not visible or the description was misleading and the site is loaded full with ads and affiliate links. Fine those are crappy sites designed specifically to milk traffic for ads and Google has to punish these sites hard and take them out.

But Danniweb is a pretty good forum, I had many of my questions answered on that forum. The popup was annoying though but I usually got the info I needed.

This Panda update is not perfect, I know Google has good intentions but they screwed up somewhere in the algo.

I still see many crappy sites with no content or misleading content loaded full of ads on page 1 right at the top and I see much better and more informative sites, ( even with no ads and aff links on them ) at the bottom or on pages 4,5,6, etc

Just makes no sense at all.

1) Whether a site had Ads or not should not matter ( unless the site has way more ads than actual content )
Because people need to pay website expenses and need to generate money from somewhere. Adsense is always an easy to implement and use solution for most webmasters

2) Google should judge the site based on the content. Does the site deliver what it promises?
The site should not mislead and trick users.

I searched many times for something and found Daniweb come up on the results. I clicked on Daniweb and in almost all cases I found the information I was looking for.
Yes that forum has ads up and down but that's not bothering me. All I care about is that I found what I was looking for.

Besides the ads on Daniweb usually are related to something I'm interested in so it actually gives a better experience to the user.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content lockedout-noreason 5/11/11 12:57 AM
Dani, it's possible that you fixed the problem. The main problem may be that Google hasn't regenerated the algo with the new changes
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Richard Hearne 5/16/11 10:44 PM
THings are looking up for DaniEb now?


Glad to see, and in the long run I suspect this may be something very positive for both the site and its users.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 5/16/11 10:52 PM
Nice Spot on  Red...

Love seeing this kinda stuff.

Sheer effort meets dedication.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content lockedout-noreason 5/17/11 1:33 AM
The trend is definitely positive but she is far from being recovered judging by the graph. http://www.daniweb.com/forums/attachment.php?attachmentid=20976&d=1305571975

I hope that this is signal that sites are being released from Panda. So far sites hit on 2/24 have not come back as far as I know, only those hit later.

Danni, if you are still here: is it international traffic or both that's increased? I know you got hit internationally the most. My international traffic is now much higher, G.Canada brings me 15% of Google.US traffic for example when I have like 3% of stuff that is for Canada and Canada is 10 times smaller than US (give or take). Google will eventually release all improved sites so this may be a taste of what to expect.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content dotmediaweb 5/19/11 10:11 AM
Dani -

It's a terrible suggestion but what about killing the RSS feeds? That might start mitigating the impact of the scrapper sites on you. Quite a few blogs have done this post-panda to keep scrapers from easily taking their content. Also, why not send out DMCA takedown requests. It's time consuming and not a lot of fun, but it could also help. I would only send them to scrapers that are not linking back to you. If Google was as smart as they want us all to believe they are they would get source attribution correct. The fact is that they suck at it. I started noticing that 'standard' descriptions such as those for products, houses, service companies and authors (bios) had one page/site with the content ranking page 1 and the rest were pushed back. Rob Snell just posted saying nearly the same thing.
http://searchengineland.com/when-panda-attacks-online-retailers-need-to-react-70403/

So it's not that Google sees that as low quality, but that they are only going to rank one of those pages in the top part of the index (page 1) and rank other relevant content around it.

Pre-Panda Google Said:
"We’re evaluating multiple changes that should help drive spam levels even lower, including one change that primarily affects sites that copy others’ content and sites with low levels of original content. We’ll continue to explore ways to reduce spam, including new ways for users to give more explicit feedback about spammy and low-quality sites. As “pure webspam” has decreased over time, attention has shifted instead to “content farms,” which are sites with shallow or low-quality content."

Basically if x% of your sites content appears on other sites you could be in for a hurting. Google likely didnt anticipate this hurting sites like yours and for all their PR spin it's clear that LOTS of innocent blogs, ecommerce and other sites were nailed while sites that scrape them seem to be doing better. That's because the sites that scrape them also scrape TONS of other websites making them look like the authority and the other domains the copiers.

Also there was an update a week ago, Panda 2.1, so if indeed you are improving it could be due to this update:
http://searchengineland.com/its-panda-update-2-not-3-google-says-76508
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content pubcrawlercom 5/29/11 10:38 AM
Not only should people kill RSS feeds, but they should start auditing their traffic.

For instance, what good does Feedburner acting as a proxy to my RSS feeds provide me?   All I know really is that Google has requested a RSS feed.  Who's on the end is anyones guess.   That's simply just bad business.  

So at minimum I'd start by blocking feed burner and other RSS proxy services.

When you go looking at your RSS related logging, I bet you will be surprised who is requesting those feeds.   Google's App-Engine makes tons of requests to ours as do profiling services and other "intelligence" gather companies.

If you want to continue RSS for real users it needs to be a new custom solution with distinct URLs and per-user codes built in the URL.  

The day of free and open to all sharing are now dead thanks to Google and Panda.

PS: GoogleBot love RSS feeds.   As soon as we publish on one of our sites in comes GoogleBot within 60 seconds to the RSS.   Now we cut those off, so Google actually has to do some work analyzing pages and making sense of data.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 5/29/11 1:35 PM
Pub I am agreed on the lessor value of the RSS feature - and stated so in another recent post.

I, for one, will not revisit that other thread where people who are unwilling to make changes and only point fingers sit.

Dani is a PERFECT example off what to do... TRY. Again - I'm not saying Google is perfect. I am saying - here are the fixes to the current issue.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content EricLegge 6/1/11 3:37 PM
Lyrical Question went back to the "Think you're affected by the recent algorithm change? Post here.        " thread today after saying she would never be back because a poster called Kunstbus produced links to her unbelievably poor sites. See page 96 of that thread that I took this from: Here you have one :) http://thelyricsexpress.blogspot.com/ Would you say that someone who produced a site like that should be heeded about getting your site back into business?

Daniweb.com is an IT forum that makes its living off the free user-generator content that Google now rates low, so the site will never recover its former position. It has recovered slightly, but that is due to the removal of forum posts that had no replies, dealing with RSS feeds, etc.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 6/1/11 5:25 PM
Eric - you're an idiot - and you came in here to do this? This is called harassment.


GFY...  *GRIN*.

This is the PRECISE reason I would not put my work out here for you idiots. Keep posting and making yourself look like the scum you are.

But of course, your site is RANKING through the rooftops.... oh yes...


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 6/1/11 5:30 PM
*grumble grumble* psychotic idiotic waterbag people off their medications again... Sigh... Eric - go take a chill down med - come back and quit prodding the snakes here.... Because UNLIKE my real websites --- I know where yours is.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 6/1/11 5:41 PM
Excuse me -- I meant - ARE.... I mean --- astrologically speaking it's easy - considering everyone has a birthday... and uhmmmm I always say people should always buyer beware when they purchase PCs... Don't you think?

You want to keep threatening and harassing?

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content dcshutterbug 6/2/11 7:21 AM
StevieD_Web said:

"Today, Google said you can't have a business based solely upon ad revenue.  Their rules.  Their ball field."


Oh, so that's the reason some of us got dumped, financially hurt?!

Then why did AdSense reps contact me and encourage me to place their ads on my site? Their optimization team reviewed the ad placement and made suggestions more than once.

My site in a educational and information site; don't have any widgets to sell so derive income from affiliate ads. I've spent hours every week for almost seven years writing content, and also hiring knowelegeable people to write artilces. I'm a retired eductator and have some professional writing experience; I don't scrape content from other sites but there are other sites who write about the same topics.

Google PR people even contacted me several times and got my site (along with some other sites) featured on national TV and mentioned in national publications (US). Why? Because Google wanted to tout our success using AdSense!!

So, if you are right and Google now says that I can't have a business based on ad revenue, then Google kicked me in the behind and pretty much dumped my site prior without notice. Traffic down more than 50%; income down more than that.

My site has good information, geared to beginners and novices. I intentionally keep articles brief, about 400-500 words, though about a third of them are 800-1000 words long.

I get penalized because I make things simple to understand for novices who neither need nor want to read about a topic in long articles, often filled with technical jargon?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content PeggyK 6/2/11 1:34 PM
I disagree with what you say StevieD_Web wrote.

I'm am AdSense Publisher (and a TC in the Adsense forum), so maybe I'm biased, but Google has never said that you can't use AdSense or earn money from advertising on your site.  Google wants publishers to use AdSense, as long as your site complies with the Webmaster Quality Guidelines.

For some AdSense Publishers the "Panda" update has meant an increase in traffic to their site, so you shouldn't consider it a penalty against having ads on your site.

If you have a site with all original unique content, then it seems like you should be able to use the suggestions people have provided here to regain the traffic to your site that you have lost.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 6/2/11 1:51 PM
@dcshutterbug
 
One of the definitions of a Content farm is Content being written for the specific purpose of being monetized.
 
If that definition is correct than ad revenue can not be the "purpose" of your site.
 
 
So why have a content website if money the content is not an option?
 
Good question..  But I do know the New York Times, CNN et al are not being hurt by Panda and those sites and others like them could easily be defined as Content Farms.
 
So whats the difference?
 
1)  NYT, CNN et al do not monetize every page of the site, especially the front page and major landing pages will typically be ad free except for self marketing purposes
 
2)  Old pages are often sanitized of ads when the page is archieved
 
3)  Content is wilfully destroyed (usually updated and combined into more meaningful content as the story develops)
 
4)  Page length (word count) varies.  Long, indepth pages are not uncommon.   Longer articles tend to be written at a higher reading comprehension level than short/fluff pieces.
 
5)  How/where ads are loaded (above/below the fold, pagination etc) vary but even so the emphasis is on the content first and the ad placement secondary.
 
 
Having a content page is not bad IF you follow good practices.  NYT, CNN and other news sources are excellent examples of good practices in terms of monetizing internal pages and following the unwritten Panda rules.
 
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content hotwinduk 6/2/11 3:48 PM
Lyrical Question it looks to me as if it is you who is threatening Eric Legge. Like Kunstbus, all he did was point out to people that they should take your advice with a pinch of salt. If you are reading this, Eric, and this nut case does anything to your site, let me know and I'll find out what her address is so that you can go and have a word with her.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content EricLegge 6/2/11 4:22 PM
Thanks, hotwinduk, but  I can look after myself. I am not hiding behind a non de plume like Lyrical Question and I gave the URL of the site that was hit by Panda on the other main thread, because people like her don't worry me. If my sites are interfered with I'll know it's her and I'll take appropriate action. I reserve the right to inform the users of a thread if a persistent poster isn't a reliable source of information. People like her think that they can get their way by threatening anyone who dares to criticise them. People who can't take criticism are just little tyrants with inflated egos whose advice is always not worth heeding.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 6/2/11 4:23 PM
Really hotwind... You have some issues to be following me around too... So you, eric (who believes in Telepathic Orgasms, mind you - from his own site and I believe Book...) and kunst ride the same short bus?

If people choose to take my advice - based on your "uncovering" (oooooh ahhhh.... scaarrrry) of old websites of mine - without realizing I have others - or not... is entirely up to them - whether with salt --- or pepper.

Now run along child.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 6/2/11 4:24 PM
And I never threatened your site ya dingbat... Are  you guys related and off your meds or something?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content shukla_vikas 7/10/11 6:57 AM
Daniweb,

did your site finally recovered from Panda ?
(unknown) 7/10/11 8:03 AM <This message has been deleted.>
(unknown) 7/10/11 9:44 AM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 7/10/11 2:38 PM
I dunno... I just asked seems they;ve picked up - from what I can see ---- most that do well - don't return to say thanks or advise.
(unknown) 7/10/11 7:13 PM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content lockedout-noreason 7/12/11 12:16 PM
dexteronline, are these 20K and 60K real or did you get them from Quantcast?

Looks like sites hit on Feb have almost an impossible task to come back, maybe Google has a set time penalty or something
(unknown) 7/13/11 1:06 AM <This message has been deleted.>
(unknown) 7/13/11 1:14 AM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 7/25/11 4:34 PM
Hi everyone,

Just a quick status update. It appears as if our traffic suddenly returned to normal over this past weekend, just as quick as it disappeared back in February. It is still to early to tell, since Monday is only half over, but it is looking like it is coming back even stronger than ever. Nearly all of the big changes I've made have been documented here and in the WebProNews video interview I gave back in May: http://www.webpronews.com/daniweb-google-panda-2011-05 ... There were no big changes made immediately before the site came back, with the exception of a significant increase in my Google AdWords budget.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content ShopSafe 7/25/11 5:06 PM
It's good news Dani. Great news, but I do not know why you thought it would be a giid idea to mention Adwords. 

What nexus could there be between the two events? 

I am not pretending I have a clue but I think that an algorithmic penalty which expired over time is a more likely explanation for the bounceback.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 7/25/11 5:14 PM
I mentioned AdWords because we use it heavily to increase registrations, which directly results in an increase in posts per day. If there was a correlation, then it was a sudden increase in new content followed the penalty reversal.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 7/26/11 1:58 AM
Excellent news and thanks for the feedback :)
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 7/26/11 11:45 AM
Thanks Dani - I've been following your work closely - and it correlates completely with the items and advice we have been giving.

I just didn't know how long exactly it would take. And was kind of worried when we didn't see an immediate change after your tweaks and clean up.

I am so thankful you came in here and announced this.

Thank you for taking the time to document every change and every item.

Your site deserves the lift - and the work you've done is really good... Kudos!


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 7/26/11 11:47 AM
Congratulations on the recovery, Dani.  Based on buzz around the Web, it seems like Google might have rolled out another Panda update this past weekend and sites that have made improvements are seeing the benefit of having done so.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Pelagic 7/27/11 8:18 AM
 
I suggest it might be just a bit premature to consider it as a recovery already, as Dani stated "It is still to early to tell", there were certainly significant fluctuations over the weekend, specifically to those sites (to varying degrees) that have characteristics that are most likely to have been affected by Panda. However and noticeably, some of those sites dropped back down (a bit) on Tuesday.
 
Larger sites that have made even significant improvements will need to wait far longer than they might expect to realize the benefits of having done so ;)
 
 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 7/27/11 8:46 AM
Where do you get your exceptional insights into the workings of Panda, Pelagic? Your pontificating has no more value than any other opinion, with the notable difference that it carries your trademark negativity.

This is a Help Forum - or had you forgotten?
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 7/27/11 11:27 AM
Without getting into a discussion about anyone's opinion, Google did confirm yesterday (through Search Engine Land) that they released a new Panda iteration on Friday, July 22.

Everyone's mileage may vary.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 7/27/11 2:06 PM
It definitely was a full recovery for us.

Check out this article for the details along with some Google Analytics screenshots: http://www.webpronews.com/daniweb-claims-110-recovery-from-google-panda-update-2011-07
(unknown) 7/27/11 8:56 PM <This message has been deleted.>
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content TheDonald 7/28/11 10:37 AM
Dani, congrats on your recovery, you must be flying high, I know I would if I ever recover. There are lots of webmasters still stuck in Panda poop, as one of the very few survivor, you are one of the very few who can offer practical advice. Google said: This update incorporates some new signals that help differentiate between higher - and lower-quality sites. As a result, some sites are ranking higher after this most recent update. You've made lots of changes. Do you attribute your recovery to any of these changes or is it possible that the last update just reclassified your website as a none Panda website? If it was the changes you've made, obviously the noindexing of tags and search pages was big. Do you think removing ad units was any help, if so how are you replacing the loss in revenue? Would you consider bringing back some of the ad units now that you have recovered? You mentioned early on that you removed the popop that encouraged new visitors to register with your site. Will you consider bringing it back now that you have recovered? Once again congratulations on getting your traffic back. Wow! That must feel like a massive load of your shoulders. I know I've had though of nothing else but Panda, since April 12 when my website was hit. Thanks for taking the time to share your thoughts.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 7/29/11 2:06 PM
> This update incorporates some new signals that help differentiate between higher - and lower-quality sites. As a result, some sites are ranking higher after this most recent update.
Definitely sounds like that's what happened with me.

> You've made lots of changes. Do you attribute your recovery to any of these changes or is it possible that the last update just reclassified your website as a none Panda website?
I did improve a handful of things that, in retrospect, I feel I deserved to be penalized for. Therefore, I think that our recovery was Google reclassifying our website as non-Panda, but the fact that we're stronger than ever before can be attributed to all the SEO work I've been doing since February. I don't think I can say for sure whether our recovery was 100% Google's algorithm update or 100% work that I've done, but rather a bit of both.

> Do you think removing ad units was any help, if so how are you replacing the loss in revenue? Would you consider bringing back some of the ad units now that you have recovered?
I've mentioned multiple times in this thread (if you were to read allllllllll eight pages lol) that there's NOTHING that I've done JUST to appease Google, but every change I've made I did for the betterment of the community and our visitor satisfaction. Fewer ad placements don't necessarily mean less overall eCPM per page.

> You mentioned early on that you removed the popop that encouraged new visitors to register with your site. Will you consider bringing it back now that you have recovered?
No, because for all of the additional registrations we got, we learned through this process that it created even more frustration and a bad taste in the mouth of many visitors.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 7/30/11 11:42 AM
I'm seeing you on quite a few searches at #2 Dani. Your site was always good aside from some poor English. Not meant as a jab, but maybe it had something to do with it. I always found the site helpful and aside from the "plz send the codes" people, I knew I could find a reference to some coding question I had.

And thank you soooo much for removing that popup! You have no idea how annoying it was, but your info was always great. I hit back a lot, but now I actually stay to read your content. 
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 7/30/11 2:26 PM
Nah - but Dani confirms what I've been saying all along. That there is a "checklist"

You can have some items on the checklist - but when you have a certain amount OVER that amount - you get pandalized...

Removing that one piece of straw that overbalanced the scales does not correct it --- you have to actually almost remove EVERYTHING on the list...

In other words - a site with -20 can still pas panda but a site that makes -21 is pandalized ==== and cannot recover until they are -10 or below.


That's my theory right now.

But Dani - Regardless - I followed everything you did closely - as you know.... I also think you're absolutely right about all the links part --- you had massive links out - and when those sites lost "value" - I think it dragged you... But I don't think that was the massive points... You REALLY did an overhaul on the site... and I'm pretty sure that your work tipped the scales back into balance.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Atul_india1 7/30/11 8:45 PM
@Dani of DaniWeb - congrats on achieving the lost traffic again I will see ur video HOpe it helps all of us also
as I m experiencing great loss for my site www.oceanofweb.com

Although, it constant for www.techofweb.com

Thanks A Lot
Atul Bansal
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content TheDonald 7/30/11 9:35 PM
Thank you Dani for taking the time to answer.

>> I've mentioned multiple times in this thread (if you were to read allllllllll eight pages lol)
Ha. I've read all 8 pages 3 times. no kidding! Nobody else has recovered from Panda and certainly no one else has documented it. I will continue to read any insights you have on recovering from Panda and so will thousands of other webmasters. Trust me, we are desperate and there is no other source of information or hope. Just a boatload of articles from so called SEO experts who don't even have a website of their own! How many times can we go to Webmaster World and read the same speculation over and over again. Google has said nothing, which is just an absolute crime, you certainly know that, you were in our situation just a short while ago. You have no idea how many people wish they were in your shoes right now.

>>that there's NOTHING that I've done JUST to appease Google, but every change I've made I did for the betterment of the community and our visitor satisfaction.
Panda has forced me to look at my website differently. It's forced me too make changes I never would have done otherwise. I've learned a lot. I've increased time on site by a full minute. The benefits I'm seeing after removing a popup (-30% of total revenue) and moving my highest paying ad unit so it's not front and center  (-20% of total revenue), is that my website is viewed as less trashy. I am getting offers from folks in the Advertising world who would not of attempted to reach out to me before. Hopefully it will open up other revenue streams because right now the 50% drop in revenue per visitor in addition to a 30% drop in visitors is really hurting.

>> Fewer ad placements don't necessarily mean less overall eCPM per page.
That is very reassuring.
I am seeing that with my own website where moving ad units will impact CTR by 50% and eCPM will only be down 30%. I'm just wondering if I need to just remove 1 ad unit entirely to recover from Panda.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 8/1/11 12:53 AM
Ha. I've read all 8 pages 3 times. no kidding! Nobody else has recovered from Panda and certainly no one else has documented it.

I have seen the data for several sites that recovered (partially or completely) from Panda downgrades.  There have been very few publicly documented cases of Panda recoveries.  The majority of recoveries are, in my experience, being shared privately.

However, no one knows exactly what led to the recoveries.  Anyone who claims to know is probably just making egregious marketing claims.

In a couple of cases the sites that recovered did absolutely nothing.  It would seem that as the Panda algorithm learns more about what the Web looks like it changes its assessment of some site designs.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content zihara 8/8/11 7:18 AM
Since the first run of Panda I saw my traffic start a rise that finally doubled my volumes by the third week in July. Then came a plateau that continues still. A lot of my competitors had been Pandalized and continued searches have seen them slowly coming back into the SERPs at close to their previous locations. In looking at their sites I don't see where they've made any real changes (and I look deep under the hoods, too).  From my perspective, it looks more like a silent policy reversal by Google than anything else, and many folks just waited it out (it's not like we haven't seen similar action from Google before). I've also noticed a virtually overnight 20% drop in AdSense earnings on essentially stable traffic volumes and click-through levels since that July Panda run...
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 8/8/11 11:59 PM
Zihara - I think there was a time frame too - like a penalty box --- if you will - that if you didn't have everything that you were penalized for fixed - you stayed in another round - or were only allowed to poke your head out...

I'm guessing this - because one of my blogs made a FULL recovery - and one made a 3/4 --- and the only difference was Tags on Images...  and one duplicated group of articles.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content zihara 8/9/11 6:17 AM
One good thing about this, Lyrical: some of us actually went ahead and started really cranking up our presentations, cleaning our messes and reorienting more towards our visitor experiences. That's ground we probably wouldn't have covered if Google hadn't shaken the tree as hard as they did.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 8/9/11 10:37 AM
LMAO - that's probably true - and I have seen a TON of better work on a lot of sites lately.

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 8/9/11 3:36 PM
Agreed that Panda forced us to reexamine things. My old philosophy was to not fix what wasn't broken. We were doing SOOO well with seo, I was just afraid to rock the boat. Panda forced me to fix a lot of the usability issues that I had wanted to work on for a long time but was afraid to do ... such as redoing the entire URL structure. In addition, it forced me to re-examine things that I hadn't noticed before and really improve the site as a whole.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content zihara 8/9/11 6:07 PM
I, too, hesitate to fix what isn't broken but I'm definitely addicted to tweaking what works. And we simply never know when the good engineers at Google might let the monster loose again, because those folks are definite tweakers, too... and they have a history of tripping over themselves to release half-baked software on the world. So as much as most of my stuff pops on the first page of the SERPs in my niche (and has for years), I spend a lot of time working to get my materials well considered by specific other modules of the Googlebot algo, too, modules that are generally ignored by the majority of folks in the business world. Imagine being able to tap into 20% of the Internet-browsing public that is basically ignored... and funneling all the relevant traffic in that group in your own direction.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 8/11/11 1:18 PM
I am constantly tweaking. However, Panda afforded me the option to completely overhaul in a way that I otherwise would have been way too nervous to tackle. The feeling that it couldn't really get much worse was a sense of comfort in a twisted kind of way.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content LyricalQuestion 8/11/11 2:22 PM
+DANI --- ha ha ha ha --- I so want to PLUS one you....


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content shukla_vikas 9/4/11 9:25 AM
Looks like daniweb is down again.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 9/17/11 9:45 PM
> Looks like daniweb is down again.

What do you mean? Traffic has actually never been higher for us. We just had our two highest trafficked days ever. Once we recovered from Panda (which I can only assume was during some type of algorithm rollback, because from what I understand quite a number of other early-on Panda losers recovered at the same time), traffic soared higher than it was pre-Panda. Presumably because of all of the SEO work that was done while we were hit finally paying off now that Panda isn't stifling us.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content zihara 9/17/11 11:19 PM
I just spent a couple days at the GooglePlex. There was almost no attention paid to Panda by the Googlers at any time during the trip, and many of us asked several times. From what I've seen of the traffic curves, Panda threw folks into a sandbox for a preset period of time and then they were released. Their movement back into the upper reaches of the SERPs hasn't been nearly as quick as their downfall was but they do all seem to be back. Some did the work, some did not. Things aren't back exactly as they were but it really looks like Panda has been de-fanged... for now.

There are other elements in the mix that we also asked about and we usually got better attention for that. The best part of the entire affair was the final session where we were closeted with Product Managers, Community Managers and Engineers and we were asked what we would like to see Google do better at. You want to see people come out with some very explicit answers... you should have been a fly on those walls. Time will tell if we were truly heard but the Googlers in the rooms got an earful of really good stuff from our perspectives.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 9/30/11 6:58 AM
DaniWeb did, indeed, make a completely more than 110% recovery. We went from averaging 280-290K pageviews (every so often hitting 300K) pre-Panda, to consistently being at 370K post-Panda.

However, we were hit again on Wednesday, September 28th, once again losing more than half of our traffic. I think this might even be a bigger hit than last time. I am still investigating whether or not this was another iteration of Panda that just went out or something different? ...
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content zihara 9/30/11 10:41 AM
I've been tracking traffic curves on a number of sites, all in different niches. When you overlay all the curves on each other and factor in the site content variations, nothing makes any sense. It's very much like Panda is a fish out of water, flapping around and gasping for breath. I don't think Google is going to be happy with what they are doing until they've demoted/penalized every site on the planet, except maybe the .gov and .edu domains and the biggest of the big boys (IBM, HP, WalMart, etc.) And just when you think they've finally got a handle on what they're doing, out comes the nuclear weaponry again: all it takes is one misplaced special character in the command line...


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content JimmyL01 9/30/11 10:48 AM
Moderators, please close this thread, it's really old and just way way too long.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Michael Martinez 9/30/11 11:24 AM
So, Google DID confirm that it's a Panda iteration.


Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Lysis 9/30/11 11:32 AM
I think it would be helpful to move the conversation to the new thread Dani made, but I hesitate to be so intrusive on a long thread and close down a long thread. 

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Becky Sharpe 9/30/11 11:38 AM
@ JimmyL01 - this is neither too long - check out its big brother http://goo.gl/bmCyx nor old. 

For those of us following it is an important and useful narrative.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content Dani of DaniWeb 9/30/11 11:39 AM
I am going to continue to post status updates and document everything only in the new thread. I don't have the time to continue to maintain both threads. However, I feel like if those who participated in this thread want to continue their discussions, they should be allowed to. Personally, I would not like some of the very drawn out back-and-forth debates that went on in this thread to overflow to the new thread. While we were hit by Panda yet again, it is a very different set of circumstances this time, and I'd prefer to start it as a fresh discussion, while letting this very long and very involved discussion continue naturally.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content StevieD_Web 9/30/11 11:45 AM
Thanks Lysis.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content technin 8/28/12 2:40 AM
i want to know that republish (only half of article not full )
any other website's  articles is fine or not ?

doing this will have something to  do to get fully de-indexed & penalized website or not ?
and in doing this is their something to follow that can make this fine and not get penalized & de-indexed 

please help me this issue

Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content luzie 8/28/12 2:53 AM
Open an own thread techn4alldotcom, DO NOT revive months old threads unrelated to your own issue.
Re: We lost over half our US traffic ... We're a discussion forum with an editorial side, 100% unique content technin 8/28/12 3:36 AM
thanks
and i did that before
but no replies till now
please check  this

https://productforums.google.com/forum/#!msg/webmasters/LhxCIKQaDjY/pkmtdhjEh34J
More topics »