|Invalid clicks problem - Possible solution||Robotvsbadger||5/16/12 2:19 PM|
So i noticed that a lot of us are experiencing invalid clicks on the image ads and after about a month of pulling my hair out and reading these forums it seems that User Pinkvisions may have found the source
It appears that it is the amazonaws.com bots that are the issue and have been "clicking" our ads...
The question is, what is the best way to combat this? Pinkvisions mentions he blocked it completely... How is this done? and without damaging SEO with the good bots? (i read that stumble uses amazonaws.com, but that was on an old thread on WMW)
I've done some research and have found no definitive answer..
Maybe together we can sort this out.... or even google might be so good to intervene and help us?
|Re: Invalid clicks problem - Possible solution||Vascant||5/16/12 3:53 PM|
I posted the solution in one of these threads... I will post it again and try to remember everything I stated then
1. This will most likely kill Amazon Affiliate, I do not use it so could not test it but based on what I see.. it will
2. Add this code to the end of your .htaccess file. Since I created it and added it to mine I have been ACS bot free and no Invalid Clicks.
deny from amazonaws.com
Deny from 220.127.116.11/14
Deny from 18.104.22.168/17
Deny from 22.214.171.124/16
Deny from 126.96.36.199/14
Deny from 188.8.131.52/16
Deny from 184.108.40.206/12
Deny from 220.127.116.11/18
Deny from 18.104.22.168/19
Deny from 22.214.171.124/19
Deny from 126.96.36.199/17
Deny from 188.8.131.52/17
Deny from 184.108.40.206/17
Deny from 220.127.116.11/21
Deny from 18.104.22.168/14
Deny from 22.214.171.124/18
Deny from 126.96.36.199/16
Deny from 188.8.131.52/17
Deny from 184.108.40.206/18
Deny from 220.127.116.11/16
Deny from 18.104.22.168/17
Deny from 22.214.171.124/15
Deny from 126.96.36.199/17
Deny from 188.8.131.52/24
Deny from 184.108.40.206/16
Deny from 220.127.116.11/17
Deny from 18.104.22.168/20
allow from all
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/16/12 3:59 PM|
That is amazing, thank you so much! I'm going to try it now!
|Re: Invalid clicks problem - Possible solution||Gracey||5/16/12 5:35 PM|
That's rather strange. I don't have the amazon bot crawling my sites at all. If it's the affiliate bot, wouldn't you think you'd need to have an affiliate code on your site for them to send the crawler?
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/16/12 5:44 PM|
I have just implemented Vascant's code (about an hour or so ago) and can confirm that so far it is working... I don't think it's an "Amazon bot" as such but rather one of many bad bots that hey host... !??
|Re: Invalid clicks problem - Possible solution||zihara||5/16/12 6:59 PM|
The Amazon cloud has been implicated in certain problems across the web... something about how their cloud offering works and who they allow to use it.
|Re: Invalid clicks problem - Possible solution||Vascant||5/16/12 7:30 PM|
As an example, ACS bots only crawl 1 of my sites. They do not crawl another site I have on the same IP nor sites I have on other IP's. These bots are not run by Amazon based on what I have learned thus far the bots come from users on their Cloud Service.
I think this explains why some people were seeing this issue months ago (something I also learned) while others are just now experiencing it first hand (like me). If Amazon does nothing to clean up their cloud sooner or later though it will get around to everyone I think.
|Re: Invalid clicks problem - Possible solution||Gracey||5/16/12 8:12 PM|
The IP addresses are not static though, and at least a couple of them can't even be resolved - which seems to mean, you need to keep a constant vigil on the IP addresses?
My IP blocking function can't find a number of them at all - "unresolved" and "can't be resolved". I haven't checked yet, but I assume the host is allocated a block of IP addresses so blocking all the IP addresses in the domain would stop a lot of it? (which is what I did)
Not sure it's really worth the effort unless there are a lot of visits from those domains though. I don't see any in my stats for the last little while. I only get the odd few invalid clicks here and there, and those I think might be from visitors using proxies.
|Re: Invalid clicks problem - Possible solution||Vascant||5/16/12 9:16 PM|
I acquired the IP's either from my own logs from attacks against my website and then finished it up with a few I found by looking up domains that Amazon CS owns. Yes, this will be an effort that required checking monthly to ensure your website is secure. Is it worth your time.. since you are not being attacked at this point in time probably not nor probably understand completely perhaps what those of us who were attacked experienced. I only say that from my personal experience of reading about bots on ACS several weeks ago but had not even seen what they were talking about so I dismissed it. Now I have taken a personal interest in it as you can see.
I merely provided the code I used to secure my website, if people wish to use it then that is good with me...I took the stance of rather then blaming Adsense for my Invalid Clicks I investigated and resolved the cause of the clicks before they became invalid :)
|Re: Invalid clicks problem - Possible solution||Gracey||5/17/12 3:57 AM|
Yeah, currently I might get three to four invalid clicks every few weeks or so, but in any case, I think it's just wise to check what IPs are allocated to AEW and block them all at the moment.
It's nice to see pubs taking the initiative to figure out what's causing invalid clicks for them and then sharing it though. That will be a good help to some other pubs, and I'm going to bookmark this thread and the one by pinkvisions for future reference.
Very helpful - thanks.
|Re: Invalid clicks problem - Possible solution||Dan B.||5/17/12 4:26 AM|
I am thinking it's a good time now to mention that people look up the htaccess blacklist from Perishable Press...I've been using it for years as a small part of my defensive measures...it blocks a HUGE list of known spambots, scanbots, downloaders, known nefarious IP addresses and such...would be wise to implement that, and then, of course, suppliment it with your own finding of bad visitors.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/17/12 5:56 AM|
I woke up this morning and it looks like your the block list has been partially effective but am still getting about 20-ish% invalid clicks so i'm missing something. Sorry to bother you again Vascant but could you tell me how you found the damaging ip's. Statcounter, Analytics etc... I'm not too up on this and i would be really grateful if you could point me in the right direction.
|Re: Invalid clicks problem - Possible solution||Vascant||5/17/12 6:17 AM|
First thing you need to do is determine where the clicks are coming from, now I know a lot of people dismiss StatCounter however it is probably the easiest and most effective measure for this kind of thing. I say this because StatCounter will count a click even if the person accidentally clicks and ad and closes it before it registers with Google, which for the purpose of finding possible security issues is a good thing in my opinion. Once I figured where they were coming from, I have kept all my logs from Apache and searched through them to collect any other ips to round out the list.
The key for me was using statcounter to get me looking in the right direction. There is just simple to much data in the raw logs to go hunting without a clue as to what you are looking for.
@Dan: Thanks, never even seen that before but like the work gone into it.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/17/12 6:25 AM|
Thanks again... I'm really grateful for all your help!
|Re: Invalid clicks problem - Possible solution||pinkvisions||5/17/12 6:56 AM|
I used Adsense in Google Analytics to find which domain invalid clicks were coming from.
In Google Analytics, click on Standard reporting tab.
At this point on the right side you will see all your pages, clicks and adsense revenue.
Now click on secondary dimension to set another filter. Set Visitors->domain as secondary dimension.
Now go through the data to see which domains had clicks that generated 0 adsense revenue. I could just see the domain, not the IP address though.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/17/12 7:49 AM|
Thanks for the info Pinkvisions... i'll give that a try as well!
|Re: Invalid clicks problem - Possible solution||Dan B.||5/17/12 8:00 AM|
Good advice, though I would suggest perhaps not filtering people on who gave zero revenue, but who gave zero revenue and had some clicks.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/17/12 11:50 AM|
I just received around 20 invalids in 30 minutes so i guess it's back to the drawing board for me :/
I tried both methods mentioned above but to no avail...
Statcouner isn't showing the invalid clicks at all and analytics is also a dead end... (i really appreciate the help though guys)
I really wish we could get some support on this..
I have even contacted my hosting provider for help... i'm that desperate :P
|Re: Invalid clicks problem - Possible solution||Dan B.||5/17/12 12:08 PM|
Your hosting provider will be of little help, their job is to keep your server up and that's about it.
If you can find the source of those clicks by IP, country of origin, or dns name, might be worth doing some reverse lookups for trends...
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/17/12 12:21 PM|
Dreamhost has helped me a few times before on non related topics so i thought it was worth a shot.. :P
That is exactly what i'm trying to find but finding any info on those clicks is near to impossible. They are simply not showing on any reports except the adsense page :/
|Re: Invalid clicks problem - Possible solution||Vascant||5/17/12 1:15 PM|
I know my solution and problem where different but perhaps my initial steps or solution might help you here...If you already do not know break down your adsense into Url Channels and find where these clicks are taking place. Once you have this, then you can start taking additional steps like first on my list would be find out why Statcounter isn't catching it because in my experience if a user just looks at an ad it tends to record a click.
Once I narrowed it down my first action was to remove the ads from that page until I could get a handle on the situation. If you have alternate ad services, my next step was to put their ads on those pages. The reason for this is I can at least email them and get a response back, from there I am usually able to resolve the problem.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/17/12 9:24 PM|
Thanks again for the advice Vascant, i know the invalids are taking place in the footer of every page and an add on the front page. Oddly enough i have one ad on the sing post pages that doesn't get effected but that is brought in through comments.php so that might be why... it may be to deep for the bot... (just a guess).
I'm looking into the statcounter variable now
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/18/12 7:43 AM|
If google is listening... i would actually pay for a couple of emails from them... I'm loosing money and sleep because of this and wasting so much time going through logs when i could be publishing content.
I would gladly pay to speak to someone!
|Re: Invalid clicks problem - Possible solution||Ad Traffic Quality Team||5/22/12 10:25 AM|
This is Andrew from the Ad Traffic Quality Team at Google. We're aware of a problem with a third-party crawler in the network that is generating invalid activity across a number of different publishers. We've recently made some changes to our filtering so that this invalid activity will be excluded more quickly from your earnings reports and the discrepancy between estimated and actual earnings will be smaller. Robotvsbadger, I'll reach out to you separately via e-mail to check in on the problem.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/22/12 12:16 PM|
Email received and replied. Thank you again.
|Re: Invalid clicks problem - Possible solution||BigCostumeJoe||5/23/12 12:28 PM|
I too am having a problem with possible invalid clicks.
Any suggestions would be helpful.
|Re: Invalid clicks problem - Possible solution||Ad Traffic Quality Team||5/23/12 4:48 PM|
|Re: Invalid clicks problem - Possible solution||MountaineerFan||5/24/12 9:10 AM|
Any suggestions on what to do if your statcounters and Google Analytics aren't showing the same thing the Adsense dashboard is reporting? My adsense page says I have more clicks than I can find via any reporting information including what is installed on my own domain. I spend time every day trying to figure out the discrepancy because it seems to be causing an earnings drop. It's almost like the adsense dashboards aren't even running stats for my page, although all the data within the adsense pages say that they are. I have disallowed all domains that aren't mine from displaying ads. I installed a blocker for spiders and crawlers on the site it's self to keep them from creating any ghost clicks and I know it's working because my spammer hits have nearly disappeared. I removed amazon affiliates and blocked their crawlers. I am reading here and trying to apply any solutions offered. I am not sure what adsense is telling people privately in Emails, but if it would be helpful to the rest of us trying to resolve this I wish you would add it to the known issues or this forum for the rest of us to try on our sites.
|Re: Invalid clicks problem - Possible solution||camp ranger||5/24/12 10:35 AM|
I agree. We had over $1k (~50%) chopped off last month and didn't even get a response from Google as to what to try to do. We're an honest publisher looking to solve this issue without a clue as to what to try. This issue has only gotten worse as the months have gone by -- and the only thing we get from Google (via In Your City meetings) is 'invalid clicks or activity' -- which I'm calling BS since ~80% of our traffic is Organic Search, with another ~8-10% being from our social efforts.
|Re: Invalid clicks problem - Possible solution||Ad Traffic Quality Team||5/24/12 10:16 PM|
@MountaineerFan, have you seen this Help Center article which describes some of the reasons why AdSense and Analytics stats may differ?
@camp ranger, I remember speaking to you at AdSense In Your City. I apologize for dropping the ball on your follow-up, I thought I had replied earlier. I've just now replied with some info that I hope will help.
|Re: Invalid clicks problem - Possible solution||meyert11||5/25/12 3:24 PM|
This is amazing! My adsense account was just fine for months, then I added some amazon associate links and within 2 weeks my account was banned. In the first week I noticed strange IP activity and whois indicated it was amazon Ec2. I filed a complaint and they wrote back that appologized and were investigating the issue. Couple weeks later my Adsense account was banned. I didn't put 2 and 2 together until now. This may finally explain the problem. Is there anyway I can get someone to review my account with this new information? I'm afraid the appeal form by now will go into the abyss.
Also, doesn't this point to a dramatic inefficiency in the whole "ban for life without explanation" system. A ton of people may have gotten banned for a stinking amazon bot.
|Re: Invalid clicks problem - Possible solution||byrev||5/25/12 6:32 PM|
For me the problem lasted three days and disappeared on Wed, May 23, 2012 ...
It seems that there are no more invalid clicks ... until now.
|Re: Invalid clicks problem - Possible solution||Robotvsbadger||5/26/12 11:56 AM|
Has anyone noticed a crawler from Proximic (http://www.proximic.com/) in their logs? I think it may be the issue... I have tried to block it but the ip's are different each time with no logical range. I tried Robot.txt but it ignores it! Any ideas or experiences with them?
|Re: Invalid clicks problem - Possible solution||stol||7/1/12 6:55 AM|
I've received an email for Google AdSense that says this.
After reviewing our records, we've determined that your AdSense account
Does this mean the crawler was generating invalid clicks on my site?
|Re: Invalid clicks problem - Possible solution||MeeandMo||7/4/12 2:53 AM|
pinkvisions! Thanks so much!
To be honest - i've never really used analytics - but with your guide i can see that i also have clicks from amazonaws.com - that result in zero revenue.
So i'm hoping that once i block this with my htaccess file - my invalid traffic will stop.
It only just started happening on the 6th of June. Unless i've just never noticed it before.
|Re: Invalid clicks problem - Possible solution||David Shen||7/6/12 3:19 PM|
I need to talk to you about my invalid clicks too, because its been almost a year since my ad sense was taken away because of my friend clicking my ads over and over...
|Re: Invalid clicks problem - Possible solution||David Shen||7/6/12 3:20 PM|
Also how do I even check for invalid clicks? I've never even seen that option even though I've heard of it
|Re: Invalid clicks problem - Possible solution||MeeandMo||7/9/12 10:58 AM|
Oh - it seemed to work for awhile.
I got Invalid traffic on Jul 3rd.
Changed my htaccess file -- and did't get any more until Jul 9th. (So no invalid clicks for 5 days)
Until the htaccess was changed - i was getting a note every 2 days.
|Re: Invalid clicks problem - Possible solution||DesertCruiser||11/15/12 12:18 PM|
Andrew: After 4 years of using your Adsense service I've just noticed that my Adsense account finalized earnings is not being updated and there was an invalid traffic notice in "details" on Nov. 2nd for our site www.in-the-desert.com. We've changed nothing or added anything other than content to the site, and use no other advertisements other than yours, and only a few from Topaz Labs software, which has been on the site for quite a long time. We do have an Amazon Store sections. We are very carefull not to click on any adds or even mention them to anyone. Is there anything you can do to help with this? We run an clean site and plan to keep doing that. Any help would be greatly appreciated.
|Re: Invalid clicks problem - Possible solution||realestate pondicherry||2/13/13 8:38 PM|
I have submitted one of my website for adsense.
Couple of months later, i received a message 'due to invalid clicks ads are blocked'
I have resubmitted for consideration but got denied.
I HAVE FIVE MORE WEBSITES, INTERESTED TO GET 'ADSENSE' SUPPORT.
IS POSSIBLE TO SUBMIT MY OTHER WEBSITE FOR ADSENSE ?