• October 30, 2014
October 30, 2014, 7:47:12 PM *
Welcome, Guest. Please login or register.

Login with your Chronicle username and password
News: Talk online about your experiences as an adjunct, visiting assistant professor, postdoc, or other contract faculty member.
 
Pages: 1 [2]
  Print  
Author Topic: Bots reading my website?  (Read 17762 times)
bwwm1
Senior member
****
Posts: 409


« Reply #15 on: April 09, 2012, 9:42:07 PM »

Try http://ip-lookup.net/.

It may tell if you the lookup is from a university or something else.
Logged
bookishone
Distinguished Senior Member
*****
Posts: 1,980


« Reply #16 on: April 09, 2012, 9:55:13 PM »

Try http://ip-lookup.net/.

It may tell if you the lookup is from a university or something else.

That's a helpful site!
But actually, Statcounter already tells me, I think, the same thing. The Indonesia bot is using Jt Telkom Indonesia, for example. Some of the hits are from universities (it'll say, "University of Exeter") but these don't have the "bot" characteristics -- they tend to come in from Google or another referring link, and they meander around the site a bit.
Logged

My tag line is false.
wet_blanket
Some kind of
Distinguished Senior Member
*****
Posts: 7,025


« Reply #17 on: April 10, 2012, 10:12:52 AM »

I'm not very tech savvy, and I do agree that the most likely explanation is bots, but aren't there things actual people users can do so you couldn't see which page they'd come from?  Various browsers have "secret" browsing options, and I bet there are more high tech and effective options for the truly paranoid.
Logged

Let us let wet_blanket have the last word.
untenured
On far too many committees
Member-Moderator
Distinguished Senior Member
*****
Posts: 6,078


« Reply #18 on: April 10, 2012, 10:47:30 AM »

This is an interesting question. I would post to a forum that specializes in bots and spam.  You'd get a better answer than from academics.
Logged

Quote from: kedves link=topic=56697.msg1152543#msg1152543
You are among the Pure and Truthful, however small their Number.
My goodness, that was an exceptionally good analysis of the forum.
bwwm1
Senior member
****
Posts: 409


« Reply #19 on: April 10, 2012, 1:27:35 PM »

My site just registered a hit from Heard and McDonald Islands. They are currently uninhabited.
Logged
scampster
Distinguished Senior Member
*****
Posts: 10,879


« Reply #20 on: April 10, 2012, 2:37:55 PM »

My site just registered a hit from Heard and McDonald Islands. They are currently uninhabited.

That's what you think! I would assume that makes them the perfect place for an evil lair.
Logged

When you are a scientist your opinions and prejudices become facts. Science is like magic that way!
bookishone
Distinguished Senior Member
*****
Posts: 1,980


« Reply #21 on: April 20, 2012, 10:30:20 AM »

I have Statcounter installed on my academic website, mostly because it was so helpful back when I was on the market to know that Someone from School I'd Applied To had checked out my supplemental materials online.

I don't check it that often these days, but recently I noticed an odd pattern. I sometimes get one-time hits from faraway places -- Russia, Thailand, South Africa -- that I cannot believe would be due to people from those countries actually hunting down my teaching materials to read them. (And there's "no referring link," so they're not coming in via Googling my name or anything). I've always just assumed that these are web bots crawling the internet touching on everything they find.

But I notice now that in each case, every bot (?) goes straight to a particular page of my site, and then departs without moving on to any other page. It's not my home page, and it's always the same page, the one with a brief description of my current research. I'm not too concerned about this, since as I said I don't really think there are people all over the world trying to access this for its content -- I keep it updated more for potential students or collaborators here in the US.

But I am wondering why this one fairly simple page -- it's just some text content under a banner -- attracts the bots' attention somehow, and none of my other twenty pages do, even when some of those pages have been updated much more recently.

Is this something I should be worried about -- could something have been embedded there without my knowing, that attracts these fly-by-night visits solely to that one page of the site? Or some wording that, unbeknownst to me, has a street meaning that is attracting these visitors? Or is it just random bots and coincidence?

I first noticed this about a month ago. I get about 4-5 visits a week of this type, and they always hop on (and off) that one page.



This happens when google or some other search engine bot find some good information on page. Google or other search bot visit every page of your site, but not store all the pages. This page content changes frequently so bot visit 4-5 times in a week.


That's what I thought, too, but I've been changing material on the other pages more frequently than the "current project" page, and yet still they hop onto the "current project" page and none other. Weird.
Logged

My tag line is false.
wcardinal
New member
*
Posts: 8


« Reply #22 on: July 19, 2012, 3:29:24 PM »

There isn't really anything to worry about since most bots are just search engine servers around the world...

statcounters, though, are very unreliable. Bots are one reason...  Have you tried using Google Analytics instead?

Logged
nordic_gray
New member
*
Posts: 6


« Reply #23 on: August 26, 2012, 1:13:21 PM »

Ah, nearly 200 views now and a dozen responses -- much better, even if 11 of you are bots.

I am leaning toward the thought that perhaps some phrase in my statement is attracting them (it is the page with my current book project listed, actually). I suppose we'll never know, but it is a little creepy to have automated "fans" in places I'll never get to visit.

I will try the "registering with Google Webmaster" thing and report back if I find anything useful. As far as I know, no one is linking to this page from anywhere else. I really don't have a big profile like that!

[Edited to add, in reply to bwwm1: Statcounter gives me an IP address and originating location: yesterday, four hits from Tangerang, Jawa Barat, Indonesia, all to that page, and two hits from Tbilisi, Dushet'is Raioni, Georgia, to that page; or, last month, hits from Rowville, Victoria, Australia; Kaunas, Kauno Apskritis, Lithuania; Sofia, Grad Sofiya, Bulgaria; Tel Aviv, Israel; Barcelona, Catalonia, Spain; Minsk, Belarus; Makati, Manila, Philippines; Jiddah, Makkah, Saudi Arabia.... You get the idea. ALL of them, ONLY to that single page, though sometimes multiple times to that page, only minutes apart. Weird. It does make me realize I've got to get out my globe to remind myself of my world geography, which is embarrassingly weak.]

And seniorscholar, I'll PM you about the conference next fall! One of the best reasons to go to conferences, as far as I'm concerned -- catching up with fellow bookish types, who definitely aren't bots.


I live and work in Barselona, Catalonia and also noticed that bots (?) check my websites from some weird  places like Norman, Fort Collins, Pocatello, Ann Arbor, New Haven, Murfreesboro, Madison, and others.

I trust this is a relatively new phenomenon as my University's almanacs from 1400s and 1500s mention no bots from these bizarre loci expressing interest in our scholarship.
Logged
southerntransplant
A man on a porcupine fence and a
Distinguished Senior Member
*****
Posts: 11,105

No recess.


« Reply #24 on: August 26, 2012, 1:56:39 PM »

Ah, nearly 200 views now and a dozen responses -- much better, even if 11 of you are bots.

I am leaning toward the thought that perhaps some phrase in my statement is attracting them (it is the page with my current book project listed, actually). I suppose we'll never know, but it is a little creepy to have automated "fans" in places I'll never get to visit.

I will try the "registering with Google Webmaster" thing and report back if I find anything useful. As far as I know, no one is linking to this page from anywhere else. I really don't have a big profile like that!

[Edited to add, in reply to bwwm1: Statcounter gives me an IP address and originating location: yesterday, four hits from Tangerang, Jawa Barat, Indonesia, all to that page, and two hits from Tbilisi, Dushet'is Raioni, Georgia, to that page; or, last month, hits from Rowville, Victoria, Australia; Kaunas, Kauno Apskritis, Lithuania; Sofia, Grad Sofiya, Bulgaria; Tel Aviv, Israel; Barcelona, Catalonia, Spain; Minsk, Belarus; Makati, Manila, Philippines; Jiddah, Makkah, Saudi Arabia.... You get the idea. ALL of them, ONLY to that single page, though sometimes multiple times to that page, only minutes apart. Weird. It does make me realize I've got to get out my globe to remind myself of my world geography, which is embarrassingly weak.]

And seniorscholar, I'll PM you about the conference next fall! One of the best reasons to go to conferences, as far as I'm concerned -- catching up with fellow bookish types, who definitely aren't bots.


I live and work in Barselona, Catalonia and also noticed that bots (?) check my websites from some weird  places like Norman, Fort Collins, Pocatello, Ann Arbor, New Haven, Murfreesboro, Madison, and others.

I trust this is a relatively new phenomenon as my University's almanacs from 1400s and 1500s mention no bots from these bizarre loci expressing interest in our scholarship.

Each one of the cities you listed has a university.
Logged

"...And on the other side of this wall is a whole 'nother studio that you'll never get to see...because, you know, fvck you guys."

Steve Albini, showing Dave Grohl and the Foo Fighters around his studio
untenured
On far too many committees
Member-Moderator
Distinguished Senior Member
*****
Posts: 6,078


« Reply #25 on: August 26, 2012, 11:59:47 PM »

Ah, nearly 200 views now and a dozen responses -- much better, even if 11 of you are bots.

I am leaning toward the thought that perhaps some phrase in my statement is attracting them (it is the page with my current book project listed, actually). I suppose we'll never know, but it is a little creepy to have automated "fans" in places I'll never get to visit.

I will try the "registering with Google Webmaster" thing and report back if I find anything useful. As far as I know, no one is linking to this page from anywhere else. I really don't have a big profile like that!

[Edited to add, in reply to bwwm1: Statcounter gives me an IP address and originating location: yesterday, four hits from Tangerang, Jawa Barat, Indonesia, all to that page, and two hits from Tbilisi, Dushet'is Raioni, Georgia, to that page; or, last month, hits from Rowville, Victoria, Australia; Kaunas, Kauno Apskritis, Lithuania; Sofia, Grad Sofiya, Bulgaria; Tel Aviv, Israel; Barcelona, Catalonia, Spain; Minsk, Belarus; Makati, Manila, Philippines; Jiddah, Makkah, Saudi Arabia.... You get the idea. ALL of them, ONLY to that single page, though sometimes multiple times to that page, only minutes apart. Weird. It does make me realize I've got to get out my globe to remind myself of my world geography, which is embarrassingly weak.]

And seniorscholar, I'll PM you about the conference next fall! One of the best reasons to go to conferences, as far as I'm concerned -- catching up with fellow bookish types, who definitely aren't bots.


I live and work in Barselona, Catalonia and also noticed that bots (?) check my websites from some weird  places like Norman, Fort Collins, Pocatello, Ann Arbor, New Haven, Murfreesboro, Madison, and others.

I trust this is a relatively new phenomenon as my University's almanacs from 1400s and 1500s mention no bots from these bizarre loci expressing interest in our scholarship.

Each one of the cities you listed has a university.

I thought the post was tongue and cheek humor.  I stand corrected!
Logged

Quote from: kedves link=topic=56697.msg1152543#msg1152543
You are among the Pure and Truthful, however small their Number.
My goodness, that was an exceptionally good analysis of the forum.
southerntransplant
A man on a porcupine fence and a
Distinguished Senior Member
*****
Posts: 11,105

No recess.


« Reply #26 on: August 27, 2012, 12:04:29 AM »

Ah, nearly 200 views now and a dozen responses -- much better, even if 11 of you are bots.

I am leaning toward the thought that perhaps some phrase in my statement is attracting them (it is the page with my current book project listed, actually). I suppose we'll never know, but it is a little creepy to have automated "fans" in places I'll never get to visit.

I will try the "registering with Google Webmaster" thing and report back if I find anything useful. As far as I know, no one is linking to this page from anywhere else. I really don't have a big profile like that!

[Edited to add, in reply to bwwm1: Statcounter gives me an IP address and originating location: yesterday, four hits from Tangerang, Jawa Barat, Indonesia, all to that page, and two hits from Tbilisi, Dushet'is Raioni, Georgia, to that page; or, last month, hits from Rowville, Victoria, Australia; Kaunas, Kauno Apskritis, Lithuania; Sofia, Grad Sofiya, Bulgaria; Tel Aviv, Israel; Barcelona, Catalonia, Spain; Minsk, Belarus; Makati, Manila, Philippines; Jiddah, Makkah, Saudi Arabia.... You get the idea. ALL of them, ONLY to that single page, though sometimes multiple times to that page, only minutes apart. Weird. It does make me realize I've got to get out my globe to remind myself of my world geography, which is embarrassingly weak.]

And seniorscholar, I'll PM you about the conference next fall! One of the best reasons to go to conferences, as far as I'm concerned -- catching up with fellow bookish types, who definitely aren't bots.


I live and work in Barselona, Catalonia and also noticed that bots (?) check my websites from some weird  places like Norman, Fort Collins, Pocatello, Ann Arbor, New Haven, Murfreesboro, Madison, and others.

I trust this is a relatively new phenomenon as my University's almanacs from 1400s and 1500s mention no bots from these bizarre loci expressing interest in our scholarship.

Each one of the cities you listed has a university.

I thought the post was tongue and cheek humor.  I stand corrected!

It might be. I'm not good at picking that up.
Logged

"...And on the other side of this wall is a whole 'nother studio that you'll never get to see...because, you know, fvck you guys."

Steve Albini, showing Dave Grohl and the Foo Fighters around his studio
Pages: 1 [2]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.9 | SMF © 2006-2008, Simple Machines LLC Valid XHTML 1.0! Valid CSS!
  • 1255 Twenty-Third St., N.W.
  • Washington, D.C. 20037
subscribe today

Get the insight you need for success in academe.