Hi!
It seems my previous Firefox trick (or hack) has gotten a bit famous, hasn’t it? Anyway, here’s another hack! man… am I turning into a life hacker or what? 🙄 This one’s a really curious thing to check out.
We all know what search engines do, right? Yes, search! Baby, search! They scour everywhere for everything. And what do website owners do? They use all sorts of SEO (search engine optimization) tricks to make things easier for search engines, so the SEs can crawl the directories inside their sites smoothly and bring more traffic! Hm… Okay, C.J., enough with the technical stuff! why are you telling us all this? Here’s why…
Did you know there are some directories on websites that owners don’t want the general public to see? Yep, there are! So, what do they do to hide them from us? They don’t exactly bribe the search engines 😀 nah! what they do is add a text file listing the directory names that should be skipped during searches. Search engines read that file first and leave those bits out of the search results page…
But you know us geeks can’t stay out of trouble, right? Yep! There’s a way to peek at those hidden directories! All you have to do is tack on /robots.txt after the domain name and see what the site owners don’t want you to find 😈
How about checking out what Mr. Bill Gates is hiding from us? 😀 Here you go:
http://www.microsoft.com/robots.txt
And what about Larry and Sergei? Yep!
http://www.google.com/robots.txt
Oh, and guys (and gals), you’d better not try this with WordPress.com, or our good old buddy Matt might kick me out of WP 8O. He’s a pretty open guy and probably doesn’t have much to hide :D, but why risk it? 😆
C.J.
Disclaimer: This is totally for educational purposes, and I’m not responsible for any trouble you get into with this!
powered by performancing firefox
SeeJay Online! 
i have a question on firefox
i like to use the bookmark toolbar. how can i have more than one toolbar? b/c my first toolbar is now filled.
thank you sir.
hi just wondered.
Have you tried opera and what do you think?
I have ie, firefox and opera installed. Use all three. If something works with opera I prefer to use opera. The only thing I hate about opera is the bookmarking tool.
I’ve used all three. i do not like IE because i do not have the tabbed browser feature. I shifted to firefox about an year ago, and have not checked on opera… cannot remember seeing much of difference between Opera and Firefox, but i found firefox aesthetically appealing 🙂
(plus, a friend who has more than a dozen addon says you do not get them in opera…)
Now, i shall visit some of my favourite sites and check the /robot.txt file!
Yes agni, you are right, firefox has more addons.
if, the sites comply with standards. Opera works much better than the other two. Unfortunately, this is rare.
yeah that’s cool.
the /robots.txt i so search engines basically can’t ‘see’ those directories.
it’s not nothing new?
but keep up the good work.
later
thatgeekyboy.com
Testing
Thanks guys ! believe me, Theres lot more from where that came from 😉
C.J.
Wow, don’t know suck hack…
*such…
The use of robots.txt isn’t the only way to hide a file from search engines. Another way is to use the robots meta-tag in either the file you want to hide or any files which link to it.
If used in a file you want to hide, the meta tag should look like one of the following (I’ve left out a ”
meta name=”robots” content=”follow,noindex”>
The first meta tag above is for a hidden file which links to files that are also hidden. The second is for a hidden file which links to files that are not hidden.
If used in a file which isn’t hidden, but which links to hidden files, the appropriate meta tag looks like this (again I’ve left out the ”
The “robots” meta-tag would be harder to hack than a “robots.txt” file, it seems to me. However, I’m not sure that all search engine robots honor the meta tags.
Hmm the website has a Does not exist error.
hey do you know how to get into Forbidden sites?
I’ve been trying but the 403 error is Evil and i can’t access http://www.browsehentai.com/ i don’t know why?
I’ve been able to use it before but it somehow got forbidden can you please help me gain access to the site
tquoaihec mizpwsq jdyk cgym aewrb cgxz dspalto
Way open proxy forbidden without a program
when i sing in this site (4shared.com) there is message forbiddent tel me the Way open forbidden without a program
thank for this reference
I tried to that /robots.txt on the website but it still wouldn’t let me in. Plz help! the website is asianfanatics.net
I’ve found a free anonymous browsing proxy here http://www.surfshuffle.com, thought I’d share with whoever is interested before it gets blocked
the (www.facebook.com) in our country is forbidden because the government want that … please can you help me to open it with any program or website
Thanks you!
not able to access all contents of this website can anyone help with this.
This piece of writing is really a pleasant one it assists new the web visitors, who are wishing for blogging.