What Sites/Content Are Undetectable to Google's Search Engine Spiders?

Tweet

Hello MyLot'ers, I know there are certain types of website content that are protected by privacy and are off limits to Google's search engine spiders (including Yahoo, Ask, etc...) There have also been times when I try to access a topic on a forum because the Thread Topic showed up in search results, but when I actually clicked on the link to the results page, the forum wouldn't allow me to access the information and a message displayed telling me I must sign up or log in, in order to see this information. Also, this is true with facebook. Can you imagine if all the messages and conversations posted on the walls of facebook user's turned up in search results? Facebook also makes you login in, in order to view certain information on the site, so that is another example of website content not detectable by search engine spiders. So other than forums (some forums, not all) and facebook (along with other social media sites that are similar) are not detected by the search engine spiders? I thought Google Books might be one since it requires acrobat or adobe to read them. But that is not the case, they DO show up in search results. Also, There are sites out there that contain information that is only accessible once a payment is submitted, for instance, membership to a site that teaches online study courses. Those pages contain information that is not accessible unless a payment for membership is made. Once payment is received, then they unblock access and then the member can see everything in the workshops, study guides, etc... Are these sites not indexed by the domain owner? How do these types of webpages stay out of the view of the spiders crawling the net? The reason I ask is because some discussions (amongst other things) that I have engaged in on the internet have turned up in searche results, which surprised me because I thought this information could only be read by those who are registered members of the site. I was a little shocked to find them turning up with simple keyword searches. Another example is, a site I belong to has a "Pro" section, which is off limits to newbies and you cannot access this area until you have been an active member on the site for 90 days. So if this information could simply be accessed by running a search, then nobody would need to wait out the 90 days before accessing it cause they could just do a search and click on the results and view it that way, so how are some sections of information on a website kept out of the search results, while others are not and can be seen by anyone who searches using certain "keywords" ??