robots.txt Category

How To Stop Search Engines From Indexing Certain Pages

by Bobby Martinez
Thursday, February 26th, 2009

Did you know that you can stop Google from indexing certain pages on your website? You can, with a certain tool called “robots.txt” Robots.txt, as you might have already guessed, is a text file that you can put on your website that will direct programs that crawl the web (web crawling bots) and give them […]

Person Coding Image

Latest & Greatest

Press Room

WEBii Ranked as a 2024 Top Ethnic Minority-Owned Business in Austin
Building Success Interview with Website Planet

Contributors

avatar for Julissa GuadagniJulissa Guadagni (4)
avatar for Jacqueline SinexJacqueline Sinex (214)
avatar for Jane Baxter LynnJane Baxter Lynn (1)
avatar for Lara HollersLara Hollers (1)
avatar for Kathleen HackneyKathleen Hackney (1)
avatar for Aimee JohnsonAimee Johnson (1)
avatar for Lidia S. HovhanLidia S. Hovhan (1)
avatar for Jessica PerkinsJessica Perkins (2)
avatar for Megan MarshallMegan Marshall (137)
avatar for Elizabeth FoleyElizabeth Foley (1)
avatar for Nathan MontgomeryNathan Montgomery (1)
avatar for Melanie ReynaMelanie Reyna (1)
avatar for Laura PattersonLaura Patterson (1)
avatar for Karina HarchandaniKarina Harchandani (6)
avatar for Maisie CantrellMaisie Cantrell (44)
avatar for Eli NewmanEli Newman (7)
avatar for Ryan FeldmanRyan Feldman (3)
avatar for Marketing EditorMarketing Editor (9)
avatar for Ruth HawkRuth Hawk (198)
avatar for Stacey WelchleyStacey Welchley (4)
avatar for Phillip SmithPhillip Smith (5)
avatar for Meredith SchraederMeredith Schraeder (2)
avatar for Jessica JonesJessica Jones (2)
avatar for Jen SayrooJen Sayroo (1)
avatar for Samantha RiveraSamantha Rivera (2)
avatar for Lizzy CederbergLizzy Cederberg (6)
avatar for Rachel SchelterRachel Schelter (3)
avatar for Alan SafaiAlan Safai (5)
avatar for Carmen SutherlandCarmen Sutherland (1)
avatar for Brian DisbotBrian Disbot (18)
avatar for Joey McGirrJoey McGirr (1)
avatar for Lanie CrowLanie Crow (3)
avatar for Ally HuggAlly Hugg (10)
avatar for Bryan LokeyBryan Lokey (13)
avatar for Bobby MartinezBobby Martinez (20)
avatar for MorganMorgan (6)
avatar for Rachael PierceRachael Pierce (27)
Top Web Developer in Austin

Web Insights, The WEBii Blog. Copyright 2022. WEBii.net. WebXess, Inc. Since 1996. 8500 Shoal Creek Blvd. Suite 4-104, Austin, TX 78757.