Google Destroyed My AI Generated Website

[@deepdivesessions-dds]

lol, when you mentioned 1M pages, i knew it already.

[@rosenancy3382]

This reminds me of the old "auto-blogging" days. Hundreds of blog posts uploaded, indexed, then a few days later, every post disappeared from Google's search results

[@AmerikaMeraklisi-yr2xe]

I think you didnt watch video, He showed 3:14, 3.71k index pages. It still live.

[@rosenancy3382]

@AmerikaMeraklisi-yr2xe I watched the video. Like I said: This reminds me of the old "auto-blogging" days. Hundreds of blog posts uploaded, indexed, then a few days later, every post disappeared from Google's search results Do u have a problem with people sharing their experiences that differ from yours And judging by the number of people agreeing with my comment, it also reminds other people of the old "auto-blogging" days too Are u going to make a false accusation against them too n say all of them didn't watch the video

[@Something9008]

Vercel may be using 'cold boot' which means if you haven't had clicks within a certain period it has to 'warm up' on the first fresh click and then your site will act normally until the site goes without clicks for a certain period.

[@MrJoe1199]

Ouch that pretty much ENSURES websites on vercel will get un-indexed, even static ones.

[@BlueJDev]

There's a few issues at play here, as some have said vercel maybe using cold starts to save on resources. But the bigger issue is if an API call is needed to load the page data it should be set to lazyload the actual data from the API call after page load and not server side rendered. SSR is best done if you already have the data fetched from the API and saved to the database. Although this won't help if vercel has to spin up the database service before queries are sent to it. You can still use SSR to build the 'frame' of the page and meta data etc, then call the API once the page is loaded. Although Google may not pick up the meat of the page this way but if it is only the businesses information it shouldn't matter too much. I would also check vercel and nextjs caching settings are correctly set too.

[@JR-be1jf]

I will share a little of my experience. Your site uses API to create dynamic pages, right? When the Google bot tries to crawl more pages, it generates too many API calls. This can block the creation of dynamic pages and appear as server problems in the Google console!

[@OPMultiplayerCoopGames]

Keep doing your thing Hamish, don't mind the smartass comments out here. Extremely impressed with your work.

[@JimWellsIsGreat]

Just a note for those new to SEO: SEO isn't just about impressions. It's about getting your page to rank in the results AND getting them to click through to the site. Anything below a 3% CTR, something is wrong, whether it be the title, description, search intent, or other.

[@davidrivers2734]

Good point on click through

[@DaveThomson]

traffic is massively important.

[@robert301990]

You are not an idiot bro, we all get that itch to see what would happen with a lot of pages. Happens to all of us, after a few days I see little increments. I will add more content and do some backlinking

[@rh619]

stop doing clickbait on youtube, google will love you....

[@sumitdas6596]

😂

[@SportPrediction]

After years, I found one of the most knowledgeable, brilliant and most frank channel.

[@Mohamedtaha11338]

Am doing this for years though Python and web scraping, the number one cause for performance drop is error pages, whether soft errors, 500 internal errors, etc.. because they consume your crawel quota and they gives the user a bad experience and google hates that!

[@KeyvanHejrati]

Big fan of your videos. I recently tested a few websites and all of them have one thing in common. Many pages published suddenly, got impression and clicks hen fall of. I think large amount of pages when published it is considered as spam

[@northloo]

I told u like 5 videos ago ur problem is client side / server side relationships which i a common a huge problem lots of coders have + you are mismanaging the metadata resulting in your application structure being poor making it hard for google to efficiently crawl so much content. Make better apps with better page structure & manage metadata properly thats gonna solve it. WP manages that sort of stuff automatically that might be why your struggling here. Minor adjustements

[@SamaSama-o2i]

There is truth here. Launched a few bolt apps but the meta data on each page is a mess. And its been a challenge to get bolt to fix it, it runs into tons of errors..

[@northloo]

@SamaSama-o2i yeah because frankly this tech is very new. You can't get AI do all the work and do it this well. Even with bolt you still need decent coding skills. At leats a minimum. Like if you can't touch or read the code you can't make money with it IMO. OR pay someone on fiverr to get the rest of the job done u know what i mean?

[@SamaSama-o2i]

@ absolutely!! 1000%

[@HDHD-r4n]

that's some game right there 🤙

[@fitnessmusic8269]

So is Wordpress still better for indexing than a next js app?

[@redsix3921]

Hamish, I don't think you ever explained why we need to sign up for the data for SEO website in any of your videos? I've watched them all but I can't ever remember seeing why we need to sign up for that to make these websites?

[@Incomestreamsurfers]

It's to get business data to show on the pages.

[@chakibabowael6711]

But it's soo expensive, and vercel too

[@Incomestreamsurfers]

Vercel isn’t massively expensive neither is dataforseo if you build things in the correct way - you could also use Google’s own cloud API I just find it very complicated

[@RolloUnden]

​@@chakibabowael6711 Just deploy to your normal hosting server. Really not so hard. Literally when you're in cursor ask in chat how to do it and it will walk you through it. You use FTP and actions by setting secrets and variables in github. The process is like 5 minutes and i am not a dev

[@morganthemogul]

@@Incomestreamsurfers Can this be done with air table/google sheet/csv? I'm thinking if you scraped 500,000 businesses in an area, could the data still be called dynamically when a person views that directory listing

[@abraraslam3850]

Thank you for sharing your experience, was wondering if you can show us how to use dataforseo

[@quizbuddha]

I may be completely wrong here, but isn't using 43,000 towns way too granular. Maybe the largest cities by population and then break out the largest ones. There are tons of small towns in the UK that will use business services in neighbouring larger towns for example. I'd say something like 500 locations wold be both more manageable, cheaper and more useful to a user. And as you suggest, using the 4000 business categories doesn't make much sense...can't imagine too many searches for "Beach volleyball club" in Glasgow. Also, why reliance on API all the time? Wouldn't it be faster and cheaper to retrieve and store all the data in a database? Maybe use Google Places API and limit results to X per location, then run a cron job to periodically update perhaps only a subset of the data like reviews?

[@priitzirnask]

Thank you for sharing. I analyzed the platforms and would try digital ocean and might try incremental setup for the website lets say all the cities but with only 20 inital cathegories and adding cathegories over the time. Having a look at your server statistics, if it can handle it and also I read that google crawlers migt get overwhelmed with such large ammount of data. Glad to hear your thoughts about my ideas.

[@tomriley6929]

I love what your doing with your creativity. But all these mass websites full of crap directories or pointless information like numerology… Google has been killing them off for years and even more heavily in recent updates. Even Directory positions are given to trusted sources like yelp. It’s so cool you can spin up a huge programmatic website very fast but I don’t see a monetisation opportunity here at all.

[@Hills-VA]

This is why you keep things to yourself Google looking up your skirt is never good

[@ggggg744]

Hamish, thank you for these videos.

[@MrJoe1199]

IMHO the problem is the relatively long delay when a page is generated when google crawls it.

[@Incomestreamsurfers]

that is indeed the issue - Google counts this as a server delay, and flagged the site because of it.

[@HDHD-r4n]

@@Incomestreamsurfers I was thinking of this when I just built mine. During testing. seeing the page delay between loading the page and the API calls etc, wondering if there's a way to setup the server to render and deliver the pages quicker, or will it always be classed a server delay because of the way it's built/stack?

[@RolloUnden]

Have you put on bing analytics?

[@Incomestreamsurfers]

Yeah I did - or at least I submitted it to bing

[@Dawn-S]

What's wrong with failing? How else will you (we) learn??

[@NVX_Ink]

💡Before you release your next "BIG THING" website; maybe run some test.... Just a thought 😂

[@Incomestreamsurfers]

@@NVX_Ink I test in public - my other similar website is still doing extremely well.

[@NVX_Ink]

@Incomestreamsurfers But you can't even explain why positively. You just have theories. That's not testing.

[@AiGeneratedOnline]

I built a site and ranked also and google index’s 5k pages and charged me $274 hahaha But I found a new data source and the site is back up 😍 will keep reporting and going through this with you 😍😍

[@RolloUnden]

You're spending way too much for indexing. Try speedy index bot. Not affiliated but 5k credits is $30

[@AmerikaMeraklisi-yr2xe]

Whats the data source can you share ?

[@abdullahalpatasoy]

Thanks for clear explanation 😊

[@bradcage7345]

That's why I follow you

[@ZenVacu]

Does it work for DUTCH websites??

[@bob-p7x6j]

NO DUTCH! Googles new policy, check out their website.....

[@AWFJ2014]

And that's why I love your content.

[@sdrweb]

Go niche. Either choose a city or choose a business type.

[@Incomestreamsurfers]

Already recreated this website with only the large cities of the UK - gonna slowly expand it this time and see what happens

[@UtterNonsenseBuilds]

Of course it did. You can't index that many pages that quick or Google will destroy you.

[@t55rer]

I mean its not really surprising

[@edekl832]

Keep in mind that google emloyees watch channels like yours and besides that - haters can report you. Don't show your pages in public, all will be clapped sooner or later

[@StaykaZ]

Google Update now

[@RolloUnden]

Why even use google analytics? Just use an indexer? Less likely to get clapped. Less risk and i have seen no ranking benefits using google products.

[@GamePlay-pl2nv]

thank you for you efforts buy please make it more realistic instead of spamy

[@musumo1908]

Surprise!! 😂😂...these sites dont stick anymore...rewind 10 years...

[@Incomestreamsurfers]

Findatradespersonireland.com is working fine, there were some mistakes in this particular website.

[@upwork-123]

Anyone with 100.000 clicks from ai content 😂

[@kangcreator186]

Hey, what is your name, is it Ahmed???

[@Incomestreamsurfers]

Hamish

[@miraridoctorofficial]

So it proves that Google has eliminated AI articles.

[@clickpulse.1]