Crawl4AI: The Ultimate Web Scraping Tool for AI🚀

[@unclecode]

Thanks a lot for reviewing Crowd4AI! It feels great to have a KZbinr I’ve been subscribed to for a while give some attention to an open-source project I’ve contributed to. Really appreciate it! The good news is we’re about to release a major update with lots of exciting changes. It’s now fully asynchronous, making it much faster, and we’ve added the ability to create structured outputs even without using an LLM, so it’s lightning quick at extracting repetitive patterns into JSON. We’ve also added some cool features like filtering for markdown outputs (e.g., removing headers, footers, keeping just the main article) and already supporting JS script execution, screenshots, and more. And we’ve got a scraper engine coming soon. Again, thanks so much, stay tuned, and I’ll keep you updated when the new version drops, hopefully for another review!

[@AIAnytime]

Thanks for your words @unclecode .... Great job with Crawl4AI and looking forward to the exciting changes.

[@unclecode]

@@AIAnytime will keep you updated, and looking forward for more reviews 🙌

[@themax2go]

that is amazing, thank you for your contribution! i'm wondering how it works with sites that use different captchas though

[@sathishbabu3322]

Hello is there anyway how to convert a Jupyter notebook file to .py file.. I don't use Google collab and I want to use my visual studio code to develop this Same app

[@anshgoyal3671]

One question - What if I want to also crawl the content from its sub links also.

[@nktthegreat]

United fan? Already liked, subscribed and commented.

[@AIAnytime]

GGMU

[@Shortwrit]

Bro Does it extract all the data from the webpages including the subpages too.

[@siddheshphaple342]

Great, I actually want to download the files(doc, pdf etc) available on my org website. Can I use CrawlAI for this??

[@syedibrahimkhalil786]

How about extracting social media data, like comments, posts? Mass data extraction for analysis on crowdsourced data?

[@RahulParmar-i6v]

will this be able to crawl the youtube videos which has captions in it?

[@irfanshaikh262]

Bro, the content is top notch. Thanks for that. But hear me out. Could you do a video on an AI which will accept prompts and snaps to return UI code like html and css. Im aware of openwebui, but need more. Thanks again

[@AbdulHadi-hl3el]

Hey there , nice tutorial...will it work for a website with a large number of followup links??

[@marcopiasentin6387]

Thank you for your work. In Europe we always have a popup that ask for accepting cookies. Do you know any scraper that bypass this?

[@GiggleM11]

Can you make apps using Replit or Cursor from scratch.

[@VenkatesanVenkat-fd4hg]

Great video....

[@AIAnytime]

Thanks!

[@charanvelugu]

Bro i want to dectect the person who is doing miscellaneous activities in shops ( like taking chocolate keeping in his pocket without paying bill) for this qwen2 can we us e it? And 2 the video is coming from CCTV i want monitor continues how to monitor continues using vision llm modal please explain ANNA

[@proudestberozgaar]

Will this work on sites with high security and authentication features??

[@AIAnytime]

No

[@mohsenghafari7652]

thanks

[@AIAnytime]

Welcome

[@user4-j1w]

❤

[@GGamerWPlayed]

no pasa nada, he visto mejores, que solo HABLANDO te da bien ordenado y visualmente lo que quieres del url

[@amrithnath8528]

Bro i have a requirement to build a bot which is managing the company policies...any queries asked by the employee to be answered on the basis of the policy plain text? How can i implement it? Pls give a proper guidance? I have seen one of your video kzbin.info/www/bejne/pmnNhYJ-lN56mcUsi=NtrCGCErgx-YpvmP Is this one helpful?