Deepfake anyone with just 1 photo - FREE & Open Source

Hallo tutorial: open-source AI to animate any photo with any audio
#deepfake #aivideo #aitools #ai #agi #singularity
Hallo home page
fudan-generative-vision.githu...
Hallo for windows
github.com/sdbds/hallo-for-wi...
HuggingFace space (This is the official one. Use this instead of "multimodalart" as shown in the video)
huggingface.co/spaces/fudan-g...
ffmpeg
ffmpeg.org/
Newsletter: aisearch.substack.com/
Find AI tools & jobs: ai-search.io/
Donate: ko-fi.com/aisearch
Here's my equipment, in case you're wondering:
GPU: RTX 4080 amzn.to/3OCOJ8e
Mouse/Keyboard: ALOGIC Echelon bit.ly/alogic-echelon
Mic: Shure SM7B amzn.to/3DErjt1
Audio interface: Scarlett Solo amzn.to/3qELMeu
CPU: i9 11900K amzn.to/3KmYs0b
0:00 Best face animators right now
04:15 Hallo examples
10:33 Settings to customize the animation
14:11 Specs
14:47 HuggingFace space
16:24 Tests
21:05 How to install locally
28:40 How to use locally

[@Zuluknob]

"realistic" you haven't seen the monsters i have created

[@theAIsearch]

lol is it some good nightmare fuel?

[@Zuluknob]

@@theAIsearch yeh, don't add people with teeth showing

[@xenn2996]

@@theAIsearch Can i install hallo with an external ssd?

[@nathanielthomas7718]

Yeah… the monsters I created just turned 21 & 18 👹🧌😳

[@Zuluknob]

@@nathanielthomas7718 Ahh, you downloaded pregnancy by mistake.

[@ihzakarunia2408]

Him: "This is a gpu that you and i could easily buy...." (RTX 4090) Me: .........

[@NetTubeUser]

Because coders only use the latest technologies available, without any prior optimization. For example, there's an optimized version of Stable Diffusion that can use older video cards, not just the latest ones. And the result is the same as with newer video cards. Of course, it is faster with newer video cards, but the result remains the same.

[@pixobit5882]

Would you rather buy a 4090 for 1.8K USD or a H100 for 28k USD?

[@BestieKing]

AI's growing really fast!

[@theAIsearch]

😃

[@NetTubeUser]

Yes, but the rending is still pretty robotic and emotionless.

[@kippwieland6464]

Like a malignant tumor.

[@azhuransmx126]

Nice!! Agents will have a face generated image but it will be animated not by prompts but in real time with no latency by an LLM behind and those same outputs can be used to animate a robotic physical version. The Last Stage of this tech during the AGI Incarnation in 2028.

[@Velvie_cakes]

Thank you for being honest about the ones that look weird. For some reason, a lot of KZbinrs are applauding these crappy looking pictures and videos.??? I like the honesty

[@theAIsearch]

No problem!!

[@I.Am.Nobody]

some are just fanboys of anything AI it seems, or they simply lack any form of taste (not a creative, but more a techy)

[@youtubesurfer134]

Man now things are really starting to pick up.

[@theAIsearch]

indeed!

[@simplereport8040]

Thanks for the nightmares on that hugginface anime one! 😂 Loved it. I’m trying it to troll my friends tonight after work.

[@theAIsearch]

Have fun!

[@GreenHatAnimation]

Hey, I'm a new sub! Loved your last vid, watching this one now!

[@theAIsearch]

Awesome! Thank you!

[@02167631]

Great video! Loving the step by step screen share tutorial!

[@theAIsearch]

Thanks!

[@TealDreamIneo-ki3kq]

Suspect: "I didn't do it." Investigator: "We have video evidence."

[@danielornstein300]

finaly i can imagine my father and sister as if they were alive again 😭😭

[@theAIsearch]

Interesting use case 🥲

[@rgarlinyc]

As for"laughing and coughing" @21:05 -->. they did say to use "Only English", so we shouldn't be surprised that it failed here.

[@islandofmisfityoutubers6734]

This is incredible!!💗

[@theAIsearch]

😃

[@researchgrouponhealthylivi108]

Excellent review and content !

[@theAIsearch]

Thanks!

[@gynowhere]

In a sequence that feel natural: VASA>EMO>Hallo>Hedra

[@numbers93]

OMG Harry Potter talking portraits here we gooooo

[@theAIsearch]

😃

[@darkflux]

6:05 watch the old guy's beard spazz out!

[@hermannrudas]

All the fears you used to have, now have been redrawn !

[@theAIsearch]

😃

[@brianmonarchcomedy]

Awesome! Do they have a version where you can input a video rather than a photo? Thanks!

[@theAIsearch]

not that I know of. will keep you posted

[@archiveglide]

Thank you for the info:)

[@theAIsearch]

no problem

[@JoeyBlogs007]

Extremely disturbing. Organised crime field day.

[@xenn2996]

thanks im gonna use this for my animations

[@theAIsearch]

😃

[@NetTubeUser]

The only drawback to this technology concerns the eyes and head movements. When you're talking to someone or looking at a video camera, cell phone, etc., your eyes aren't staring blearily at the lens all the time, or even at a person, all the time due to our process of thoughts. Even our face moves when we talk. Only robots, or AI, do not do that. But coders are not psychologists anyway.

[@benfrank6520]

Im not really sure if i should be happy or terrified that this is closed source. Those who have full access can do insanely good deepfakes. Propaganda will reach a whole new level.

[@theAIsearch]

this tech can definitely be dangerous. imagine all the forged evidence it can create

[@shooteru]

I went in this panic in February 2023. I realized the internet is on a risk to die. I archived lots of movies, documentaries, lectures, went on eBay and ordered encyclopedias for all fields, just in case. Helped me be more relaxed working with AI and accepting the journey

[@yeeehaaa-rb4ku]

@@theAIsearch I think in court it will probably be not too hard to figure out if it's AI generated, by proffesionals who can examine it (but who knows, maybe people will find a way to make it hard to prove, we'll have to wait and see) But it will have huge consequenses on social media in general Most people already don't read beyond the headlines. So even if it will be debunked as AI generated, the damage will already be done So yeah. The potential of this tech is terrifying

[@NoidoDev]

It's obvious: People just need to learn that they can't trust such videos.

[@yeeehaaa-rb4ku]

1. The tech is in early stages and developing very quickly. Eventually it will not be obvious at all 2. Many are already fooled by obvious false social media posts.

[@FriscoFatseas]

good coverage appreciate it

[@theAIsearch]

No problem 👍

[@NoidoDev]

Looks like an interesting alternative to Hedra. I like that it is more configurable. Future models should maybe have an option to describe the character and take this into account when doing the animation. For example age, or if it's an anime or some kind of doll.

[@theAIsearch]

that'd be cool

[@sherpya]

it says English only but you played a demo clip with Italian audio

[@theAIsearch]

good point. it'll work if the language consists of mostly english syllables

[@thegoodthebadandtheugly579]

This is great for animating drawn images.. 👀

[@djr3386]

Thanks for the sharing the video. Is there an app or AI plugin to correct weird AI generated faces?

[@theAIsearch]

you could try using stable diffusion img2img & inpaint to fix weird parts. are you familiar with how to use stable diffusion?

[@djr3386]

​@@theAIsearch Sorry no. There should be an AI app made specifically for correcting faces and fingers. AI can make other things.

[@sirdrakey]

Debates this year are going to be wild!😅

[@leandrogoethals6599]

the memes are gonna be fire this year!

[@theAIsearch]

😃

[@KryzysX]

11:27 that's really the most believable

[@eSKAone-]

Dude with high movement and high expression it looks way more convincing

[@theAIsearch]

agreed. tweaking the settings can make it very natural

[@Scorpiove]

Hallo can be installed easily with pinokio

[@invitia901]

Still looks like the video has been dubbed, almost right, but still somewhat wrong

[@dariustakeda1609]

How do you see the open source alternatives comparing to the copyrighted ones? Are they getting close? I think that with open source, we could not only deal better with AI ethics ("Skynet", "big Brother", etc.), but we could get better results, since AI needs lot of data and open source helps with accessibility = it's free. I really appreciate your work btw :D

[@theAIsearch]

open source is as good or only a bit behind. eg. Hallo is as good as Hedra, which is the best closed model you can use right now. i think Meta is also releasing an open source competitor to GPT-4o which will be natively multimodal. expect open source to only be a few months behind

[@dariustakeda1609]

@@theAIsearch thank you!

[@Capy_random]

Absolutely crazy and beautiful. Imagine a utopia where there are no degenerates hungry for power or malicious intentions. Humanity, being kind, would achieve greatness in such a short time.

[@Derpyzilla894]

I am confused how this achives that future.... I just want ai gene editing and then boom utopia.... hopefully.

[@theAIsearch]

@Derpyzilla894 hope we figure out a way to live forever before i die!

[@Capy_random]

​@@Derpyzilla894 It's not about 'image' generation, you need to see further. There are many powerful models out there, probably some really close to AGI, maybe from Anthropic or OpenAI or some national research private labs. The point is, if we were in a utopia, those models would already be out there. But you can't let an AGI roam free with so many mentally unstable people, asking how to create explosives, making deepfakes of people in NSFW contexts; the list goes on. That's why many AIs are so restricted.

[@Capy_random]

@@theAIsearch soulkiller from cyberpunk 2077? hahaha

[@Capy_random]

@@Derpyzilla894 It's not about 'image' generation. There are many powerful models out there, probably some really close to AGI, maybe from Anthropic or OpenAI or some national research private labs. The point is, if we were in a utopia, those models would already be out there. But you can't let an AGI roam free with so many mentally unstable people, asking how to create explosives, making deepfakes of people in NSFW contexts; the list goes on. That's why many AIs are so restricted.

[@paultimson6674]

the image of the beast, yaah. off to heaven we go...

[@ghhdgjjfjjggj]

If only we had this around during the peak of StarCraft! The custom mods we can make!!

[@theAIsearch]

😃

[@MarcoCholo-iz9js]

Now you're thinking

[@MrCrazyGameGuy]

I think I'll give it another 5 months, then look back at how you said "it's the most realistic" and then be like "That looked like shit" lmao

[@theAIsearch]

yep, that's the world of AI

[@mrfounderr]

I can't find the links to those songs you mentioned in the video. Btw, great job.

[@theAIsearch]

thanks. it's in the middle of this page fudan-generative-vision.github.io/hallo/

[@I.Am.Nobody]

​ @theAIsearch I need a higher version of Python for other AI stuff I do. How to run this on the lower version without deleting my higher version?

[@budekins542]

Thanks A.I Search.

[@theAIsearch]

You're welcome

[@JCtheMusicMan_]

I would love to be able to help train a model if I have lots of video for reference. I would think that could make things more realistic?

[@TomiTom1234]

While the installation, a red error message appears says: ERROR: Package 'hallo' requires a different Python: 3.9.5 not in '=3.10' Although I checked the Python version, and it is 3.10.11 .. Why that?

[@povang]

They hardly blink at all...

[@Keenan686]

19:47 this image is haunted

[@theAIsearch]

lol

[@Yogsoggeth]

I've been using Vidnoz to do deep fakes of streamers and send the video to them in discord. Never fails to get a laugh! You can make them say what ever you want? Ever wanted to be a VIP or a mod in chat? Now you can make a fake video of the streamer promoting you! KEKW

[@holdthetruthhostage]

The problem with Hedra minute limit face morphs

[@mmmvvv5134]

jajaja, los ejemplos cripis son graciosos Hahaha, the creepy examples are so funnies

[@Sayan_B]

can you mention the minimum GPU requirement for running it locally ?

[@sherpya]

you don't need admin prompt when doing git clone, but only to enable ps1 script execution

[@theAIsearch]

thanks for the tip!

[@NeilRoy]

This is both amazing, and worrying, as I wonder how this could be abused, especially in politics.

[@theAIsearch]

I'm sure this will be heavily abused in politics

[@gingerversestudios]

gooona try it hope it works

[@chunlingjohnnyliu2889]

We can finally make the new generation meme with sound.

[@theAIsearch]

😃

[@Darknight_84930]

Hm..... More and more I look at these new models I feel like someone is definitely gonna slap a model into bot and let em learn continuous and wallah we made the first ever humanoid.

[@studiosoho]

Did you know that TikTok already had this "effect" before all this AI hype, right?

[@SS801.]

Ty 👍

[@theAIsearch]

no problem

[@budekins542]

Can someone please explain something in plain English. .if I increase the RAM of my computer with an external hard drive will I be able to install HALLO?!

[@theAIsearch]

no, you need VRAM which is your GPU's memory. increasing just RAM won't work

[@thee_streets]

Man I installed everything and followed every instruction, but upon trying to make a video, it takes me a very long time to render. I use an RTX 3080 8GB GDDR6 VRAM. What Can I do to speed up the rendering. Now it takes like 7 hours to make a video, help me please.

[@Avalon19511]

Well I tried this with an actual gradio interface that someone was kind enough to send me the link and for right now it's just too slow and when I say slow I mean really slow, for a 2 minute video it was going on 5 hours when I finally shut it down. I hope at some point they optimize it because it looks interesting

[@xrayban2]

Maybe AI will take care of the Earth after wiping us from it ...

[@theAIsearch]

It'll do a much better job than us

[@johnangel5480]

26:56 double clicking the .ps1 file doesnt run anything for me, it only opens the notepad and displays the file's code what am i doing wrong?

[@theAIsearch]

right click and run with powershell instead of notepad

[@marcthenarc868]

@@theAIsearch@johnangel5480 He's probably using a conda environment and that's a big issue for those who don't want to install python system-wide but rather in the conda sandbox. Search for "How can I activate a Conda environment from PowerShell?" in Stack Overflow.

[@bigdaddy5303]

Ug 10 mins with a 4090. Here i was hoping for real timr

[@MaverickM1]

Looks very unnatural tbh, I am wondering why people hyping this ai shit all the time

[@hbasm3271]

I have CUDA version 6 with 512mb ram. Can it work on CPU only

[@JohnnyVent]

The way the Hedra anime girl just slowly got worse 😭

[@theAIsearch]

lol yes

[@TheMCDStudio]

Pinokio has a one click script install for this.

[@theAIsearch]

Thanks for sharing!

[@AceGod7]

this release is going to force the others to make it free. with tech is who gets their version out first. especially if its free.

[@BeardofBeesPool]

reface had something similar to hello a couple years ago. I think they removed it.

[@user-qr4jf4tv2x]

with this ai eye contact it will help you make confidence afraid of you

[@MaxSMoke777]

Nobody's cooked up a little web interface for this? Having to do it all through the command prompt is very basic.

[@ArcanePath360]

"A GPU which you or I could easily buy" ... if you don't care about eating and heating for the rest of the year and want to dip into your mortgage.

[@theAIsearch]

lol just gotta live in your parents basement, sell your car, eat costco free samples every day, and save up for a few months

[@budekins542]

Thanks

[@theAIsearch]

my pleasure!

[@DaaHela]

Microsoft one is the best tbh

[@CfopCubing]

Well this is terrifying.

[@theAIsearch]

😃

[@plagiats]

Do we REALLY need that comment under every single AI video? It's progress, embrace it.

[@RiiahTV]

my folder didnt come with the powershell ps1 file

[@dknc7393]

If this tech was more matured, cheating could be possible in the presidential debate 2024

[@9A4GEMilan]

Does not work: ModuleNotFoundError: No module named 'triton'

[@BazzilioSPb]

Error: This Space only works in duplicated instances.

[@theAIsearch]

yes, you need to duplicate it and pay for compute unfortunately. around $0.8/h

[@BazzilioSPb]

@@theAIsearch And for pc version to?

[@IamS.S.I]

👍👍

[@silentpenguin3001]

Washington DC is already using this but it doesn’t look as convincing.

[@theAIsearch]

they are definitely using this!

[@skullwund8676]

i don't understand why this exist and what is the porpouse of this

[@DezorianGuy]

I only have one question. Ist this Open source and free?

[@theAIsearch]

yes

[@DezorianGuy]

@@theAIsearch OK, then I will take a look.

[@PavelReyes]

After all that, and I still get ModuleNotFoundError: No module named 'torch'

[@S.I.F.A.R]

yeah finally I can made harry potte moving pictuf

[@theAIsearch]

😃

[@SimonFeay]

I duplicate the space and follow all the steps and just get a red "error" message.

[@theAIsearch]

what's the error?

[@SimonFeay]

@@theAIsearch The error is literally "Error" in red writing with no explanation. This error appears in the Output Window. It goes through all the steps of building with lots of sliding bars and then the red error is the output. It gives me absolutely nothing to go on. Thanks for spending some cycles on this.

[@Entity303GB]

finallyy

[@theAIsearch]

😃

[@Entity303GB]

@@theAIsearch I love your Videos about new AI tools 🫶🏻

[@LydianMelody]

Might wanna cut out the racial slur from the movie clip or whatever it is at 5:35 …

[@theAIsearch]

Thanks! I'm not sure what he's saying but I'll trim it out just in case

[@LydianMelody]

@@theAIsearch I figured it was something like that, no worries. Keep up the great work

[@Lorentz_Factor]

How is sad talker doing these days ? Here's the thing that was out a year and a half ago

[@veggieterry8853]

Please cough to verify you are human. 😅

[@MuradBeybalaev]

That doesn't look good for anime. Everyone knows animes speak in low framerate.

[@baheth3elmy16]

2:40 Really?

[@TheMCDStudio]

There is a ComfyUI node for this.

[@theAIsearch]

Thanks for sharing!

[@Avalon19511]

I keep getting a import error for Hallo, any thoughts?

[@Avalon19511]

Something is odd, finally got the node to appear all nodes are in the green, but when I hit Que the Hallo node flashes, then the preview node but nothing happens, is your install working?

[@theAIsearch]

@@Avalon19511 i haven't tried the comfyui fork. hopefully someone else can help you out here

[@potts995]

I feel like companies could provide a layer of security by making users generate profile-style images only, kinda like how thispersondoesnotexist does, for example. I could also see a lot of potential if we were easily allowed to save and reuse those same faces via face swapping, rather than having to rely on a new image with a new face each time.

[@derbybOyzZ]

Taking 10 mins for a 10 second generation is too long. If people want to have chatbots with an image of their face it needs to not be video and instead something else.

[@musicbro8225]

lol, this is literally putting words into other peoples mouths! The legitimate uses for this are virtually zero. All it proves to me is developers of AI have lost the plot, if they ever 'had' the plot in the first place...

[@squirrelhallowino29]

I mean you could change the title a bit right, use your platform, share the info, but you know, titles are important when it comes to these subjects. I will dislike the video for the lack of self awareness +-+

[@RiiahTV]

is there anyone with the ps1 file?

[@Zenless260]

lol not downloable, and censored, dying as soon as a real open source releases. thats probably in one week lmao.