i cud c that this video got uploaded 3 years back, is that still valid , I mean the features navigatons

in what way do you pay that $1? as i am starting to see groups of talent or upcomming talent that understand that infrastructure costs money. but yet at the same time they usually are still learning or either they are trying to learn new technoglogies for a new job or afterrr having lost a job. but it is one thing to be able to have enoug recources to learn at your own pace but i can tell you iff you have to pay for cloud compute that require a credit card for which some of these people cannot even get acces to. and that i notice increasing amounts of people not being able to learn or study new technologies because it is 1$ there and a couple api requests to there. but in the end it is either that or being able to feed yourself for every day of the week.

Is there a way to connect this to a custom domain? I want to create a simple web app of chat interface

Thank you!

Thanks so much for the crystal clear explanations! You understand them so well and it's even more amazing how you show them in bullet points and graphs to make your audience understand as well!

Hi Julien, thank you for the valuable contribution. your vids are impressive, love it. May i have one question, that do deploying Llama3 on sagemaker have met HIPAA compliance, Because the data is highly confidential. Hope you will help with an answer. Thank you!

Can i use these transformers models for persian language?

very informative as always !

Hi I have question , I have deployed finetuned llama3 on AWS , but it generate repeated answers if I adjust payload it cut off the end of response , please provide me solution on this I have multiple times changed max lengths, max tokens size also still facing that issue

Hey Julien, Where can we find the training model video for food dataset? Also, I am trying to use a model and deploy it on Hugging Face Inference, but it errors out saying I need a config.json file. I'm not sure how to create it. Any leads would be really helpful. Thanks!

Thank you for your videos😊 Can I know if, when we have 5 pictures for each of two different subject(assume a1 person and a2 person), using stable diffusion can we generate these tree situation in one model? 1.a1 person in different background 2. a2 person in different backgrounds 3. a1 and a2 in the same image with different background . Do I need to create 3 model for that task? How to do that type of task ?

Theres very serious academic individuals that swear that LLMs reason

hi Julien! Do you have any tips on how can I convert a ComfyUI workflow SD1.5 based model to 🤗 or run directly on INF2 ?

Thanks a lot!

Thank you for this. Please I will like to know how can I query this endpoint from a web service? or if there is any guide you can point me to.

Thanks for the overview!! Excellent!!! In terms of sizing for inference, is there a way to calculate the KV Cache or maybe the overall HBM memory usage base on these optimizations.

Thank you. Agreed, there is zero reasoning.

Seems like a great video, but I can't move from the starting line. You seem to be skipping over very important details about how to deal with the HuggingFace tokens, AWS security keys, regional compatibility settings with Sagemaker, etc. For example, when running the copied SageMaker code, I get "ValueError: Must setup local AWS configuration with a region supported by SageMaker", but no region seems I try seems to work. Did you cut all the authentication code from your demo? Obviously you don't want to disclose security keys, but at least show/explain that part of the setup code and simply redact the sensitive information.

Thanks a lot for this!

Hi julien, is it possible to finetune this model using my own data consisting of videos?

Thanks Julien, your videos are really helpful.

OK but how do I use any model on hugging face I want? Who wants to deploy a model that doesn't have any value prop over GPT4 or Claude (e.g. uncensored)?

This is AWESOME! I've been pulling my hair out with AWS custom chips over the last few days. Thank you!!!

Thanks for the great video! The costs per M tokens seem really high in the neuron inf2 benchmark table. For Llama-3-70B they are around $115 per M tokens. Even with 3-Year reserved instances the prices would still be way above GPT-4 token price and with a similar throughput . Are VLLM or TensorRT-LLM on GPU’s better options than Inferentia2? or can VLLM be combined with this?

great video

Thank u Julien appreciate this so much right now

Would you mind sharing the slides please Sir? Thank you!

I have been wanting to understand quantization for a very long time. Thank you! Would you mind sharing the slides please? Thank you.

很快就讲清楚了，好厉害！爱来自瓷器。

do you need to have a description to fine tune it? is it possible to just have an image dataset without a description? and it can learn to produce those images at random?

Thanks for the tutorial. Greatly appreciated..

Amazing! Julien do you have any plan on doing a distributed parallel algorithms? (Data par, inter layer, intra layer etc)?

Love your content! thank you!

Is it possible to fine tune a model such that I feed it images of OpenMaps locations and then it randomly generates its own locations? In this case, would I even need to label the data? Is it possible to train the model on unlabelled data? I just want it to take a set of X maps locations, and then start inventing its own. Is this possible? Thanks!

Thanks for this. Loved the video ! Love what they do at Arcee ! You really got me at the 20th minut: maybe autoML works after all 😂 100%

I can't help wondering if there is an experiment which really fully discovers those technique like applying to all kinds of models or combining different methods together?

Thank you Julien.

Can I deploy my own huggingface model to vertex? My model says it is endpoints compatible but I don't see anything indicating vertex ai compatibility or how to achieve that

How do you export to onnx using cuda? It seems optimum doesnt support it - is there an alternative?

Thanks Julien for the two demos. Good options for developers.

what the difference between setfit.exporters.onnx and optimum.onnxruntime (optimizer = ORTModelFromFeatureExtraction.from_pretrained(...) optimizer.optimize()) etc.?

This is fantastic!

Amazing video!

I am going to use inf2 to run finetuned llama 3 70B should be great, I am curious about token gen speed on inf2 different sizes, if you can as a side note mention that in your next video, like this created it at x token/s

Great video!

The interface and everything has changed since this video. Can you provide an updated video that walks through the process of loading a module from huggingface into stagemaker jumpstart?

is that work with 3rd gen or 4th gen cpu's ?

Hi Julien, thanks a lot for your videos. I am pretty much new in this word and I have the task to understand how to deploy serverless using SageMaker. Will you do a new video about Serverless using Sagemaker? There is something new that we can use ? In the video you mention we needed to use boto3, is this still like this? Thanks a lot!

thanks julien i dont like this service im struggling myself

Great video Julien, thank you! Does the model have to be pre-compiled to run on AWS (EC2 or SageMager)?