Open Source AI Inference API w/ Together

Exploring the Together Inference API (www.together.ai/)
Together API basics jupyter notebook examples: github.com/Sen...
Neural Networks from Scratch book: nnfs.io
Channel membership: / @sentdex
Discord: / discord
Reddit: / sentdex
Support the content: pythonprogramm...
Twitter: / sentdex
Instagram: / sentdex
Facebook: / pythonprogramming.net
Twitch: / sentdex

[@Indie0]

Expecting a farming video to drop at some point. Congrats and Merry Christmas!

[@sentdex]

Self driving tractor incoming

[@jonathan-._.-]

Is it legal to build a nuclesr reactor if its on your own farm ? (Or how do you plan to power all those gpus ?)

[@filthyE]

⁠wake up babe, new @@sentdex farming video just dropped

[@Tideo123]

@@sentdex I always dream of having a hand effector robot to pick button mushrooms.

[@shyama5612]

This is really fascinating - that an API is so well engineered, its faster than a local inference. Kudos Together team. I'd love for Harrison to have them on and talk how they did this. So much to learn here.

[@carlosrivadulla8903]

Keep pushing the open source bro. No all heroes wear cape.

[@yossefdawoad4311]

recently used Together AI w/ Langchain and it's the BEST for developing with OSS LLMs , even go a head and added support for streaming in langchain itself, make it even better

[@tunadurnal3765]

I literally grew up watching your tutorials. Also learned English by them ! Thanks.

[@sentdex]

Hah, awesome to hear!

[@__napcakes__1911]

I use ollama for my local setup. It takes less than a minute to query new models (34b or less sizes, I have 64gb ram, rtx2080, and a 16 core Intel in my desktop pc) if you've downloaded the model already. Ollama sounds a lot like together api, maybe even more convenient. I change models all the time without rewriting any code, because there is only one http API that works with all models.

[@JC.72]

I am a bit surprised that sentdex isn't covering some of those tools out there which have good potential or are seemingly steering towards a correct path. I may be over thinking it but seriously hope it's not $ driven. Simply because at least to me he is like THE Ai. Guy.... Sort of like people's champ ai guy to me. In some way, its a bit letdown. why I see tons of other ai posters talking about it but not my guy. ;(

[@timjx3675]

Agree I use ollama its real easy and free, if you have 16-32 GB ram models like Mistral run really well on cpu

[@technolus5742]

Same, using ollama locally. But to run in the cloud, I rather use a service like together instead of setting up my own machine. And pricing probably ends up being lower (I haven't done detailed math on it tho). Someone also pointed out anyscale as a potentially cheaper (and faster) alternative. I think if sentdex was sponsored he would disclose it.

[@EnricoRos]

Ollama also has a /chat endpoint, which handles the Chat prompting for all the models, and you don't need to spend hours finding the exact prompt template. Very convenient.

[@ibbobud]

@@EnricoRoscombine ollama with litellm, and it even gets better. They just added support for ollama in silly tavern as well. Ollama for the win.

[@Sw3nssoN]

Just got the NNFS book in softcover! Its such a hefty book! Going to really enjoy reading into things! Keep doing what youre doing!

[@EnricoRos]

I want to love this API service - however, I can't adopt a provider that does not provide an Instruct and Chat template out of the box for every model. The job of figuring out the optimal prompt template is better done on that side of the API rather than forcing each and every developer to study and spend considerable time on any model. 15:00 This video exemplifies the challenge. 15:43 "would have saved me some trouble" So please, if Together is watching let's add: - a model list endpoint for UIs (mine is big-agi) for dynamic discovery. See the Openrouter list API, it's a good reference - chat and instruct templates, with chat built into the endpoint (see probably the Ollama chat endpoint for a good reference) Once these 2 very very simple changes are implemented, I'll add support in Big-AGI

[@GeorgeHill-Weffan]

Thanks

[@eliasgamezgarcia3414]

Merry Christmas dude ;)

[@sentdex]

Same to you!

[@pw7225]

If you're planning to become a farmer, get some Llama 2s or Alpacas or Vicuñas ;)

[@mitchelcharette8211]

Did you drop the neural networks from scratch series? It was just getting to the best part!

[@KellenChase]

Congrats on the move

[@sentdex]

Thanks!

[@custom3828]

Hey sentdex! I recently found your neural network from scratch playlist, and it's really great! However, I was wondering if you can finish you series? I know you have a book on it, a e-book might I add, but I'm almost 16, not old enough for a card, my country has a higher age to apply for a card, and my parents won't get me the book/e-book. So I'm wondering if you can finish the series as it would be really appreciated. I know it's not related to the video, but I would thought I'd just ask.

[@prakyathkini5540]

hi, i really needed help, can you please tell me how to summarize a pdf using together ai api, i am not able to code it properly

[@technolus5742]

They should also add at least one model with vision capability like llava or bakllava

[@KurtacusRex]

Im just getting into programming, peaking ahead to see what im getting myself into, and this dudes doing stardew valley for real.

[@tomski2671]

Wouldn't it be nice if the LLMs could tell you their prompt structure. 😁 Also nice - LMs that would help interface/translate between code and various LLMs

[@ALTINSEA1]

Merry Christmas 🎄🎄

[@sentdex]

Thank you, same to you!

[@trentleslie2888]

I used the KZbin Summarizer GPT on this video and it casually said "The presenter, who is in the process of moving to a ranch to become a farmer, mentions that Together is quick at...". Well here I am to confirm that unexpected statement and leaving a comment 😂 Also, I'm thinking that leveraging this for use with AutoGen or other agent group applications could be preeeeetty sweet.

[@DaRealDJ]

Wish this had chapter breakpoints so I know what parts to go back to for instructions.

[@AIVisionary-me8hd]

I am new to the channel. Do you know where I can learn ML for AWS Deep Racer?

[@DianaGuzman-c6l]

Is their a way to add history to the model?. In the playground they use "sessionKey" but that does not work when making an api call.

[@fire17102]

Good luck on the ranch! Dig a well & get solar FTW

[@sentdex]

Got a well. Any suggestions on solar?

[@serene3Dsoundscapes]

Api pricing looks insane!

[@awm19delta]

I’m sure this is a rookie question, but can someone tell me what python notebook is being used in this? I like the look of it better than what I’ve experienced so far with Jupyter. TIA!

[@sentdex]

I'm using the Jupiter extension with VSCode. Probably just the vscode part that's the difference.

[@Elzelgator]

@Sentdex did you find a open source model which is decent?

[@wurstelei1356]

Is Mixtral 8x7b open source ? I thought this is open weuights only...

[@Zomgwtfbbq1228]

you going mono crop or regenerative?

[@sentdex]

Not educated enough to share opinions on mono vs regenerative in terms of what you should do. I don't plan to focus on any 1 specific crop, would want to mainly subsistence, but also probably wouldn't qualify as regenerative. On a scale, I would be closer to regenerative and would be attempting to improve in that direction probably.

[@wege8409]

Are you planning on getting into agricultural robotics? Cause if you are that's dope, food is the means of our survival. I think it's important to start thinking about automating food production.

[@sentdex]

Thinking about it. Do you know of any good resources on it?

[@dan110024]

Just take your gta AI and hook it up to a John Deer and a camera. Profit. And some demolished farm houses.

[@phoneywheeze]

Hello, I'm new to this stuff. Would you make a video on how to prompt to get a json response? I have some text and I want to analyze it on some predefined parameters.

[@MagagnaJayzxui]

Thanks boss man 🎉

[@myhificloud]

@24:17 Farmers' Almanac to Neural Networks

[@satyasibhawvana]

Hey Sentdex, It's very fantastic and I am learning this quickly. Can you also post a video regarding how to build or extract 3D images from 2D images without data loss. Which would be helpful. I know it's a vast subject. Thanks to you for making it possible.

[@phils744]

Hi, like most people I'm working for a corporation the upside (it's indoors %98 of the time, free coffee and treats most of the time) including free water to drink. Not to mention climate controlled. The flip side is that I really don't enjoy it any more, the technology they are using is so outdated it hurts [mentality to work on] it is what it is. What I have go going on for myself is a HA (cluster, powered via proxmox) I have 6+ systems connected via dac 100G connet, looking to explore options and other opportunities. Suggestions are welcome

[@python_byte]

Are you gonna become the Farmer style from Interstellar?

[@zlarosav]

Ty

[@ylazerson]

you are awesome!

[@CM-zl2jw]

Yup... you have it figured out. Open source and going off-grid. I'm so done with chatgpt.. it was a fizzle.

[@issssse]

Please finish your Neural Network from Scratch series! 😢🙏

[@TomM-p3o]

Everybody wants to use local models, while API is much more convenient, flexible and cheap for many use cases.

[@Stinosko]

Hello!! 👋

[@sentdex]

How you been?

[@Stinosko]

@@sentdex Good! Very busy like almost everyone else i know 😅 How have you been, any crazy projects you are working on? 🙂

[@jeffmofo5013]

You are complaining about 1 hour to setup and test a model You know how long it takes to grow a plant? jk But I am curious as to your journey from AI to farmer, if you are serious about the farmer bit It's why I study AI, I'm fascinated by people's behaviors and why they do the things they do.

[@user-hg6nz1cz3l]

farmer life

[@Ryan-yj4sd]

Anyscale is way faster and cheaper. I just did a test. It's 30% faster and 15% cheaper.

[@arielweinberger]

I've compared Together and Anyscale against the same models - and while Together is cheaper and faster, the results are different. Together's results are MUCH worse, to the point of being unusable. Not sure what they do to the models.

[@brytonkalyi277]

_`∆_ I believe we are meant to be like Jesus in our hearts and not in our flesh. But be careful of AI, for it knows only things of the flesh which are our fleshly desires and cannot comprehend things of the spirit such as true love and eternal joy that comes from obeying God's Word. Man is a spirit and has a soul but lives in a body which is flesh. When you go to bed it is the flesh that sleeps, but your spirit never sleeps and that is why you have dreams, unless you have died in peace physically. More so, true love that endures and last is a thing of the heart. When I say 'heart', I mean 'spirit'. But fake love, pretentious love, love with expectations, love for classic reasons, love for material reasons and love for selfish reasons those are things of the flesh. In the beginning God said let us make man in our own image, according to our likeness. Take note, God is Spirit and God is Love. As Love He is the source of it. We also know that God is Omnipotent, for He creates out of nothing and He has no beginning and has no end. That means, our love is but a shadow of God's Love. True love looks around to see who is in need of your help, your smile, your possessions, your money, your strength, your quality time. Love forgives and forgets. Love wants for others what it wants for itself. However, true love works in conjunction with other spiritual forces such as patience and faith - in the finished work of our Lord and Savior, Jesus Christ, rather than in what man has done such as science, technology and organizations which won't last forever. To avoid sin and error which leads to the death of your body and your spirit-soul in hell fire (second death), you must make God's Word the standard for your life, not AI. If not, God will let you face AI on your own (with your own strength) and it will cast the truth down to the ground, it will be the cause of so much destruction like never seen before, it will deceive many and take many captive in order to enslave them into worshipping it and abiding in lawlessness. We can only destroy ourselves but with God all things are possible. God knows us better because He is our Creater and He knows our beginning and our end. The prove text can be found in the book of John 5:31-44, 2 Thessalonians 2:1-12, Daniel 2, Daniel 7-9, Revelation 13-15, Matthew 24-25 and Luke 21. *HOW TO MAKE GOD'S WORD THE STANDARD FOR YOUR LIFE?* You must read your Bible slowly, attentively and repeatedly, having this in mind that Christianity is not a religion but a Love relationship. It is measured by the love you have for God and the love you have for your neighbor. Matthew 5:13 says, "You are the salt of the earth; but if the salt loses its flavor, how shall it be seasoned? It is then good for nothing but to be thrown out and trampled underfoot by men." Our spirits can only be purified while in the body (while on earth) but after death anything unpurified (unclean) cannot enter Heaven Gates. Blessed are the pure in heart, for they shall see God (Matthew 5:8). No one in his right mind can risk or even bare to put anything rotten into his body nor put the rotten thing closer to the those which are not rotten. Sin makes the heart unclean but you can ask God to forgive you, to save your soul, to cleanse you of your sin, to purify your heart by the blood of His Son, our Lord and Savior, Jesus Christ which He shed here on earth - "But He was wounded for our transgressions, He was bruised for our iniquities; the chastisement for our peace was upon Him, and by His stripes we are healed", Isaiah 53:5. Meditation in the Word of God is a visit to God because God is in His Word. We know God through His Word because the Word He speaks represent His heart's desires. Meditation is a thing of the heart, not a thing of the mind. Thinking is lower level while meditation is upper level. You think of your problems, your troubles but inorder to meditate, you must let go of your own will, your own desires, your own ways and let the Word you read prevail over thinking process by thinking of it more and more, until the Word gets into your blood and gains supremacy over you. That is when meditation comes - naturally without forcing yourself, turning the Word over and over in your heart. You can be having a conversation with someone while meditating in your heart - saying 'Thank you, Jesus...' over and over in your heart. But it is hard to meditate when you haven't let go of offence and past hurts. Your pain of the past, leave it for God, don't worry yourself, Jesus is alive, you can face tomorrow, He understands what you are passing through today. Begin to meditate on this prayer day and night (in all that you do), "Lord take more of me and give me more of you. Give me more of your holiness, faithfulness, obedience, self-control, purity, humility, love, goodness, kindness, joy, patience, forgiveness, wisdom, understanding, calmness, perseverance... Make me a channel of shinning light where there is darkness, a channel of pardon where there is injury, a channel of love where there is hatred, a channel of humility where there is pride..." The Word of God becomes a part of us by meditation, not by saying words but spirit prayer (prayer from the heart). When the Word becomes a part of you, it will by its very nature influence your conduct and behavior. Your bad habits, you will no longer have the urge to do them. You will think differently, dream differently, act differently and talk differently - if something does not qualify for meditation, it does not qualify for conversation. *THE BATTLE BETWEEN LIGHT AND DARKNESS (GOOD AND EVIL)* Heaven is God's throne and the dwelling place for God's angels and the saints. Hell was meant for the devil (satan) and the fallen angels. Those who torture the souls in hell are demons (unclean spirits). Man's spirit is a free moral agent. You can either yield yourself to God or to the devil because God has given us discretion. If one thinks he possesses only his own spirit, he is lying to himself and he is already in the dark. God is light while the devil is darkness. Light (Holy Spirit) and darkness (evil spirit) cannot stay together in a man's body. God is Love (Love is light) and where there is no love is hell, just as where there is no light is darkness. The one you yield yourself to, you will get his reward. The reward of righteousness to man's spirit is life (abundant life) and the reward of sin to man's spirit is death. Sin and satan are one and the same. Whatever sin can cause, satan also can cause. Sin is what gives the devil dominion or power over man's spirit. When God's Word becomes a part of you, sin power over you is broken, you become the righteousness of God through Christ Jesus. Where Jesus is, you are and when He went (to the Father), you went. In the book of John 8:42-47, Jesus said to them, “If God were your Father, you would love Me, for I proceeded forth and came from God; nor have I come of Myself, but He sent Me. Why do you not understand My speech? Because you are not able to listen to My word. You are of your father the devil, and the desires of your father you want to do. He was a murderer from the beginning, and does not stand in the truth, because there is no truth in him. When he speaks a lie, he speaks from his own resources, for he is a liar and the father of it. Which of you convicts Me of sin? And if I tell the truth, why do you not believe Me? He who is of God hears God’s words; therefore you do not hear, because you are not of God.” May God bless His Word in the midst of our hearts. Glory and honour be to God our Father, our Lord and Savior Jesus Christ and our Helper the Holy Spirit. Watch and pray!... Thank you for your time and may God bless you as you share this message with others.

[@datarake7222]

Hail Satan

[@freecode.ai-]

If you're in south east Texas let's go fishing!