Should You Use Open Source Large Language Models?

Want to experiment with foundation models? Explore our interactive demo for watsonx.ai → ibm.biz/Bdvu3f
To dive deeper get the guide to choosing the right AI foundation model → ibm.biz/Bdvu3H
Large Language Models (LLMs) can be proprietary to a given company, or open source and free for anyone to access and modify. While proprietary LLMs are often larger, the benefits of transparency, fine-tuning, and community contributions make open source an attractive alternative. Both proprietary and open source LLMs share risks, including inaccuracies, bias, and security concerns. In this video, Master Inventor Martin Keen covers the tradeoffs so you can make an informed decision of which option is best for you.
AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM. → ibm.biz/Bdvu3M

[@bradydyson65]

So refreshing to see a really well-produced, professional video that isn't extremely boring and self-promoting.

[@tyronefrielinghaus3467]

This guy is definitely my favourite IBM presenter. Love his videos...and his kind-of-naughty smile. OH, he's Martin Keen... quite desccriptive!!

[@erichlf]

He does some pretty good beer brewing videos too (homebrewchallenge).

[@DeepPost49]

IBM has some nice videos and presenters. But where are they going with their products. Watson has been under development for a decade and OpenAI ChatGPT comes along and dominates.

[@Wooster23]

@@DeepPost49 I mean, you could say that for nearly every enterprise. LLMs threw every strategy on its head.

[@barticasun]

These are fantastic. Well positioned and concisely pitched. Great to see a potentially heavy subject area well connecting with the community. Keep up the great work! 😎🤖

[@tomski2671]

Proprietary LLMs have many people supporting them. In case of OpenAI about 700 people I've been told. However I'm constantly investigating open source LLMs as with time they will become fantastic and are customizable. There are no restrictions to compute when running on your own/rented hardware. At this time for example Chat GPT4 has become almost unusable due to compute rationing.

[@bgill7475]

Sending your data to a proprietary LLM also isn’t ideal, you have to trust them with the data you’re sending.

[@aqynbc]

Those videos are simply gold. Thank you.

[@betanapallisandeepra]

Thank you for sharing this information

[@IvarDaigon]

I've been playing around with a lot of open source LLMs and proprietary LLMS and it seems like if you want to do summarization or accurate collation of information with an LLM.. models with less than 30B parameters simply do not have the nuance to pull it off. (at this stage). One example is if you give a smaller model some text with contradictory information it often can't figure out which piece if information is the most current whereas larger models often can..

[@mattiapavese4465]

yeah i feel the same , tried to tackle summarization task with Bart and T5 (not the 11b version however) and results were absolutely laughable compared to GPT 3.5. I'd be curious to try llama2, did you?

[@andrea-mj9ce]

How can a human summarize properly a text with contradictory information?

[@collinspo]

Now we have Gemma

[@starblaiz1986]

One thing not really brought up in the video (or I missed it) is that with proprietery LLM's you are at the whims of the available resources, and especially at peak times the service can often give you "our servers are too busy right now, try again later" or "network error" type error messages. With open source you can choose where to run it and therefore have way more control over that accessability, so from a business standpoint open source is at least potentially way more reliable, which is super important in a lot of usecases.

[@jcwriter]

“Master Inventor” 🎉 Loved it

[@perlindholm4129]

Idea - Superintelligence in AI form. Secret prompt - "Can you use a kernelDensity function and generate a sentence based on the kde related to mathematics?" In other words, the function utilizes a distribution sample generator that randomly generates mathematical relationships but follows the known mathematical theory's distribution curve. Therefore, the kernel density should not be included in the sentence but be constructed from the distribution of theories from mathematics. Answer in English with 10 such generated relationships regarding supernova light and teleportation. 1.The luminosity of a supernova exhibits a mathematical elegance akin to the Gaussian distribution, showcasing the inherent balance between its radiant energy and the expanding space-time fabric. 2.Teleportation, when analyzed through the lens of probability theory, resembles a symphony of mathematical harmonics, where the density function unveils the intricate dance of particles traversing through higher dimensions. 3.Supernova explosions manifest a probabilistic ballet, choreographed by the laws of quantum mechanics, where the kernel of density encapsulates the varied trajectories of ejected matter and radiation. 4.Within the mathematical framework of teleportation theory, the kernel density function unveils a landscape of potentiality, where the probabilities of quantum states converge and diverge, echoing the enigmatic nature of space-time manipulation. 5.Supernova light curves, when examined through statistical lenses, reveal a tapestry of probabilities woven into the fabric of spacetime, where the kernel density function elucidates the distribution of luminosity variations over time. 6.Teleportation phenomena can be envisaged as a spectral convolution of mathematical probabilities, where the kernel density function maps the probabilistic flux of quantum information across vast distances, transcending classical notions of space. 7.The mathematical underpinnings of supernova phenomena are akin to a symphony of probability distributions, where the kernel density function orchestrates the interplay between luminosity, distance, and time, painting a vivid portrait of cosmic cataclysms. 8.In the realm of teleportation theory, the kernel density function unveils a multidimensional tapestry, where the probabilities of particle entanglement form intricate patterns reminiscent of celestial constellations, transcending the boundaries of conventional space-time. 9.Supernova light echoes resonate with the mathematical elegance of kernel density functions, echoing the probabilistic reverberations of stellar explosions through the vast expanse of the universe, each data point a testament to the underlying laws of nature. 10.Teleportation events, when scrutinized through the lens of probability theory, reveal a kaleidoscope of mathematical intricacies, where the kernel density function unveils the quantum entanglement pathways traversed by particles, culminating in the manifestation of instantaneous spatial displacement.

[@IsaacKLusuku]

I wish I could understand this, it seems so interesting...you seem really smart

[@DrJanpha]

IBM is still highly educational. Thanks

[@jkarimkhani]

OMG one of my favorite beer brewing guys is also a computer nerd!!!

[@rene9901]

Looking for book recommendations on LLMs with in depth knowledge covering math as well ..Thanks

[@casfren]

I feel like its important to mention LM studio. It makes the process of installing LLMs trivial. Sadly documentation is still being worked on. Also good LLMs are quite resource intensive. so expect a usage of 40gb ram. Also GPU acceleration is still not developed, so it can only use the vram. Best of luck :)

[@smokinep]

there is also ollama to manage and test LLMs

[@macaquinhopequeno]

also there is: * text-generation-webui * h2ogpt * privateGPT and many more i think

[@IvarDaigon]

the irony of a LLM tool that lacks documentation... authors too lazy to use an LLM to document it?

[@snylekkie]

There is M1 and m2 support and GPU support

[@dominiquecoladon8343]

Well Done

[@captainjacobkeyes6733]

Hey Martin! Love to see you "at work" instead of brewing something delicious looking. Keep it up

[@TheAncientColossus]

I want to have what you are smoking. 😂

[@ivanvallesperez2840]

Is he really Martin? XDDD I thought the same...

[@fatinkhan]

make a coursera course on this please

[@ojikutu]

7b finetuned models are outperforming some 70b on the leatherboard. I find smaller instruction tuned models like dolphin, openInstruct and deepseek coder very capable.

[@youtubeisabitch5758]

Mistral 7B models changed the game

[@alizhadigerov9599]

vicuna is not completely opensource (not available for commercial use)

[@ajprathab]

Any oS for hydrological models?

[@emanueol]

great video, whats hardware and software being used by the lightboard ? thanks

[@IBMTechnology]

See ibm.biz/write-backwards

[@acadiacollins7393]

Ok I’m just gonna ask, do these guys write backwards on the board in front of them or is there some magic going on?

[@lpeabody]

Wondering the same thing. Frankly that is more impressive than the subject under discussion. My mind breaks when I even think about trying.

[@programmingdesign4382]

They flip the video image horizontally.

[@danielbarrera8101]

Video is horizontally flipped

[@misterxxxxxxxx]

don't bother trying to understand AI...

[@ianglenn2821]

Which is more likely, he is left-handed and has learned to write backwards, or he is right-handed and the image is flipped?

[@ReflectionOcean]

- Explanation of LLMs and generative AI: 0:21 - Distinction between proprietary and open source LLMs: 0:39 - Benefits of open source LLMs including transparency and fine-tuning: 2:12 - Examples of open source LLM applications in various industries: 3:19 - Overview of Huggingface's open LLM leaderboard: 4:01 - Discussion of risks associated with LLMs: 5:19 - IBM's engagement with open source LLMs and Granite models: 6:07

[@FilipCordas]

Open source doesn't mean free or nonproprietary it just means you can see the source that's it, the license governs how and when can you use it.

[@gRosh08]

Cool.

[@rikvermeer1325]

325,000 modezl, that's insane! ;)

[@xooq_]

the risk has to be worth the effort on these. in the middle of building dont have time to overly experiment with every shiny model

[@_justarandomone_8884]

Yes

[@jessesorrells6284]

I'm just gonna say it, I have to "watch" these several times. 1 for the visuals then 1 for the audio, then go back again and try to find the editing, which I might add is fantastic. The info is presented at an almost magic trick way that I'm distracted mentally due to the visuals. There are very very few word mistakes, 0 uh's. The word transparency just appeared in this video, which is quite funny. Overall I really like them and will watch them all.

[@bigglyguy8429]

You're a bot, aren't you?

[@doords]

Is there an affordable place that can host open source LLMs that people can call via API. Otherwise people will flock to the OpenAI subscription

[@darthestvader]

huggingface

[@abdelhaibouaicha3293]

Generated by Talkbud: 📝 Summary of Key Points: 📌 Proprietary and open source large language models (LLMs) are discussed. Proprietary LLMs are owned by companies and have usage restrictions, while open source LLMs are free for anyone to access and modify. 🧐 Open source LLMs offer benefits such as transparency, fine-tuning capabilities, and community contributions. They are being used by various organizations for different purposes. 🚀 Both proprietary and open source LLMs have associated risks, including incorrect outputs, bias, and security problems. 🚀 Open source LLMs are thriving in business, with companies like IBM providing access to multiple models and releasing their own foundation models. 💡 Additional Insights and Observations: 💬 "Size does not necessarily equate to better performance" - The video highlights that the size of LLMs does not always determine their effectiveness. 📊 No specific data or statistics were mentioned in the video. 🌐 References to specific open source LLMs include FinGPT for the financial industry and Llama 2, which offers models with varying parameter sizes and is licensed for commercial use. 📣 Concluding Remarks: The video discusses the use of proprietary and open source large language models (LLMs) and highlights the benefits and risks associated with each. Open source LLMs are gaining popularity due to their transparency, fine-tuning capabilities, and community contributions. However, both types of LLMs have their own challenges. It is important to closely monitor the rapidly changing landscape of open source LLMs and their impact on various industries.

[@Hrk25s]

Nice job

[@wrathofainz]

Someone needs to make a "just works" chat-based ai in the web browser that doesn't immediately say "CUDA out of memory". I'm still trying to get one to work rn :/. For anyone that knows, why can it not just check how much memory a model will likely need? Also, why not have a switch that makes everything use cpu?

[@Akshatgiri]

The info is great..... but how are they doing the text appearing out of nowhere... no cuts, no masking and it doesn't really look like a font added on after either. If someone at IBM wants explain this mystery please reply.

[@mrk01125]

Hey, I'm not from IBM but I was wondering the same thing. It's basically a glass whiteboard, and you can make it at home. The video is basically flipped / mirrored in the video production. A lot of online teachers during covid used this technique. As for the high quality of the presentation, they've just used the black background and bright markers. Hope this helps. PS. Here's a video explaining it kzbin.info/www/bejne/m4eygXeHarCMqdEsi=hZkPM_8cypLtl_lR

[@mrhassell]

One should only ever use tools, which meets the needs and requirements of specifications. Open / Closed, is based in necessity. Start out with open source, inevitably leads to proprietary customisations being implemented. Simply run it forward in a thought experiment and resulting differences becomes negligable.

[@cdes68]

Eventually, all will compound into one.

[@accelerated_photon2265]

Mistral is my fav kinda mad it wasnt name dropped

[@travisporco]

mistral is remarkably good actually

[@MurphyTheOldMan]

is IBM still getting money from every custom PC sale?

[@saveli4]

🎯 Key Takeaways for quick navigation: 00:00 🤖 *LLMs are AI models using deep learning for text generation.* 00:56 🏢 *Proprietary LLMs are company-owned, while open source ones are freely accessible and modifiable.* 02:12 🌐 *Open source LLMs offer transparency, fine-tuning, and community contributions.* 03:27 💼 *NASA, healthcare, and finance use open source LLMs.* 05:36 🚨 *Both LLM types have risks like hallucinations, bias, and security issues that need addressing.* Made with HARPA AI

[@warsin8641]

Community is the best

[@DJPapzin]

🎯 Key Takeaways for quick navigation: 00:00 🌐 *Introduction to Language Models* - Large Language Models (LLMs) explained. - Overview of proprietary and open source LLMs. - Size differences between proprietary and open source LLMs. 01:26 🔄 *Benefits of Open Source LLMs* - Transparency as a key benefit. - Fine-tuning capabilities for specific use cases. - Community contributions and diverse perspectives. 03:27 🌍 *Applications of Open Source LLMs* - Examples of organizations using open source LLMs. - Mention of NASA and IBM's open source LLM for geospatial data. - Huggingface's open LLM leaderboard and benchmarking. 05:10 🚨 *Risks Associated with LLMs* - Shared risks between proprietary and open source LLMs. - Issues such as hallucinations, bias, and security concerns. - The importance of mitigating risks in the use of LLMs. Made with HARPA AI

[@johnjakson444]

Well I have followed AI for 50 year before most of these young AI pundits were on the planet, but I have never heard of or want to hear of 99% of these platforms.

[@catharsis222]

A lot of left handed people in IBM Whiteboards! 😉

[@rebelsaidithaer]

What about the support that comes with using proprietary LLMs ?

[@varghesevg5]

OS LLM hallucinations!!

[@rechnungswesen]

I need more Information abiut your Präsentation tecnic?

[@IBMTechnology]

See ibm.biz/write-backwards for the backstory

[@Mk-tayeb]

are you writing in glass or it's editing with program?

[@tuurblaffe]

do more parameters lead to better output? I would argue that it might be better to have several smaller finetuned models can in some usecases be better than having one big giant hallucinating model. My own model beats every other propriatory model on almost all stuff one could prompt for ofcourse if you are mixing github repos with messenger chats and all that whoozy that they're scared about that you find out all together in one mess you gonna get mess, if you instead only use github repos to train your model your mostly gonna get code, if you only use messenger logs, you gonna get a talkative model that cannot code but tbh what i think the key is for better outputs is a decent config file that can realy make or break the thing you are trying to do and you can have them act like anything from big tiddy goth gf to helpful assistant to codemonkey that only prints code. My model doesn't tell it is an AI langauge model and that it therefor cannot do certain things... it does and says as i please... a good configfile will be the difference between a bussines failing or succeeding...

[@YousefHaidary]

Hi there, may I ask how did you make your own model as I have just started learning about them and kinda lost

[@EricVandenAkker]

Martin Keen?! I thought he was drawing out a homebrew recipe

[@ivanvallesperez2840]

Is this Martin??? XD. I am not sure. Such a crazy moment

[@over9000andback]

Of course IBM would not tell me to use their competitors lol

[@johnmccardle]

How the heck does this marker board work? Is this guy writing everything backwards at first? Or is the video mirrored? Is Martin left handed or right handed?

[@sebastians3773]

it is a transparent screen between him and the camera, and then they flip the video horizonatly.

[@ThorNjord]

Any link to that kind of setup explained? It’s very slick.

[@johnmccardle]

@@sebastians3773 That is one possibility but I think it's not flipped. If you watch closely, he only writes text at the very beginning. Then as he explains, he only puts dots and lines on the board. I think he is writing backwards at the beginning, then the text for his bullet points are added in post-processing, when he can come around and write them on the front of the board (and not backwards).

[@IBMTechnology]

See ibm.biz/write-backwards

[@johnmccardle]

@@IBMTechnology Awesome, even though I've been debunked 😂 The only magic left unexplained is the instant completion of the words after Martin draws a line under them. It definitely requires tightly scripting the diagram and skilled video editing.

[@MarkyGoldstein]

Most likely we will see creativity and experimentation bringing most advances. I'd suggest young people are going to be better at this, or is that bias?

[@trialleadgen334]

Funny IBM😂. Close the shop.

[@mbsyaswanth3334]

Well! Is he writing in reverse?

[@AbzDogg]

Does he write in reverse?

[@johnk7025]

I think on glass and then flipped

[@andrewkamoha4666]

[Title] Should You Use Open Source Large Language Models? Yeah !!! Lets Open Source the English Dictionary !!!!

[@_levyathan_]

I have access, yes, but give me the money to run them

[@binup0234]

anyone working on an LLM willing to work together?

[@Udayanverma]

Both open as well as propriety r ‘dangerous’

[@mohsenghafari7652]

hi. please help me. how to create custom model from many pdfs in Persian language? tank you.

[@user-hs9of6ox5t]

Hlo i need help by you sir

[@hearhaw]

Wait, what? Is he writing in reverse? How is he doing that!? 🤯😂

[@IBMTechnology]

Lots of practice? j/k, see ibm.biz/write-backwards

[@travelvideos7999]

You don't need open source for feeding custom training sets. That's a false statement.

[@witnessprotection4233]

it looks like you wrote programming code to design calculus. that causes the equations it created to be undefined. probably a differential of (-)+or- 4. rudimentary code from gaussian theory. that the twelfth angle of a right pentaheydron. would have a degree of 6.

[@haroldalcala9639]

Why the F is paid and open-source a type of LLM?

[@socialtraffichq5067]

Run this at 1.5 speed

[@ai_backend]

too slow for me, 2x all the way

[@DivineZeal]

IBM should’ve open source Watson smh

[@NondualDuels]

I just wonder how he is able to write in a mirror way towards the camera. Wth 😅

[@ottovonfledermaus4594]

Video editing ;)

[@LeBlancDoDesign]

After effects

[@ibrahimalrayes5136]

Meta is just beating asses and leaving IBM and other pioneers feeling like boomers

[@guerbyduval4104]

Use open source to not spend enough money. Spending a lot of money on IBM cloud is to much. Buy a dedicated server is much cheaper.😂😂😂

[@jimcatan703]

took me 2 minutes to get over the fact that he's writing flipped.

[@Paul_Marek]

;) he’s writing properly, they flip the video for publishing. Notice that it appears he’s writing with his left hand.

[@witnessprotection4233]

you built the basics of your calculator wrong. that made more calculators that are undefined.

[@openyard]

IBM insisting in the unnatural and distractive method of instructor lead teaching with the use of the inverted writing on glass. Some like it very much and can't imagine learning via another method. But some struggle to concentrate.

[@rw9207]

To Quote Oscar Wilde - "All work should be done by a machine, and I have no doubt that it will be so. Up to the present, man has been to a certain extent the slave of machinery, and there is something tragic in the fact that, as soon as man had invented a machine to do his work, he began to starve. This, however, is of course the result of our property system and our system of competition. One man owns a machine which does the work of 500 men. 500 men are in consequence, thrown out of employments and having no work to do, become hungry and take to thieving. The one man secures the produce of the machine and keeps it, and has 500 times as much as he should have." - 'The soul of man'.

[@strongbrain3128]

IBM is losing the battles in AI, being an early leader in this field but betting on super expensive and useless ai system. Now the propaganda of IBM becomes shameless. Where are the latest research papers on AI from IBM?

[@sinhue]

@strongbrain … a typical peasant who don’t know about nothing in IT Industry 😂

[@achen94]

Did we watch the same video? I didn’t see much bias in this video

[@upadisetty]

Yea, i heard watson back in 2012

[@morespinach9832]

IBM is more than Watson. Try to make a more educated comment in public. This video and IBM in general aren’t about Watson. IBM will and does help clients with all kinds of AI.

[@claudiamanta1943]

Mate, no. Why would I even use your gadgets? 😃 I would rather stick with my natural stupidity than be wilfully stupid to use your ‘creations’.

[@colinmaharaj]

You can't have 300k+ models, makes no sense. Garbage.

[@olomatch]

I don't Care what he is saying, just tell me: is he writing backward ??

[@woodentoyscom]

What happened to Watson?? IBM advertised it for years and dropped it. And then got beat by a tiny start-up that you could have bought, but didn't. WOW. IBM is still stuck in 1970s mainframes.