Claude-2 meets LangChain!

[@channel_panel193]

it's really hard to get claude to return objects. It just wants to print.

[@klammer75]

I’ve been waiting for almost 3 months now for anthropic API access…bookmarking this for when I do🥹🙏🏼🦾

[@liy0na]

Hey Sam. Thanks for the video. I couldn't find this in your github repository. Do you have a link for it?

[@samwitteveenai]

Is the note in the description? I have had YT removing some of them for some strange reason. I will look into it. Could be I missed upping to Github.

[@heyheni]

Hey sam. I'm into bicycle touring. And claude 2 and GPT4 32k does a good job planing routes adding campsites, tourist attractions and restaurants. However it refuses to export it to a GPX xml navigation file for a bicycle computer in full. It only does 3 examples in xml. Even when i ammend the required data step by step like gps coordinates and elevation. Could i circumvent that using langchain and a vector database like in your video? Example route, Zurich, Chur, Bozen, Lienz, Ljubljana. 21 Days. Thank you for your very educating videos.

[@barefeg]

Could you clarify what you mentioned about anthropic last video about allowing on prem deployment?

[@samwitteveenai]

best to reach out to their sales people for this.

[@julian-fricker]

I'll have to try to follow your method. I tried a basic prompt with an embedding search and it wouldn't work, kept giving me the default response I taught it when it couldn't answer a question. But if I used the LangSmith trace tool I could run it through their playground and it worked really well.

[@clray123]

As to the 100K token limit, you must also consider that the attention algorithm essentially averages values of the tokens it attends to (and normally these are all tokens to some extent). I have great doubt that as you increase the number of tokens you average you will get any sensible information out of them. The whole point of attention is to be selective wrt to evaluated tokens, but the more tokens there are, the less selective it is going to become.

[@Dani-rt9gk]

So you’re saying the 100K system will require more algorithmic complexity? Or is entirely pointless?

[@barrettjacobsen6614]

Then do some tests and assuage (or confirm) your doubt. Feed it a chapter of something as well as in different instances the full book and varying levels of synopsis and have it explain how the chapter relates to the whole. Report back with your results.

[@samwitteveenai]

I agree that the 100k contexts seem to lose information in the middle. I totally disagree with the people saying you don't need vectorstores etc, personally I see the best way to use this is just giving it more semantic results returned etc. The other thing is I am not sure if they have actually fully said how they are doing the 100k context. There are a number of rival ways to do it and more coming out each day. Also getting people to use 100k each time when they don't need to seems a great way to increase business for these LLM startups. While there are some, not many uses cases can justify spending a few dollars each time you ping the API.

[@clray123]

@@Dani-rt9gk ""Algorithmic complexity" is a property of an algorithm, not of the data size, so it is constant. Depending on the attention algorithm you use this complexity is quadratic (if you want to process each token in context, as in the original transformer) or linear (if you cut corners by processing fewer of them, e.g. linformer or the newest "dilated attention"). So actually you may have a huge context on paper together with an algorithm which does not really use much of it to make it fast enough. I suspect there are tradeoffs to be made, but my intuition is that making contexts superlong will not really make the systems much better. I consider context length similar to human short term memory: you can do a great deal of work with not so much memory if the interface to the longer term memory makes sense. But currently this interface doesn't really exist in LLMs; there are vector dbs and there are tokens in context, but these do not really interact or exchange information of their own accord, it is left to application to do it "somehow" (probably poorly).

[@clray123]

@@barrettjacobsen6614 I don't even have access to the API at this point.

[@nicewook]

Thanks for the video. :-)

[@GeoffY2020]

Dear Sam, thanks for your wonderful efforts again. In this Claude 2 demo, are the text file structured or unstructured (i need to do structured file) ? Greeting from Hong Kong after Typhon Talim just passed !

[@ZhengChen-w7t]

八号风球🤣

[@KararaJawaab]

Any Plan For Videos Explaining Self-attention Or Diluted Attention Mechanism In Simple Terms?

[@samwitteveenai]

I would have thought there are a 100+ vids on Self attention out there now. I am thinking of making more DL coding vids etc if there is the appetite for it.

[@nandakishorejoshi3487]

Did you try generating python or SQL codes using this model?

[@micbab-vg2mu]

Thank you for another great video, Sam. I am still waiting for the API. I will use my company email. I work for a large pharmaceutical company with over 100,000 employees. Maybe this information will help to get it.

[@AlonsoGHS]

I just got the API Key today 😊

[@micbab-vg2mu]

Me too - to late - we have Mistral now:)@@AlonsoGHS

[@AIWRLDOFFICIAL]

Nice vid!

[@jgfitzpatrick]

I gave up on ever getting access to anthropoic api lol

[@Peter-oz1oo]

LangChain is a pain in the ass

[@FarhadKumer]

langchain is amazing

[@clray123]

Hallucinations are actually the elephant in the room for all language models. The point is that making up ANY stuff destroys trust, and the effect is magnified when the party who does it is otherwise competent. Even if the bot doesn't hallucinate 99% of the time, that one time it does will feel like a betrayal to the human it communicates with. It is equivalent to having an intelligent person who you previously found reliable starting to make up stuff or lie to you one day. In human relations it seldom works out as "oh well, (s)he must have had a bad day". We may have this leniency toward kids or imbeciles, but not toward responsible adults. Instead it acts like poison and creates suspicion about all the future interactions. And we humans especially do not like to deal with other humans who are repeatedly unreliable. So there you have it, the rather bleak future of language model AI (unless the "I don't know" response is sorted out to work correctly 100% of the time).

[@nickmillerable]

Nicely put. Will you be sitting this one out then?

[@henkhbit5748]

Very informative, by interfacing claude2 using code. 👋 Can we also count the tokens used like in openai? Testing how the system role compared with openai would be nice...