Demo and Code Review for Text-To-SQL with Open-WebUI

Рет қаралды 5,495

Jordan Nanos

Күн бұрын

Пікірлер

@Alex-os5co 3 ай бұрын

What an awesome introduction to pipelines - thank you so much!

@bittuk575 2 ай бұрын

I am really curious to know how to integrate this particular pipeline text_to_sql_pipeline in Open web UI and enable it? I successfully verified the Pipeline connection but in 'Pipeline' section, when I upload this text_to_sql_pipeline, it shows me No Valves. Could you please explain me how you enabled it in your Open Web UI? I am using the docker compose containing Ollama, Open Web UI and Searxng service. I run the Pipeline Container separately. Please guide me

@georgetorres1535 11 күн бұрын

It seems great, have you managed to do something like this but with a vector database and embeddings? The only way that exists as far as I know is through n8n

@jordannanos 11 күн бұрын

@@georgetorres1535 pgvector works for vector search, and vector DBs would work for RAG. Primarily RAG is for unstructured data.

@bosmach 21 күн бұрын

Hi Jordan, thanks for great video. My question is about the accuracy, did you measure the percentage of correct answers? The main problem with text-to-SQL is the accuracy, as even 85% is not enough for business, what do you think?

@jordannanos 21 күн бұрын

@@bosmach depends on the model. Some are better than other, and you can improve the prompt or the model to improve performance.

@jordannanos 21 күн бұрын

For now I just have the pipeline display the SQL it ran, and another pipeline that lets you submit SQL directly. So if you notice a mistake it’s easy to edit it.

@bosmach 20 күн бұрын

@@jordannanos thanks for answer. Your video is very helpful. My point is that I cannot understand whether it is already possible to use text-to-sql for real business tasks with performance close to 99.99% or it is the matter of an additional year or two so that technologies will improve more to give such performance.

@geovannywing1648 3 ай бұрын

i installed open we ui local in a docker container but when i access don't see the option to upload the pipelines files, there is a special config you running?

@jordannanos 3 ай бұрын

@@geovannywing1648 if you’re using docker you’ll also need to run a separate “pipelines” container from the open-webui project, make sure networking is setup correctly between the containers, and then a connection is established between the two containers.

@jordannanos 3 ай бұрын

@@geovannywing1648 docs here: docs.openwebui.com/pipelines/

@mohamedkhalifa-p4k Ай бұрын

thank u sur can you make the steps of setup from scratch as there are some problems to run the project

@Earthvssuna 3 ай бұрын

thanks so much, i will try all of it! but first Im curious how you settuo vllm for openwebui instead of ollama...do you have any good installation docu source for that?

@Earthvssuna 3 ай бұрын

so maybe the whole setup if relevant... like is ollama on a linux server? is openwebui on windows or on the same linux in a container etc..?

@jordannanos 3 ай бұрын

@@Earthvssuna you can run vLLM as a docker container or k8s deployment. For docker use this doc: docs.vllm.ai/en/latest/serving/deploying_with_docker.html Once the model (typically one from huggingface, like mistral or llama) is running, it’s an OpenAI-compliant endpoint. You can use the OpenAI python client for custom apps, or just add it as an endpoint in open-webui in the admin settings page. If you’re interested I was considering making some more videos describing how to install docker, kubernetes etc on a GPU server?

@Earthvssuna Ай бұрын

Yes definitely interested into this and as far as iv been watching theres interest of others as well. I saw also your other video where you put ollama and vllm on the same gpu server. Interested also on this so one for testing out llm models and the other for production. Let me know if I can help you gather some more questions that a newbie might have in this field :)

@jordannanos Ай бұрын

@@Earthvssuna sure sounds good. I'll work on some videos for installing the different components of that setup with docker + kubernetes.

@azmat8250 3 ай бұрын

Great review, Jordan! Quick question. I have a pipeline that calls Replicate to generate an image based off the user_message (prompt) fed in from open-webui. However, when i get the response from Replicate, I'm having some issues displaying the response back in open-webui. Do you know if the return type of the pipe function has to be a string in order for open-webui to render text? What is open-webui's interface expectation on the return from the pipe function?

@jordannanos 3 ай бұрын

@@azmat8250 I’ve only seen a string work when returning from a pipeline. Even a list throws an error for me. However it’s all open source… if you look at the component during a web search, or with an audio input, it seems like you could create something custom.

@jordannanos 3 ай бұрын

@@azmat8250 looked into this and v0.3.30 of open-webui has experimental support for image generation via OpenAI’s api and a few others. It’s not via a pipeline, but still may be worth upgrading and checking out if you haven’t seen it yet.

@azmat8250 3 ай бұрын

thanks, @jordannanos . I'm on .30 but it seems like it's not working...at least for me. I'm still toying around with it. If I find something, I'll share here.

@jim02377 3 ай бұрын

This was a great intro, I think the information about Open webui pipes on the site is a bit vague. I would love to see more about how to use pipes for things like filtering user inputs or outputs, if pipes are the appropriate thing to use for that kind of thing. I work for a school district and would like to be able to do that to allow students access to local models.

@jordannanos 3 ай бұрын

@@jim02377 I haven’t played with filters, but that concept does exit as a type of pipeline: github.com/open-webui/pipelines/blob/main/examples/filters/detoxify_filter_pipeline.py

@KeesFluitman 2 ай бұрын

so I'm a little bit new to this area. You've add the xml file to the database yourself, and are just querying it via the pipeline? I was thinking about whether this would be a viable solution for an app that would allow people to easily find information that is stored in a database. But I guess that would be a huge security risk, since you basically allow them direct database access.

@jordannanos 2 ай бұрын

@@KeesFluitman no xml file, it’s a csv but yes it’s direct access to the database. In a real environment this would run against a data warehouse, lake, or some backup/export of the database

@jordannanos 2 ай бұрын

@@KeesFluitman there has to be a way in which people can find information stored in a database today. And generally they write SQL to find that information, or have to ask an analyst to write the SQL for them and send them the results. This pipeline just simplifies that process a little bit.

@gilkovary2753 3 ай бұрын

Hi, how do I execute the python lib installation on the pipeline server?

@jordannanos 3 ай бұрын

@@gilkovary2753 you’ll need to docker exec or kubectl exec into the container called “pipelines” Then run: pip install llama-cloud==0.0.13 llama-index==0.10.65 llama-index-agent-openai==0.2.9 \ llama-index-cli==0.1.13 llama-index-core==0.10.66 llama-index-embeddings-openai==0.1.11 \ llama-index-indices-managed-llama-cloud==0.2.7 llama-index-legacy==0.9.48.post2 \ llama-index-llms-ollama==0.2.2 llama-index-llms-openai==0.1.29 \ llama-index-llms-openai-like==0.1.3 llama-index-multi-modal-llms-openai==0.1.9 \ llama-index-program-openai==0.1.7 llama-index-question-gen-openai==0.1.3 \ llama-index-readers-file==0.1.33 llama-index-readers-llama-parse==0.1.6 \ llama-parse==0.4.9 nltk==3.8.1

@netixc9733 4 ай бұрын

Thanks for sharing this awesome project! I tried running the 01_text_to_sql_pipeline_vLLM_llama.py file from your GitHub repo, but I'm having trouble uploading it on Open WebUI even though I've installed all the requirements. Do you have any idea what might be causing this issue? Thanks again!

@dj_hexa_official 4 ай бұрын

Did you configure well pipeline ?

@netixc9733 4 ай бұрын

@@dj_hexa_official what do u mean with that ?

@jordannanos 4 ай бұрын

@@netixc9733 what error are you seeing? docker logs -f or kubectl logs -f your pipelines container and it may report an error

@firstland_fr 3 ай бұрын

You think we can use custom model with api for rag ?

@jordannanos 3 ай бұрын

@@firstland_fr yes I don’t see why not. Ollama will work with any model in GGUF format (llama.cpp). And vLLM works with just about any transformers model from huggingface: docs.vllm.ai/en/latest/models/adding_model.html Both ollama and vLLM are tested with this pipeline