In my organisation we have GPT 4 model which is trained on custom dataset and currently we are validating using BERT model, which validates expected output and actual output from AI bot, if cosine similarity scores is greater than 0.5 then we are passing the TC
@naveenautomationlabs Жыл бұрын
Good one Atul. Thanks for sharing it.
@aravindmak Жыл бұрын
@atul bharadwaj - can you please explain it a little further ? What will be your validation strategy ? Is there a Human in Loop revie for the output that comes out of ChatGPT model that validates it's correctness ?
@athiramurali7599 ай бұрын
Hi,i need some help for my upcoming project regarding this. How can I contact you
@bhhada Жыл бұрын
I am using Botium for testing the Chatbots.
@bowthunell6 ай бұрын
Hi, Did you use Botium with Chat -Gpt? or which LLMs? and how was your experience with Botium? I'm considering using it with my Chatbots project.
@Bbangaram-d9u6 ай бұрын
Could you please give more info on this please?
@sakinathmubarak4267 ай бұрын
Hi Naveen, Can we automate the Chatbot UI?..If that is possible Can you explain and make a video on that UI testing of chatbots like speech input,text input,selection inputs and how to validate the response like comparing with actual response. If there is a video already on this topic pls give me the link
@puneetaneja62963 ай бұрын
Good Content, thanks Naveen.
@i4iirfan Жыл бұрын
Please make a video on how to automate ai based chat bot and which tools to use
@parveenchauhan5132 Жыл бұрын
Hi Naveen, It was insightful. Thanks
@shankarganesh1858 Жыл бұрын
Hi Naveen, Thanks for the above video, Healenium is a open source AI powered self healing Technology available now, if you could make a video on the same it would be helpful. It supports both Selenium and Appium. if you could explain the concepts of installation and configuration by your style, it would help users to understand better and use it in the project.currently i struggling to implement in my project. Thanks in advance.
@STELLADCSR18 күн бұрын
In our organization we validate the chatbot by first creating a folder that contains all the files of our documentation. We pregenerate questions for each page and save it in a json file. we then randomly select a question and ask the chatbot. Multiple questions can be asked using multiprocessing(completes load testing). We then send the response, the question, and the respective contents of the file to LLM and ask LLM to validate its accuracy on a scale of 1-10. Is there any other techniques where I can validate chatbots that has only knowledgebase and QnA knowledge fed into it? Cause I dont have any files to validate them.
@STELLADCSR18 күн бұрын
any tool to help me in this?? or any other algorithm?
@prasanthss420 Жыл бұрын
Good
@sarahgao8300 Жыл бұрын
My data science friend told me they don’t have QA position in her machine leaning project.
@user-xb7bc5eo1w Жыл бұрын
Hi Naveen, Thanks a lot for the information. Suggest me any testing tools available to test an Avatar based AI application
@pabitra_swain Жыл бұрын
Hi Navin, do we have an option to Run Playwright on GCP? Please make a video on it or please reply if you have done your poc on the same. That will really help. Thanks in advance!
@KrishKoli-qw9vb10 ай бұрын
Can you provide happy path and negative path test scenarios?
@chandrasekhar-sy6uj10 ай бұрын
Hi Naveen I have no idea on what certifications need to be done for AI automation testing, can you help me with this
@naveenautomationlabs10 ай бұрын
As such no certification is available for AI in testing
@atulbharadwaj774 Жыл бұрын
Can you suggest some python based libraries to test these bots
@naveenautomationlabs Жыл бұрын
I heard from one of friends, working as a data scientist suggested to use nltk: The Natural Language Toolkit (NLTK) is a widely-used library for NLP tasks. It provides tools for tokenization, stemming, part-of-speech tagging, syntactic parsing, and more. It can be used for preprocessing text and verifying the correctness of NLP operations.
@atulbharadwaj774 Жыл бұрын
Thanks, will check that
@kshitijvairale4877 Жыл бұрын
Is there any tool for performance testing chat bots lets say I want to run a test for 1000 users accessing a bot at more or less same time
@naveenautomationlabs Жыл бұрын
Hi Kshitij, Use load testing tools to simulate high user traffic and measure how the chatbot performs under heavy loads. Monitor response times, resource utilization, and system stability during peak usage scenarios. Tools like Apache JMeter or LoadRunner or Gatling or K6 can help in load testing. Performance Monitoring: To collect data on response times, throughput, error rates, and other performance metrics. Monitor the chatbot in production or a staging environment to identify bottlenecks, performance degradation, or potential scalability issues. Tools like New Relic, Datadog, or custom monitoring solutions can assist in performance monitoring. Benchmarking is important: Compare the chatbot's performance against industry standards or competitors' chatbots. Benchmarking allows you to assess the chatbot's performance in relation to similar systems and identify areas for improvement or differentiation.
@kshitijvairale4877 Жыл бұрын
@@naveenautomationlabsThank you sir for your response But the chat bots which are build in my organization is based on web socket and on client server manner we have to launch url in browser to build a connection In this scenario do u know any tool...
@naveenautomationlabs Жыл бұрын
@@kshitijvairale4877 Tools like Locust, Gatling, Jmeter (As I suggested above) supports WebSocket connections, send messages, and measure the performance and scalability of your chatbot.
@Dipsravi6 ай бұрын
Hi kshitij, Have you done performance testing for above usecase? looking for same use case so i need small help