Lessons From Fine-Tuning Llama-2

[@DizroAI]

I already have a small database, and I'm attempting to develop a process for handling large news texts. What model is best suited for this scenario, and how should it be properly configured? The model will be fed extensive news content as input, with the goal of obtaining a formatted and condensed version of the text as output.

[@moonly3781]

I'm interested in fine-tuning a Large Language Model to specialize in specific knowledge, for example fish species, such as which fish can be found in certain seas or which are prohibited from fishing. Could you guide me on how to prepare a dataset as an example for this purpose? Should I structure it as simple input-output pairs (e.g., 'What fish are in the Mediterranean Sea?' -> 'XX fish can be found in the Mediterranean Sea'), or is it better to create a more complex dataset with multiple columns containing various details about each fish species? Any advice on dataset preparation for fine-tuning an LLM in this context would be greatly appreciated. Thanks in advance!"

[@nullrox]

The direction of LLMs is eerily similar to how human specialization works. We have doctors, entomologists, biologists, etc, and all of them are "fine tuned" in their respective fields. I'd imagine that eventually we'll have a network of interconnected LLMs all specializing in specific fields that talk to each other. So when you ask chatgpt about a fish species or something it can reach out to FishGPT or something who's an expert in fish species and then relay the information back to you.

[@Jappie1999]

Look into Retrieval Augmented Generation (RAG).