LoRA Land: How We Trained 25 Fine-Tuned Mistral-7b Models that Outperform GPT-4

[@JulianHarris]

Have you guys looked at the next generation of quantisation: eg ternary/1.58 bit quantisation? It’s a different technique to conventional quantisation because you have matrices that only have 0, 1, -1, and you eliminate matrix multiplication almost entirely. The intuition is that the combination may not bring quite as many benefits, but it might be interesting to see how it performs in CPU architectures for instance.

[@ofir952]

Thanks! How did you manage to remove the surrounding text of the LLM response?

[@pieromolino_pb]

It's a side effect of fine-tuning on output that contains only the JSON without tany other text

[@ofir952]

So, we cannot achieve this without fine-tuning? Llama2 keeps on adding it all the time 🥲@@pieromolino_pb

[@jeffg4686]

Nice !

[@tankieslayer6927]

FINE-TUNED MODEL RESPONSE Named Entity Recognition (CoNLL++) {"person": ["Such"], "organization": ["Yorkshire"], "location": [], "miscellaneous": []} Yeah, I am not impressed with the result of this fine-tuning.

[@pieromolino_pb]

The input text is: By the close Yorkshire had turned that into a 37-run advantage but off-spinner Such had scuttled their hopes , taking four for 24 in 48 balls and leaving them hanging on 119 for five and praying for rain. Yorkshire in this case is a sports team, so organization is correct, and Such is a a player, so both model's predictions are correct indeed. I'd suggest to try to understand better what is going on next time.

[@The_Real_Goodboy_Link]

Found the real solution, @tankieslayer6927, click on your icon on the top-right screen here, then settings, advanced settings, delete channel. Then go over to Google and do similarly for your account there. Problem solved!