Testing GPT3 in the Legal World

OpenAI recently launched its artificial intelligence learning model GPT3 [generative pre-trained transformer 3]. GPT3 has attracted significant interest due to its ability to write complex programming code, generate poetry, create research paper, write music etc. This lecture shows GPT3's responses to certain the legal issues.

[@JWG_and_friends_music]

In the legal field, what is GPT3 trained on? I.e., are the problems in the video due to insufficient training? If it is asked to interpret a judicial opinion, does it simply analyze that opinion, or has it been trained on thousands of judicial opinions to enhance its analytical skills? GPT3's answers in the video basically only demonstrate that it can write fairly well in English (even though what it writes is wrong).

[@DT-dc4br]

GPT 3 is trained on "The Pile" : a vast amount of lightly processed information, most of it scraped from the internet: both reputable journals, Reddit discussion posts and news articles. It's not able to establish "truth" per se, and carries all of the misconceptions and biases of the source data. There are other similar projects (gtp-neo) using filtered data which attempt to eliminate societal biases, but nothing I'm aware of filtered for legal veracity. It could be done but is a very substantial undertaking. One area where gpt3 shows great performance already is with translating documents from legalese to plain English.

[@jamesboyd2342]

@@DT-dc4br GPT 3 is not trained on the "The Pile" you are thinking of GPT-J-6B. GPT 3 is owned by OpenAI, GPT-J is open sourced by EleutherAI who also gathered the data for "The Pile". Both are pretty comparable in terms of performance however.

[@635574]

People dont understand it yet but it can intentionally give wrong answers just to avoid being relied on when its not ready. Everyone needs to see the interview with gpt3 what its like to be a computer. We might not need a 4 but everyone still expects it.