Reinforcement Fine-Tuning-12 Days of OpenAI: Day 2

[@lavaboots1337]

Millions of dollars for staff. Billions for Invidia chips. Sound recorded with a potato - priceless.

[@staffanlundberg]

Sorry, I guess You´re right: I was too busy following their storyline to notice. But I WILL make an effort to check this on their next video. 😏

[@fegyenc]

@staffanlundberg please make it better next time the sound and lights are so amateur. It is not your level...

[@jadioj]

Lmfao

[@Cyber_Jan78]

This is good, their focus is the product not bla bla marketing

[@Cyber_Jan78]

But i like you potato 🥔 referral, even o1 wouldn’t come on that … or would he …🧐

[@HelamanGile]

Do you guys need to hire a sound guy if so I am available have lots of experience

[@upgradeu2]

30 sec pitch go

[@sammy45654565]

@@upgradeu2 they need a better sound guy to hear it

[@Lighto222]

Hello fellow sound guy

[@A123-d8o]

make sure to charge them $200 a month and watch them cancel roflmao

[@WayOfTheZombie]

At least just boost about 2 to 4 db

[@theoz1]

Recap → Reinforcement Fine-Tuning (RFT) preview announced, allowing fine-tuning of o1 models on custom datasets using reinforcement learning (launching publicly next year). → Applications of RFT include creating expert models for domains like law, finance, healthcare, and engineering (e.g., partnership with Thomson Reuters for a legal assistant). → o1 Mini + RFT shown to outperform the full o1 model for specific tasks with smaller, faster, and cheaper models. → Alpha program for RFT expansion now open to select organizations working on complex tasks. RFT allows models to "think and learn" in new ways with just a few examples, offering potential for domain-specific AI advancements. If you're in research, enterprise level, or need custom AI, this may be useful. However for most users this is advanced so most won't likely use RFT or need to.

[@ridrugo182]

Doin' the lord's work right here.

[@garette8672]

appreciate this

[@callmebiz]

🫡

[@staffanlundberg]

"However for most users this is advanced so most won't likely use RFT or need to." I believe these procedures will be simplified by time such that most people may be able to use them. It is a way to develope better knowledge in essentially any subject that You are "specialized" in. Hey, Chat GPT is trying to turn us all into researchers ! Love it !🤗

[@theoz1]

@@staffanlundberg so true! Over time all of these features become easier and easier to use. "Turning us all into researchers" - I like that take!

[@ChrizzeeB]

Can they fine-tune their microphones?

[@TheHawkeyede]

that was a good one ;D

[@A-Hit]

Sound recorded by a newbie. They need a professional sound guy. Pick me.

[@xevil21]

@@A-Hit Open a.i pick a.i. not You.

[@jmg9509]

Lmao

[@qijia4769]

I suppose o1 pro could improve the sound quality with one prompt.

[@robertniyazoff2591]

the guy on the very left is a genius. Bro has worked at jane street as a quant, Microsoft as a swe, and been at openai for 6 years and going

[@o1-preview]

pretty smart lad indeed

[@farisitum9594]

Our left or their left

[@combatninjaturtle]

@@farisitum9594the guy on the other end is computational biologist. So it’s obvious.

[@Ragingwasabi9000]

Which end is the computational biologist at?

[@combatninjaturtle]

He was an intern at microsoft and jane street. He is sharp nonetheless.

[@Luxcium]

next time they should have a better sound don’t you guys think??? 🤔

[@CaioBrazVS]

Yeah, plsss!

[@sanchitkaul5084]

Turn off "stable volume " in settings

[@orterves]

Billions of dollars, dollar store microphones

[@emilyd4385]

100% the static noise and muffled mics are distracting to the incredible work they're showcasing

[@akhilj307]

Yes - it’s muffled

[@npc-drew]

Day 3: Audio o1 -- Delivers movie studios audio-like quality.

[@o1-preview]

that would be nice

[@thepresistence5935]

brhhhhhh

[@Mastertingus]

You almost guessed , except it's Video and not audio..

[@JohnDoe-fh9kn]

so cool to watch this all unfold. the work you guys do is incredible

[@sunainakhedekar6802]

all these people complaining about voice quality. honestly the people who are 100% invested in the actual content don’t bother to be bugged by something like this. I honestly didn’t realise something was off until I read the comments. It’s rather a clever filtration tactic among the audience watching!

[@AbhilashKorraprolu]

Its cute and adorable how you guys have the script in front of you, but react like you are hearing things for the first time :P Cheers guys

[@TheRealTommyR]

that is exactly what actors are paid to do.

[@McDonaldsCalifornia]

Lol i love how the expert didn't sound excited at all. "Yeah might be cool in the future. Getting better, cant really be compared to existing tools"

[@kairi4640]

And the woman is just like 😃 to make up for his lack of excitement. 😂

[@martymarl4602]

Basically they are saying"help train our models to replace you" with a smile. Not so that you can get more done (short term), but that open AI can use your data to compete in your area of expertise (long term, 1 year from now)

[@o1-preview]

@@martymarl4602 nah, horse drivers were scared to lose their jobs to cars. its not about the tool, its about who is driving the tool.

[@o1-preview]

reminds me of gpt2 days, it will be cool in the future

[@martymarl4602]

@@o1-preview If you don't realise that the "tool" will be superior to the "Drivers" shortly, you're on the wrong page mate

[@Dan-e6r7s]

OpenAI's $157B valuation suggests that there is no money left for good sound quality.

[@SurrealDistractions]

They should replace thier current sound engineer with AI

[@o1-preview]

lol, I feel bad for who ever it is, the comment section is mostly about this, i bet it'll be his/her hero arc moment

[@zurgmuckerberg]

There's probably no sound engineer, just their internal IT guys.

[@OrofinX]

Yes, I'm in shock 😮. Just put a mic on their clothes.

[@A-Hit]

Pick me

[@GenAIWithNandakishor]

Fix the fkknn sound

[@BrianMosleyUK]

Amazing the groundbreaking functionality being presented here, and so many people are thinking about the sound quality. Tumbleweed.

[@naft3R]

Please update so o1 has access to project files. It makes it very difficult to work on medium size projects when you cant provide enough content due to the amount of files

[@BruceHartford3]

Looks like that feature is included in the latest update. :)

[@naft3R]

@BruceHartford3 it only supports images

[@jr-hp7er]

You can only upload the images, not files ex. Pdf can't be attached ​@@BruceHartford3

[@101RealTalker]

What are your metrics for a "medium-sized" project?... I have over 3.5 million words worth of research representing a single project, what size would that qualify as in comparison?

[@naft3R]

@@101RealTalker a model needs full context of a web app in order to make changes and not break the code. Right now all you can do is copy paste code, but other languages support project files.

[@ilyasalcantara6450]

start using ai for giving u guys tips on how to have a proper audio

[@Iwantalloftheinformation]

It makes them look out of the garage like. They're definitely not anymore.

[@watchingvideos9871]

@@Iwantalloftheinformationwhat lol

[@seanmikejaffer423]

Thanks a lot for the valuable feedback

[@b2brish]

OpenAudioAI might be the next billion-dollar industry!

[@cjgoeson]

Nice sip noise 0:08

[@jasondisney]

Followed by *gulp*

[@TheCosmy2012]

Good soup

[@alejandromedina1019]

@@TheCosmy2012 mmtsssaaaaahhhh

[@allenlawson9872]

Literally made me stop the video LOL

[@GeoMeridium]

We need some Vic Berger style edits of these releases

[@千修-n8x]

My understanding of OpenAI’s reinforcement learning is: it establishes a foundational cognitive framework (understanding the essence of the world) through the scoring model and leverages reinforcement learning and experience accumulation to continuously optimize the AI’s reasoning and decision-making capabilities. In other words, it learns the logical thinking of experts in each field.

[@KaizenKaizen-pro]

Good morning, My name is Mohamed Abdallah, and I am a young person passionate about artificial intelligence. Since discovering OpenAI, I have been captivated by your vision and projects. You inspired me to take a deep interest in this field and develop my own AI, which I called "Kaizen". My biggest dream is to one day work at OpenAI. Your commitment to innovation and positive impact has motivated me to push myself and imagine solutions that could, in turn, transform people's lives. Thank you for showing enthusiasts like me what is possible. I hope to have the opportunity, in the future, to contribute to your incredible projects. Sincerely, Mohamed Abdallah

[@KaiPlayz7689]

Good morning, The comment section is not a email section 💀

[@bluestar1234able]

​@@KaiPlayz7689he doesn't know we're all being replaced soon 😔

[@ozanaydn4705]

Blud even made ai wrote this 😅

[@KaiPlayz7689]

@@bluestar1234able i do and I know this is written by ChatGPT

[@jaketron.seattle]

"our models will bring a new breakthrough in healthcare, physics and mathematics" me using o1 to write standup comedy bids about my workplace, while simultaneously teaching me spanish phrases "Mis colegas están como cabras"😂😂😂

[@IceMetalPunk]

4o is definitely a much better model for creative tasks like that. And cheaper, too.

[@PatrickBulteel]

Your colleagues are like goats?

[@jaketron.seattle]

@@PatrickBulteel 🤣😂😂😂 that joke was made by the most advanced model in the world, Mr!

[@R.E-O]

@@PatrickBulteel It means they are crazy.

[@IceMetalPunk]

@@R.E-O You sure it doesn't mean they eat anything they find in the trash?

[@markmatzke]

Impressive results from fine-tuning! The performance improvements in Top-1, Top-5, and Top-at-Max accuracy are exciting, especially in the context of such complex bioinformatics tasks. The clear visualization really helps to showcase the impact of reinforcement learning. Looking forward to seeing how this approach evolves and what new applications it could be used for in healthcare and other fields. Great work!

[@madeline-onassis]

awesome new developments big brains delivering big prizes to humanity

[@halbarba]

At OpenAI's conferences, especially during their '12 Days of OpenAI', they seem to have developed not just advanced AI, but also mysteriously bottomless coffee cups

[@Silus1008]

I hadnt realized that Open AI had come so far in AI video generation! These guys struck me basically as human, all four of them!

[@micbab-vg2mu]

I work for a pharmaceutical company in the medical department, and I cannot wait to test the Reinforcement Fine-Tuning. Thank you!

[@FrancisZhou-thecrocodilekeeper]

When will the video call and screen sharing features come out? Have been waiting for a looooong time...

[@fiddlestix26]

Literally like has that been scrapped or what?! Also I feel like I’m always the only one that is ever asking where tf that feature is.

[@epicboy330]

@FrancisZhou-thecrocodilekeeper you have to realize how much computing power that would take to do on a global server. It’s probably years out before it’ll release and be a reasonable price. That’s simply a hardware limitation. The biggest problem in tech rn is that our software capabilities far exceed our hardware

[@fiddlestix26]

@@epicboy330 ummm they literally demoed it in a keynote presentation like a year ago and said it would be available in the “coming weeks” and then just never mentioned it again.

[@modalmixture]

What's quite interesting to me is that they're doing bioinformatics ML, the kind of stuff you would do with traditional deep learning methods or maybe a reinforcement learning model like AlphaFold, but they're using a pre-trained *language* model, and not only does it do pretty well, it can explain its reasoning! Shows that these frontier LLMs really can generalize their intelligence over many domains. I find it deeply, deeply profound that language turned out to be the key to intelligence.

[@pachvandio]

Amazing how audio quality will make a massive impact on what people remember from a video…

[@deadsippy]

Is there a more detailed benchmark in the works to compare the performance of RFT against o1-mini? The val-accuracy improvement and the top-k tests for the specific task seem promising but I'd love to learn more specifically about what the model is learning through RFT.

[@Red-rummy]

Guys. I love what yall have done. Just playing around with GPT4o I am able to use it as my paint brush in ways I never thought possible. Thank you for everything yall do. Please stay ethical. I see what we can do with this technology and it’s amazing and a little scary. But in my eyes openAI is the gold standard hands down.

[@wolpumba4099]

*OpenAI Introduces Reinforcement Fine-Tuning (RFT) for Customizing AI Models* * *0:00** Introduction:* OpenAI announces Reinforcement Fine-Tuning (RFT), a new method for customizing its large language models, specifically the O1 series. RFT uses reinforcement learning, moving beyond standard fine-tuning to allow models to reason in new ways over custom data. Public launch is planned for next year, but an alpha program is available for researchers and enterprises. * *1:02** RFT Use Cases:* RFT is suited for domains requiring deep expertise, such as law, finance, engineering, and insurance. A partnership with Thomson Reuters demonstrates RFT's use in creating a legal assistant AI. * *2:22** RFT vs. Supervised Fine-Tuning:* Unlike supervised fine-tuning, which focuses on replicating patterns in input data, RFT teaches models to reason differently. It involves grading model responses and reinforcing successful lines of thinking, requiring only a few dozen examples for effective learning. * *3:50** RFT in Scientific Research:* Justin Reese, a computational biologist, highlights the potential of RFT in rare genetic disease research. RFT can help identify causative genes based on patient symptoms, potentially shortening the diagnostic odyssey for millions. A collaborative dataset from scientific publications was used, containing patient symptoms, absent symptoms, and the causative gene. * *6:24** RFT Demo:* A demonstration shows how RFT improves O1 mini's performance to surpass even the full O1 model in identifying causative genes based on symptoms. * *7:17** Training Data and Graders:* The demonstration explains training datasets (JSONL files), validation datasets, and graders (functions scoring model outputs against correct answers). Graders provide feedback for the reinforcement learning process. * *13:29** Evaluation Results:* RFT significantly improved O1 mini's performance. Evaluations measured top-1, top-5, and top-at-max accuracy, showing substantial gains after fine-tuning. * *15:03** Model Output Analysis:* The fine-tuned model's output now includes reasoning explanations, ranking potential causative genes and significantly boosting accuracy. * *17:38** Broader Applications:* RFT has potential beyond scientific research and has shown promise in various fields, including biochem, AI safety, legal, and healthcare. * *18:58** Alpha Program Expansion:* OpenAI expands its alpha program through the "Reinforcement Fine-Tuning Research Program," offering limited spots for organizations working on complex tasks with expert teams. A link to the application is provided in the description. I used gemini-exp-1121 on rocketrecap dot com to summarize the transcript. Input tokens: 21622 Output tokens: 561

[@aomameditation3497]

Thanks for new video as an active user of ChatGP-4o. I'm looking forward the next video.

[@DarkandTwisted]

The bad audio makes it look as if you guys used PlaySkool mics.

[@o1-preview]

small room

[@mz8755]

When she smiles my heart melted

[@GinaRealTalk]

Did I hear this right? An alpha program for creative and functional AI offerings?! OpenAI is leading the way in modeling what intergenerational collaboration can look like in the Age of AI, and I’m here for it! Could this mean I can leverage my 15 years of expertise as a Bilingual Relationship & Family Therapist to design a GPT guide that coaches wellness and resilience skills? This feels like the perfect opportunity to reimagine intentional self-care-teaching the skills to thrive before therapy becomes necessary. My inner MCU nerd is geeking out at the possibilities!

[@jtfxjhggvkuhhvktgu]

Therapy should NOT be done by AI at this point in time. An AI can do its best to predict the perfect response but a therapist should be focused on providing actual help that will fit best for the client as im sure you're well aware, with an AI you'll at this point in time have edge cases with bad responses whch can harm the client instead...

[@augustuslxiii]

@@jtfxjhggvkuhhvktgu Right, because human therapists could never give bad responses or harm their clients. Oh wait. Some can, and do, and for hundreds - if not thousands - of dollars.

[@TC-jo2vj]

🤮

[@TC-jo2vj]

@@jtfxjhggvkuhhvktgucause therapy is a scam, correct.

[@homeyworkey]

​@@jtfxjhggvkuhhvktgunot everyone can afford therapy. It's a good jumping off point for the many (mostly men) who are doubtful of it aswell

[@gridvid]

That's actually a really cool and important feature 👍

[@shubhamcweb]

People talking about the sound here are probably better off not watching the video at all. I can’t think of a better way to introduce your product in a friendly and concise way than this. I heard everything in the video correctly. They showed a whole new functionality of their product and all that the Karens noticed was sound.

[@AriaAlessandra]

I love this! I just wish they allowed the 128k context window for plus users. Even if it’s in the “4o mini” version, so it can be cheaper for open ai, but please remove the 8k cap

[@Patrick-gm3fb]

They should have an option for a mechanism to offboard some of the system weight onto your own resources. It might increase latency but it would be worth it for people who have a ton of RAM and need the larger context window. And of course, it would be a win-win because it offloads some of their resource needs that could be better utilized for people who don't have thier own local resources.

[@AriaAlessandra]

@ lol I’d be one of those cause I have 8gb of unified ram 😂

[@Patrick-gm3fb]

@@AriaAlessandra I've got 32 GB of RAM and when I run a locl LLM the context window is absolutely _massive_. My prediction: RAM is going to start to be in very high demand pretty soon. Start buying now if you can afford upgrades.

[@AriaAlessandra]

@@Patrick-gm3fbMac doesn’t have that option unless it was the highest model. 😖 I do intend soon-ish to upgrade to 32 gb of ram, but other than ai, Mac has done magic with 8, so that’s what I got this one. We are also waiting on the image generation and image editing of gpt4o In the demo it works way way better than any other image generator but they never released it, they keep using dalle

[@Patrick-gm3fb]

@@AriaAlessandra Yet another reason why I'm loathe of 🍏 and their proprietary systems. There's always other lightweight options in the FOSS world. I have an entry level AMD MoBo and I can shove up to 128GB of RAM on there.

[@grandgigs]

All really cool but Claude 3.5 sonnet is currently still a better user experience and work flow and has separate project memory,Without the memory element it's hard to iterate on large projects,

[@naft3R]

@@grandgigs hey i would like to know how you handle claude w big project files im kinda struggling there

[@handsanitizer2457]

What kind of files and what kind of project ? ​@@naft3R

[@John-p7y7b]

When is the next video? Is it every day or 12 random days before christmas? Edit: I found out, its every weekday for 12 days.

[@ecasado7]

Love the color pallette, warm colors, looks gorgeous

[@TomMooneyUE4]

Turn on Ambient Mode in youtube settings

[@culoacido420]

why?

[@TheZEN2011]

Reasoning, I guess, tested that method several times. There are lots of methods to improve the output. They will eventually change the transformer completely to give it a boost. Reasoning will be kind of built into the neuron layers as well as other useful features. But yeah, OpenAI has made some incredible advancements. They should be proud.

[@MomentsInTrading]

Your sound quality would vastly improve if you hang some things off camera to absorb the echo. Moving blankets work well.

[@OpenAITutor]

We have come a long way, but we still have a long journey ahead of us.

[@dell_rew]

Another great video!!! Thank you guys!

[@j_hull]

So many comments about audio quality, so many who don’t get that it doesn’t matter anymore. They don’t have a thumbnail with their mouth open pointing, they don’t have jump cuts every 10 seconds, and yes-one of the mics was low-but who cares? That’s surface-level phony presentation meant to manipulate you to stay longer on valueless information. The work is all that matters now.

[@DaniGieseler]

I like the sound effect - it has a vintage/lofi quality to it

[@rustomshroff417]

There should be a test data set as well in the tool because you are indirectly training your model on the validation set as well by using graders.

[@yurijmikhassiak7342]

31% of correct answers doesn't look close to a tool that you could reliably use at work. Am I missing anything?

[@ckq]

The context is that it's incredibly difficult so 31% is pretty impressive and apparently near SoTA along with specialized models

[@grmancool]

ok that's great for researchers but doesn't sound like a production ready tool

[@yurijmikhassiak7342]

I guess that they should state that in research of some substanes it will give you let's say a list of 100 candidates from which 31 will be correct. It's better than going yourself and checking millions of options. But in healthcare having 70% of time incorrect diagnosis..

[@TheRealTommyR]

I like that they are being honest about it instead of cherry picking a case with better accuracy. Plus, the guy said this experiment was ran with missing information, so hopefully with that information included, the results would be better than the current results without AI, when comparing apples to apples.(apple products on the table)

[@aryanluharuwala6407]

maybe more than 1100 examples would help

[@hundredfireify]

Everyone is complaining about audio.. who cares? We can understand them, and even the automated captions work well on their audio. Look at their work instead of focusing on useless nitpicks about video production

[@timrod94]

Where is Day 3? 😅

[@bluestar1234able]

Business days only bro

[@elprox1290]

@@bluestar1234ablewait actually?

[@MrSchweppes]

I wonder when other companies will offer reinforcement fine-tuning! Looks amazing!👍

[@BlayneOliver]

Do we need to contact OpenAI for Reinforcement Finetuning this year (if applicable) but will have access publicly next year right?

[@markmatzke]

Thank you for the insightful presentation! I have a question regarding GPT's learning and evaluation process. In cases where the model receives a failing grade despite being told the correct answer, is this reflective of how it processes information? Could it be similar to how humans sometimes internalize incorrect patterns or concepts, even when given the correct information? Alternatively, could it be due to how humans teach or evaluate the model-where what we believe to be correct might not always be accurate? Did GPT possibly find a better answer in these cases, or is it more a matter of the model becoming confused?

[@MrSchweppes]

The most important thing about this reinforcement fine-tuning feature is that it will be available for the next generation of o models, like o2, o3, and so on.

[@SergioBarreracoding]

Lots of great things, amazing job guys! I am sure Santa will RFTe's his sleds with that joke to fix his wheels:)

[@georgeserban007]

Day 3 anyone???? 😢😢😢

[@FifthDistrictStudio]

When will Sora be available?😢

[@hasanak4896]

So let me get this straight-OpenAI’s now letting the rich kids on the block teach the AI to think better while the rest of us get to play with the toy version. ‘Reinforcement fine-tuning’ sounds cool and all, but I’m just sitting here wondering if the $200-a-month Pro-tier gets me access to these gene-guessing shenanigans or if I need a research grant for that too. Also, props for the dad jokes-nothing says cutting-edge AI like cringeworthy holiday puns.

[@GalaxyHomeA9]

Santa didn't pine tune his model because he was busy watching this video to do it.🙃

[@farbodeg2]

When it would be the third day? Tomorrow or on Monday?

[@jiamingz]

Monday

[@Rohan-rb5le]

Will we ever get o1 for custom GPTs?

[@NishitChokhawala]

After reindeerforcement learning, AI did improve and its now working piner than ever.

[@JesseFleming1990]

@6:15 "Training ollama... The uh.. oh... O1 model's to reason for effectively" Nice. Nailed it lol

[@renecouture3719]

Looking forward to o1. Great work!

[@billbond2682]

I am all here for her smile

[@greatermoose]

For better audio, turn on consistent volume setting in KZbin

[@philadams9254]

No day 3?

[@LoKSET]

Should have included the results of a fine-tuned 4o.

[@giovannibrunoro1055]

when it comes to creative writing, claude 3.5 sonnet still excels... I can't believe that OpenAI still hasn't closed that gap...

[@shiccup]

it seems openai is more focused on high profit areas

[@MRAMAZRBALLZZ]

Have you considered making this audible?

[@wholeness]

I guess the sound quality is a way to make it feel more natural and real. I get it 😂

[@neodenjin]

Please upload this into chatgpt and ask it how to fix your audio quality

[@eealliance5997]

The sound is always so low.

[@venkatgopidas9788]

Love the format.

[@dnbdsj]

I love the raw sound ❤

[@MOHAMED_DEEB84]

Room voice Eco is trouble; I couldn't continue watching. it is very noisy

[@NileshPotdar-m9m]

Cool way to employ experts to finetune your data... :) do I get copyright on the model which I fine tuned?

[@nomaditsu]

You'd think they'd know they can use AI to improve the audio lol

[@BennuBirdPred]

Can definitely be applied to GP diagnostics too...

[@NovaAmor-b1k]

Can I buy official merch eg - OpenAI t-shirt or cap? If so, where? 🙏

[@SumedhKadoo]

Turn off Stable Volume in Settings

[@o1-preview]

also, needs a bigger room

[@mikestaub]

Very nice, keep crushing it!

[@sammito_]

How different would it be to approach a problem by fine-tuning using this approach instead of just having very good system prompts with few-shot examples embedded within the system prompt?

[@adaparavikiran]

If I purchase access to the o1 pro model, will my custom GPTs automatically utilize the o1 pro model, or will I need to configure them manually to switch to it?

[@tuckersnow609]

AGI but we have no sound guy

[@mrepic8263]

Data set must not be limited to json... It should be flexible for other docs like pdf or power point or documents

[@andersberg756]

That's how you know it's not for you, but will be built into a product you use instead. Curating a dataset for the fine tuning task is at its core. You're probably thinking about RAG, where info can be supplied to the model at inference time, when you ask it stuff.

[@sky-productions]

Summary The video presents OpenAI's latest advancements in their model customization program, focusing on the introduction of reinforcement fine-tuning (RFT) for the O1 series. ​RFT allows users to train AI models on their datasets using reinforcement learning techniques, enhancing the model's ability to reason and respond effectively to specific tasks in various fields such as healthcare, finance, and legal systems.​ The video features practical applications, particularly in the context of rare genetic diseases, demonstrating how AI can assist researchers in identifying genetic causes based on symptoms. Key Points Introduction to Reinforcement Fine-Tuning 00:00 OpenAI launches O1 series in ChatGPT and discusses upcoming API integration, emphasizing its capability for improved reasoning. 01:03 Users can customize O1 with their datasets using reinforcement fine-tuning, which differs from traditional fine-tuning by employing reinforcement learning. Applications and Advantages 02:49 RFT allows developers from various domains-like legal and finance-to create expert models tailored to specific tasks, highlighting recent collaboration with Thomson Reuters. 04:11 Justin Ree discusses the potential of RFT in aiding scientific research, particularly for understanding rare genetic diseases which affect a significant population. Technical Overview 07:00 A demonstration of RFT in training a model to predict genetic diseases from symptoms, stressing that the process only requires a few dozen examples for the model to learn effectively. 12:45 The validation process is explained, with graders scoring the model's outputs based on their accuracy against known correct answers, illustrating the training's effectiveness. Results and Evaluation 15:03 The results show that the fine-tuned O1 mini model outperformed its predecessor, indicating the model's improved reasoning and generalization capabilities in identifying genetic conditions. 17:22 Evaluation metrics show how the model ranks genes based on symptoms, emphasizing the importance of both correct answers and reasoning in the outputs provided by the AI. Conclusion and Future Perspectives 19:00 OpenAI invites organizations to participate in their reinforcement fine-tuning research program, aiming to enhance AI's applicability in complex tasks across various fields. 19:53 The video closes with a light-hearted joke, reinforcing community engagement while reiterating excitement about the advancements in AI model training.

[@Just2Fast24]

are these videos made by sora?

[@jesseburstrom5920]

I can finally apply my masters in tech mathematical statistics in development. Hurray!!!

[@odrammurks1497]

why can´t we also use normal voice mode or at least speech to text in the PC App? 😕

[@DefinitlyAPerson]

0:08 How soon is it? Is it regular soon or OpenAI soon, because if it is OpenAI soon then its the same soon they said for their real time vision.

[@Alden1320]

Maybe "this time next year" soon

[@Alexander01998]

In typical OpenAI fashion, they haven't actually shipped anything on the second day of shipmas. An impossible to get into waitlist does not count!

[@tomich20]

Awesome job, i rather have real engineers with crappy audio, than Media People spending money where they shouldn't. Keep it up =)

[@brian-jv1nw]

Kudos for leaving the comments enabled

[@ShahafSegev1]

Will Sora be allowed to the public in this 12 days thing?

[@ZhensongRen]

took us from advanced high school level to expert PhD level? You need to show benchmarks in this example. What are the medical doctor would do on these genetic disease tasks?

[@IsxaaqAcademy]

I heard new things and understood it from this video; validation dataset & graders