CORE-Bench Presentation

  Рет қаралды 35

Zachary Siegel

Zachary Siegel

Күн бұрын

Пікірлер: 2
@MarkRoth8
@MarkRoth8 7 күн бұрын
Thank you for sharing this research, Zach! Were there any common patterns you noticed from the instances of published research that tended to make it easier for the agents to be able to reproduce the results?
@ZacharySiegel
@ZacharySiegel 7 күн бұрын
Hi Mark, great to hear from you! We included papers from 3 disciplines: computer science, medical sciences, and social sciences. Agents scored about 20% higher on computer science tasks compared to the other two fields. It turns out that this is explained by the fact that computer science tasks tend to be written in Python, whereas medical and social sciences tasks are more in R. These R tasks are harder to reproduce because the dependency installation process is generally much more tedious, and the results are outputted in long PDF, which can be difficult to search through. You raise an interesting application though; if agents could be used to identify why certain projects are not reproducible, more tailored guidance towards authors in various fields could be provided to preemptively address certain issues!
Tools EVERY Software Engineer Should Know
11:37
Tech With Tim
Рет қаралды 23 М.
Hardy's Integral
13:47
Michael Penn
Рет қаралды 15 М.
«Жат бауыр» телехикаясы І 26-бөлім
52:18
Qazaqstan TV / Қазақстан Ұлттық Арнасы
Рет қаралды 434 М.
What's the REAL Reason Writing Things Down Actually Works?
6:47
How to make a Multiplayer Cloud Platformer Game on Scratch!
21:52
Zachary Siegel
Рет қаралды 18 М.
37. OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement (ACL 2024) 리뷰
27:00
Human-AI Collaborative Programming Platform
Рет қаралды 13
Scratch Presentation
25:05
Zachary Siegel
Рет қаралды 509
How to make an INFINITE Multiplayer Cloud Game in Scratch 2.0!
10:40
Zachary Siegel
Рет қаралды 15 М.
China | Math Olympiad | Nice Algebra Exponential Problems
11:09
Scratch 2.0 Tutorial Video: Pen Tools
7:21
Zachary Siegel
Рет қаралды 157