Qumeric t1_itkg1un wrote on October 24, 2022 at 8:59 AM Reply to comment by rehrev in [R] Scaling Instruction-Finetuned Language Models - Flan-PaLM- Google 2022 - 75.2% on five-shot MMLU / Forecasters expected this SOTA would need until 2024! - Public checkpoints! by Singularian2501 There are things called prediction markets and prediction aggregators. See https://www.metaculus.com/questions/11676/mmlu-sota-in-2023-2025/ for up-to-date predictions for MMLU SotA. Also, see https://www.lesswrong.com/posts/arveXgFbJwascKtQC/forecasting-ml-benchmarks-in-2023 for a detailed prediction from a single forecaster. Permalink Parent 2
Qumeric t1_itjko7p wrote on October 24, 2022 at 2:51 AM Reply to When do you expect gpt-4 to come out? by hducug My median is 2 months, and 80% is 6 months. I heard such estimates from 2 independent sources with alleged knowledge of matter a couple of months ago. Permalink 1
Qumeric t1_itkg1un wrote
Reply to comment by rehrev in [R] Scaling Instruction-Finetuned Language Models - Flan-PaLM- Google 2022 - 75.2% on five-shot MMLU / Forecasters expected this SOTA would need until 2024! - Public checkpoints! by Singularian2501
There are things called prediction markets and prediction aggregators. See https://www.metaculus.com/questions/11676/mmlu-sota-in-2023-2025/ for up-to-date predictions for MMLU SotA.
Also, see https://www.lesswrong.com/posts/arveXgFbJwascKtQC/forecasting-ml-benchmarks-in-2023 for a detailed prediction from a single forecaster.