RAND Forecasting
Initiative
Archive
The RAND Forecasting Initiative has concluded. This is a permanent read-only archive. Accounts, forecasting, and comments are no longer available.

← All questions

Will Google score more wins than any other submitter in the next round of the MLPerf training benchmarking suite?

Yes/No Opened 4 August 2022 Closed 16 November 2022 Resolved 14 November 2022 80 forecasts 27 forecasters

MLCommons hosts MLPerf, a set of biannual benchmarking competitions to assess how fast different machine learning programs are at various tasks including image classification, object detection, speech recognition, and natural language processing (MLCommons, EnterpriseAI). Google has been using MLPerf to test the speed of its Tensor Processing Unit, an application-specific integrated circuit (ASIC) designed to accelerate AI applications (Google Cloud). In the June 2022 (v2.0) round, Google scored 5 wins. NVIDIA scored the second most wins with 3. 


Resolution criteria: This question will resolve using results reported on the MLCommons website for the "Closed" division of the December 2022 round. We expect results for the next submission round to be available in early December 2022.
Historical data:
  • All results to date are available here. Results from previous rounds can be viewed by selecting them from the “Other Rounds” dropdown box.
  • To find the winner in each category, scroll across to the relevant column under "Benchmark results" (e.g., image classification). The winner will have the fastest time (i.e., the lowest number) in that column. Scroll left to see the submitter for that time.
  • If multiple submitters have the lowest time on a benchmark test, they will both be considered winners of that benchmark.

Outcome

Outcome Final crowd forecast Resolution
Yes 23% Did not happen

“Final crowd forecast” is the last aggregate crowd probability computed before the question closed – the right-hand end of the chart below, which this table is the accessible equivalent of.

Crowd forecast over time

0% 25% 50% 75% 100% Aug 2022 Sep 2022 Nov 2022 23% Yes
The aggregate crowd probability, one point per day, from RFI's primary aggregation method. Gaps are days with no computed aggregate. Individual forecasts are not shown.