RAND Forecasting
Initiative
Archive
The RAND Forecasting Initiative has concluded. This is a permanent read-only archive. Accounts, forecasting, and comments are no longer available.

← All questions

How many total unique AI systems will Baidu and Alibaba submit for the next round of the MLPerf benchmarking suite?

Multiple choice Opened 28 July 2022 Closed 16 November 2022 Resolved 10 November 2022 72 forecasts 24 forecasters
The MLPerf Benchmarking suite measures how fast systems can train models to a target quality metric. MLPerf has emerged as an industry standard for companies to publicly show how fast their hardware has become for solving machine learning problems. Here is a short summary of the current benchmarks and metrics with a detailed description of the motivation and guiding principles behind the benchmark suite.
This question will resolve using results reported on the MLCommons website for the December 2022 round. We expect results for the next submission round to be available in early December 2022. All results to date are available here. Results from previous rounds can be viewed by selecting them from the “Other Rounds” dropdown box. To count the total number of AI models submitted by Baidu and Alibaba:
  • First, under the “Closed” tab of the spreadsheet, look for the rows where Alibaba or Baidu is listed under the “Submitter” column. 
  • Each row represents an AI system with a unique number in the “ID” column.
  • The total number of unique AI systems is equal to the total number of rows where either Baidu or Alibaba is listed under the “Submitter” column.

Baidu made 4 submissions in v2.0 (Jun 2022) and 2 in v1.1 (Dec 2021). Alibaba made 4 submissions in v0.7 (Jul 2020) and 1 in v0.6 (Jun 2019).

Outcome

Answer Final crowd forecast Resolution
Less than or equal to 4 34% Happened
Between 5 and 9, inclusive 57% Did not happen
More than or equal to 10 9% Did not happen

“Final crowd forecast” is the last aggregate crowd probability computed before the question closed – the right-hand end of the chart below, which this table is the accessible equivalent of.

Crowd forecast over time

0% 25% 50% 75% 100% Jul 2022 Sep 2022 Nov 2022 57% 34% 9%
Between 5 and 9, inclusive Less than or equal to 4 More than or equal to 10
The aggregate crowd probability, one point per day, from RFI's primary aggregation method. Gaps are days with no computed aggregate. Individual forecasts are not shown.