MLPerf Mobile

MLPerf Mobile is a free, open-source benchmarking tool designed to measure the performance of mobile devices like smartphones and tablets in various artificial intelligence (AI) and machine learning (ML) tasks. Workloads tested include image classification, language understanding, super resolution upscaling, and image generation based on text prompts. This benchmark makes use of hardware AI acceleration on many of the latest mobile devices to ensure the best performance where possible.

MLPerf Mobile is built and maintained by the MLPerf Mobile working group at MLCommons®, a non-profit AI/ML engineering consortium made up of 125+ members including industry firms and academics from a host of different institutions around the globe. MLCommons produces world-class benchmarks for AI training and inference across many system scales, from large data center installations to tiny embedded devices.

Features of MLPerf Mobile include:

- Benchmark tests across various domains based on state-of-the-art AI models, including:

- Image classification

- Object detection

- Image segmentation

- Language understanding

- Super resolution

- Image generation from text prompts

- Custom-tuned AI acceleration on the latest mobile devices and SoCs.

- Broad support for Android devices via TensorFlow Lite delegate fallback acceleration.

- Tailored test modes for everyone from casual users wanting a quick performance assessment to MLCommons members intending to submit official results for publication.

- Customizable cool-down delays between tests to avoid thermal throttling and ensure accurate results.

- Optional cloud-based results storage so you can save and access your past results from multiple devices in one place. (This feature is free of charge but requires account registration.)

MLPerf Mobile is typically updated multiple times each year with new tests and acceleration support as AI models and mobile hardware capabilities evolve. Please note that some benchmark tests may not be supported, and therefore may not show up as available for testing, on older devices.

Source code and documentation for the MLPerf Mobile app are available on the MLCommons Github repo. For user support or questions, please feel free to open issues in the app’s Github repo:

github.com/mlcommons/mobile_app_open

If you or your organization is interested in becoming an MLCommons member, please contact [email protected] for more information.