EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Motivation – scaling under budget constraints

self.conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, stride=1, padding=1)

You've all been there:
You're tuning your CNN — more layers, more channels, or higher resolution?
Industry reality:
You built a prototype on a tight budget. But how to scale it for production?

Problem – Insight – Method – Result

Problem:

Scaling CNNs by tuning depth, width, or resolution — but always one at a time, which is suboptimal

Insight:

Depth, width, and resolution are interdependent — scaling them together in a fixed ratio results in better accuracy & efficiency

Method:

A compound scaling rule (ϕ) to jointly scale depth, width, and resolution together

Result:

EfficientNet — a family of models (B0–B7) that outperform prior networks at a fraction of the parameters

Importance of resolution

What is it?

Photo: Henrique Ferreira / Unsplash

The CNN Arms Race

2012

AlexNet

ImageNet winner 2012

60M Params

2015

ResNet

ImageNet winner 2015

60M Params

2018

GPipe

ImageNet record 2018

557M Params

Observation 1 – scaling any single dimension improves accuracy

~80% accuracy ceiling

Redrawn from values read off Tan & Le (2019), Figure 3 (EfficientNet-B0 baseline).

Observation 2 – but scaling all dimensions together is better

Redrawn from values read off Tan & Le (2019), Figure 4 (width scaled at each depth d and resolution r).

Compound scaling rule

d = α^ϕ, w = β^ϕ, r = γ^ϕ

subject to α · β² · γ² ≈ 2 with α, β, γ ≥ 1

EfficientNet vs. the Field — same accuracy, fewer resources

Model	Top-1	Top-5	#Params	Ratio	#FLOPs	Ratio
EfficientNet-B0	77.1%	93.3%	5.3M	1×	0.39B	1×
ResNet-50	76.0%	93.0%	26M	4.9×	4.1B	11×
DenseNet-169	76.2%	93.2%	14M	2.6×	3.5B	8.9×
EfficientNet-B1	79.1%	94.4%	7.8M	1×	0.70B	1×
ResNet-152	77.8%	93.8%	60M	7.6×	11B	16×
Inception-v3	78.8%	94.4%	24M	3.0×	5.7B	8.1×
EfficientNet-B2	80.1%	94.9%	9.2M	1×	1.0B	1×
Inception-v4	80.0%	95.0%	48M	5.2×	13B	13×
Inception-resnet-v2	80.1%	95.1%	56M	6.1×	13B	13×
EfficientNet-B3	81.6%	95.7%	12M	1×	1.8B	1×
ResNeXt-101	80.9%	95.6%	84M	7.0×	32B	18×
EfficientNet-B4	82.9%	96.4%	19M	1×	4.2B	1×
SENet	82.7%	96.2%	146M	7.7×	42B	10×
NASNet-A	82.7%	96.2%	89M	4.7×	24B	5.7×
AmoebaNet-A	82.8%	96.1%	87M	4.6×	23B	5.5×
EfficientNet-B5	83.6%	96.7%	30M	1×	9.9B	1×
AmoebaNet-C	83.5%	96.5%	155M	5.2×	41B	4.1×
EfficientNet-B6	84.0%	96.8%	43M	1×	19B	1×
EfficientNet-B7	84.3%	97.0%	66M	1×	37B	1×
GPipe	84.3%	97.0%	557M	8.4×	—	—

We omit ensemble and multi-crop models, or models pretrained on 3.5B Instagram images.

Transfer Learning – What is it?

Option A – Train from scratch

Hire someone who has never cooked before — train them from scratch

Option B – Transfer Learning

Hire a French chef — teach them sushi

Transfer Learning – EfficientNet Performance Results

	Comparison to best reported results
	Model	Acc.	#Param	Our Model	Acc.	#Param(ratio)
CIFAR-10	†GPipe	99.0%	556M	EfficientNet-B7	98.9%	64M (8.7×)
CIFAR-100	GPipe	91.3%	556M	EfficientNet-B7	91.7%	64M (8.7×)
Birdsnap	GPipe	83.6%	556M	EfficientNet-B7	84.3%	64M (8.7×)
Stanford Cars	‡DAT	94.8%	–	EfficientNet-B7	94.7%	–
Flowers	DAT	97.7%	–	EfficientNet-B7	98.8%	–
FGVC Aircraft	DAT	92.9%	–	EfficientNet-B7	92.9%	–
Oxford-IIIT Pets	GPipe	95.9%	556M	EfficientNet-B6	95.4%	41M (14×)
Food-101	GPipe	93.0%	556M	EfficientNet-B7	93.0%	64M (8.7×)
Geo-Mean						(9.6×)

EfficientNet learns better general representations, not just better ImageNet performance
Compound scaling → better feature representations

EfficientNet is Gordon Ramsay

Key takeaways

Compound Model Scaling:

single dimension scaling << compound scaling

Key formula: d = α^ϕ, w = β^ϕ, r = γ^ϕ ⟹ α = 1.2, β = 1.1, γ = 1.15

EfficientNet:

Achieves higher accuracy with less parameters and FLOPs

EfficientNet learns better general representations - not just better ImageNet performance!

Outlook – what happened since?

Direct successor:

EfficientNetV2 (Tan & Le, 2021)

Vision Transformers dethroned CNNs in 2020 (ViT):

but the scaling insight lives on

Compound scaling rule – constraints

Subject to the constraint:

α · β² · γ² ≈ 2 with α ≥ 1, β ≥ 1, γ ≥ 1

FLOPS scale differently per dimension:

Doubling depth → FLOPS ×2 (linear)
Doubling width → FLOPS ×4 (quadratic)
Doubling resolution → FLOPS ×4 (quadratic, pixels scale in 2D)

Each ϕ step → total FLOPS × 2^ϕ