Once your training runs become material in terms of wall-clock time and hardware budget, it's time to look at improving your batch size. If your batch size is too small, your training runs are taking longer than necessary and you are wasting money. And if your batch size is too large, you are training with more expensive hardware than you need.
Previously, we showed that throwing more training data at a deep learning model has rapidly diminishing returns. If doubling your labeling budget won’t move the needle, what next? Consider semi-supervised learning (SSL) to unlock the information in unlabeled data.