Human Capacity

Carbon Emissions and Large Neural Network Training

Summary

Measured the carbon cost of training big AI models and found the footprint is large, swings widely with choices the builders make (model size, hardware, where the data center sits), and is fully knowable in advance to the people training the model.

Topics: Environment, Work
Access the study →

Reference

Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M., & Dean, J.. (2021). Carbon Emissions and Large Neural Network Training. arXiv (subsequently presented at conferences). https://doi.org/10.48550/arXiv.2104.10350