The outbreak of COVID-19 has become a threat to the entire world. Wearing a face mask is an effective method to prevent from droplet spreading of COVID-19 in public areas. Face mask detection is an emerging technique that can be utilized on IoT devices to monitor and remind people to wear face mask. This paper focuses on face mask detection using deep learning on IoT devices. Specifically, four convolutional neural networks, namely, MobileNet V2, Inception V3, VGG 16, and ResNet 50 are implemented for face mask detection and tested by running inference on mobile GPU powered IoT devices including NVIDIA Jetson TX2 and NVIDIA Jetson Nano. Performance comparison and analysis are carried out using various sizes of training data, and the experimental results demonstrate that these deep learning models are robust to various size and quality of the available training data.