Internet-of-Things (IoT) devices are widely deployed nowadays. A large number of smart home IoT devices are hosted on a cloud server for easy management. Users can use their accounts to initiate operations and management on IoT devices through a cloud server, such as updating firmware and configuring devices. However, the cloud account may be hacked resulting in adversarial attacks to the hosted IoT devices. As a consequence, an adversary may perform malicious operations through the cloud remotely to the hosted IoT devices without user awareness. Motivated by this, in this article we propose gateway-based 2 factor authentication (G2F), a secure user authentication framework dedicated for a gateway based on the universal 2nd factor (U2F) protocol to enhance the security of IoT devices management. In G2F, the user authentication on the gateway is completed utilizing a hardware token that interacts with the local gateway node to guarantee the token owner’s presence. Furthermore, G2F can grant multiple simultaneous operations on IoT devices through just one user authentication. We implement a prototype to further evaluate the performance of G2F. Based on our realization on the commercial IoT server, i.e., Alibaba Cloud, G2F demonstrates the ability to protect against malicious attacks with high authentication efficiency.