Internet of Things (IoT) and Internet of Everything (IoE) are emerging communication concepts that will interconnect a variety of devices (including smartphones, home appliances, sensors, and other network devices), people, data, and processes and allow them to communicate with each other seamlessly. These new concepts can be applied in many application domains such as healthcare, transportation, and supply chain management (SCM), to name a few, and allow users to get real-time information such as location-based services, disease management, and tracking. The smartphone-enabling technologies such as built-in sensors, Bluetooth, radio-frequency identification (RFID) tracking, and near-field communications (NFC) allow it to be an integral part of IoT and IoE world and the mostly used device in these environments. However, its use imposes severe security and privacy threats, because the smartphone usually contains and communicates sensitive private data. In this chapter, we provide a comprehensive survey on IoT and IoE technologies, their application domains, IoT structure and architecture, the use of smartphones in IoT and IoE, and the difference between IoT networks and mobile cellular networks. We also provide a concise overview of future opportunities and challenges in IoT and IoE environments and focus more on the security and privacy threats of using the smartphone in IoT and IoE networks with a suggestion of some countermeasures.