Coronavirus brings laptops back to life: Lattice brings new features to laptops

The outbreak of the coronavirus has brought about a dramatic change in how people work, and the laptop market has boomed as a result. Long before the virus spread across the globe, industry analyst firm Gartner noted in January 2020 that “strong business demand for Windows 10 upgrades has led to growth in the PC market for the first time since 2011, particularly in the United States, Europe, the Middle East and Africa (EMEA), and Japan.” To stay competitive in this growing market, laptop OEMs are looking to differentiate their products by adopting new technologies and adding new applications to improve productivity. While many of these technologies were originally developed for devices such as smartphones, they can also shine on laptops, giving them the power of a full-fledged laptop with the convenience and security of a mobile device and the ability to adapt well to their surroundings.

Many new features on laptops can be implemented through software, but this will shorten the battery life. To solve this contradiction, Lattice Semiconductor is developing hardware-based solutions to achieve higher performance with extremely low power consumption, while also providing great convenience for future upgrades due to the use of Lattice FPGA devices. Because FPGAs can be reprogrammed in the field, OEMs can also implement new applications on hardware by downloading software and firmware updates without redesigning the hardware.

Laptop usage continues to grow

It was once predicted that laptops would be replaced by smartphones and tablets, but due to changes in the work environment and the development of emerging technologies, the use of laptops has increased rather than decreased. Cloud computing and ubiquitous broadband connections mean that employees can also maintain good productivity outside the workplace. Laptops are often the first choice for business productivity applications such as email, word processing and video conferencing because they are equipped with full-size keyboards, high-resolution displays and cameras.

Although many of the technological innovations around power saving and artificial intelligence ( AI ) are mostly applied to smartphone platforms, they can also be applied to laptops without affecting their power consumption. For example, the display of a laptop is the main source of power consumption. We can intelligently monitor the user to determine whether the user's attention is on the computer screen. When the user's attention is shifted elsewhere, the screen brightness can be automatically reduced, thereby greatly extending battery life. This application is already widely used on smartphones, and now a similar experience can be obtained on laptops.

AI is becoming more common

Although AI is still an emerging field with great potential, it is increasingly demonstrating its value to consumers. Some AI applications have already become mainstream, and users are very happy with the convenience they bring. Facial recognition is one of them, especially as a security feature on smartphones. It is also now widely used in airports around the world to track passengers entering or leaving the country as they pass through immigration. In each of these cases, the camera can recognize the face, which increases the confidence of the computer user (or tourist entering a country) in their judgment of their identity. Now that consumers are satisfied with AI experiences, they expect smart vision features such as facial recognition to be more widely used on laptops.

Security and Privacy

Security and privacy are important for desktops, but even more so for laptops. Whether used for work or personal use, security and privacy (although technically different concepts) can be considered a requirement, as the need for security to protect sensitive data is inseparable from the need to keep that data private.

We all know that maintaining security and privacy often means sacrificing convenience. While cumbersome security measures can protect laptops used in public areas, they can be annoying or even unnecessary in familiar, controlled environments such as work or home.

Laptops can use AI to identify scenarios, which has the benefit of adjusting security settings based on where they are. When used at home, the objective risk is lower, so the settings can be adjusted to make it easier to use. In contrast, when used in a coffee shop, the security settings can be automatically increased because of the unfamiliar environment and people.

Of course, it’s not just about detecting the laptop’s environment. The same camera that scans the environment can also detect when a user leaves, such as to get a cup of coffee or go to the bathroom. The computer can then quickly lock itself without the user having to do so. Similarly, the camera can detect when someone is looking at the user’s screen from behind, taking steps to block unauthorized users from viewing the screen. With computer break-ins becoming more and more common, AI can also help monitor behavior and activity inside computers and report any suspicious events to users.

The above are some examples of laptop upgrades designed to meet the needs and expectations of modern workers. Due to limited space, this article only covers two laptop development trends:

• The features of real-time online and instant start are mainly for power saving.

• New laptop form factors. This means you can detach the screen from the body, try out new laptop shapes, or fold the laptop and screen together.

Lattice FPGAs, particularly the iCE40 UltraPlusTM and CrossLink-NX™ FPGAs, and the Lattice sensAI™ solution set for real-time edge AI applications provide an efficient way to implement these capabilities in future laptops at a fraction of the power consumption of ASIC-based solutions.

Real-time online and instant start

Today, there are only three ways to put a laptop to sleep or wake it up: press a button (perhaps with a mouse), close the lid, or wait for the computer to go to sleep. Newer laptops may be able to wake up with a voice command.

The main problem facing laptops is their SoC chip. The SoC is responsible for controlling the various activities that take place in the laptop, and therefore, it is involved in the vast majority of tasks performed by the computer. The SoC contains the CPU core and may also include other processing circuits and peripheral circuits, and it consumes a lot of energy when it is working.

Because the SoC is involved in so much of the laptop's activity, the computer can do almost nothing while the system (especially the SoC) is asleep. The only features available during sleep are options related to waking the system. No other tasks can run on a sleeping system. In some cases, if specific tasks need to be processed (such as downloading emails, network monitoring, etc.), the operating power can reach 100 milliwatts, greatly reducing battery life.

If a laptop has AI capabilities that respond to sensor signals, the computer can perform many tasks even when the SoC is asleep. The AI ​​capabilities can work or make decisions while the SoC is asleep, either waking up the SoC for further work or keeping the SoC asleep and letting the AI ​​handle the tasks that need to be done. Low-power FPGAs are ideal hardware platforms for implementing AI capabilities. They have enough processing power to perform basic AI tasks (such as presence detection) but consume much less power than a SoC. If the AI ​​determines that the SoC needs to participate in the task, it wakes it up to cooperate with the AI ​​for processing or take other actions.

The Lattice sensAI solution stack can be equipped with iCE40 UltraPlus, ECP5™ and CrossLink-NX FPGA to easily implement the above AI functions. However, so far, such AI is mostly applied to Internet of Things (IoT) devices. For example, a smart doorbell may have the following functions:

• Detect the presence of people and other objects, such as packages.

• Detect and/or identify specific faces or people. This could be a specific person registered as an authorized user (such as the occupants of a house), or someone who regularly passes by the doorbell camera, such as a uniformed courier or gardener.

• Understand some gestures or voice commands. Combining facial recognition with voice keyword or password verification can achieve two-factor authentication login and improve security.

• Use an inference engine that is compatible with a variety of sensors to minimize data transmission to the cloud.

In such applications, Lattice FPGAs can provide higher performance than standalone CPUs at lower power consumption when the SoC is not in use, because the CPU must perform tasks in software, while the FPGA uses a more efficient, hardware-based implementation. For example, a CPU controlling a smart doorbell or security camera can process 1-2 frames per second while consuming up to 100 mW of power. The Lattice iCE40 UltraPlus FPGA can process 5 frames per second at 7 mW of power, which is 3-5 times the performance of the former, and only 14 times the power of the former.

If Lattice’s sensAI solution can be used in a smart doorbell or security camera, it can also be used in a laptop to:

• Detect when the user is looking at the screen. When the user's eyes move away from the screen, dim the screen to save power; when the user's attention refocuses on the screen, increase the screen brightness.

• Adjust security settings based on where your laptop is located.

• Detects the direction the user is looking. This can simplify tasks that would normally require a mouse. Eye tracking can eliminate the need for manual operation for certain frequently performed tasks.

• Optimize power consumption of graphics rendering. You can focus on rendering the part of the screen that the user is looking at. The rest of the screen remains relatively blurred or has a lower rendering effect, thus saving power until the user's eyes move to other parts of the screen.

• Influence the narrative trajectory of the game through the player's eye movements.