Android from the perspective of an architect | Android's overall design and its significance

1. The practical significance of Android design

The engineering significance of architecture is to define and solve a class of problems, and to provide a guarantee for the smooth transition from demand to implementation. The traditional Android architecture (Figure 1) is well known, but different roles have different perspectives. For example, some people think that Runtime and framework are its core, or Android is considered a specific JVM platform, and some people think of it as Linux from an embedded perspective... In fact, Android is one of the very few systems that uses design to solve its own development problems. Its core is to provide a foundation for development through three capabilities: hardware abstraction, componentization, and interface layer, and to reserve a lot of room for operation and mediation for many variables.

Figure 1. Android traditional architecture

1.1 Prerequisite for Development: Hardware Abstraction

In 2008, my country was on the eve of entering the 3G era. The transformation of infrastructure made the mobile field full of variables, and no matter the equipment, hardware or software, it was still undetermined. Google, which excels in architecture and software, needed to unite all possible and even unknown forces to survive and develop in this field. Obtaining the support of mobile operators, chip suppliers and mobile phone manufacturers was the first step to survival.

The hardware abstraction layer (HAL) serves a certain purpose: it defines standard interfaces for various hardware drivers in the mobile field that have different standards, avoids Android's over-reliance on Linux, makes subsequent expansion and whole-machine integration more efficient, and meets the important demands of mobile phone manufacturers; it also isolates the Linux kernel, avoids the driver source code full of hardware secrets from being open sourced due to the GPL agreement, and protects the core interests of chip and other hardware manufacturers. The customization and integration process of traditional mobile phone OS requires modifying a lot of code, which is a lot of burden. From this perspective, the design of Android HAL is leading. Combined with AOSP's excellent code branch and module management, plus the Android build system based on GNU automake macros, manufacturers enjoy unprecedented convenience.

However, there is no fixed approach for HAL (as shown in Figure 2). Before Android 8.0, the old version of HAL was initially widely used, in which the framework directly loaded *.so and relied on it, mainly in modules such as network and Bluetooth. The old version led to too tight coupling between the framework and the specific driver interface, and later formed the traditional HAL approach, which provided certain specifications and interfaces for improvement to reduce direct coupling. However, each time manufacturers supported a new version of Android, there were still a lot of changes and adaptations. To solve this problem more effectively, Android 8.0 launched the Treble project. From then on, chip manufacturers can provide stable interfaces through Binder-based HIDL, and manufacturers can directly update the Framework without being affected by chip manufacturers, and even obtain the ability to OTA without recompiling HAL.

Figure 2. Android's design for hardware drivers

Thanks to the HAL design, Google has gained wider support around the world, especially the rapid adaptation of Android 8.0 by domestic manufacturers. HAL provides a foundation for the continued growth of Android devices and promotes powerful manufacturers to develop in depth in the two fields of device upper layer and infrastructure (Figure 3). This is reflected in the fact that manufacturers who master core technologies (such as Qualcomm, Huawei, and MTK) strengthen their competitiveness by continuously building system capabilities (supporting 5G standards, hardware capabilities, software and hardware integration, and deep customization of system capabilities, etc.), while mobile phone manufacturers with channel and resource integration advantages (Huawei, OPPO, Xiaomi, VIVO, etc.) continue to build more efficient applications based on OS to expand their territory (UI, push, store, light application, etc.). This reflects the cohesion and influence of Android HAL on the entire industry, and indirectly makes up for many shortcomings of Android itself.

Figure 3. Development trend of manufacturers with core competitiveness

1.2 Capability Hub: Componentization

How to organize and reuse capabilities is the biggest challenge of architecture, and learning from existing capabilities is a shortcut to development. Whether it is Microsoft's COM, OMG's CORBA, or from EJB to Spring, from SOA to Serverless, as the capabilities of infrastructure such as networks and terminal devices are improved, the development of these technologies shows a trend from heavy to light, from heavy dependence on the center (bus) to light dependence. Android fully combines advanced technologies in various fields and forms its own technical features based on the biggest feature of limited mobile resources: AIDL is derived from complex CORBA IDL, components are streamlined from SOA, and each independent System Service is similar to a microservice. Binder can be regarded as a DBUS that weakens the bus, has better performance, and can communicate point-to-point. The UI layout system is greatly affected by SWING, and manifest is actually the component interface description file required for APP to communicate with the system...

The technologies in the fields mentioned above are indeed beneficial to the development of Android, but they are far from enough. Recalling the HAL and the overall architecture mentioned earlier, we can see that Android is actually a hodgepodge, using a mixture of many technologies. In the past, except for Web OS such as Palm, whether it is a system built on Linux/Unix such as Meego, or Symbian, MTK, UCOS, WindowsCE, whether it is a real-time system or a non-real-time system, these mobile systems are mainly based on C/C++ and are compact and powerful. They are extremely particular about memory usage and requirements. Although they meet the usage requirements of resource-constrained devices, they bring barriers; virtual machine platforms such as KJava and .NET on Windows Phone are relatively generous in terms of memory usage and energy consumption, but they win in R&D efficiency and fault tolerance, and are therefore welcomed by many developers.

Therefore, for Google, which lacks the support of a complete mobile industry chain, choosing a hybrid architecture is both in line with its own technical philosophy and has the greatest chance of success. Therefore, the customized componentization capability takes on this mission, allowing the components to be organically combined, and the communication between applications and between applications and systems to be clearer and more constrained, ultimately helping the entire system to operate flexibly and rapidly amplify its capabilities.

By observing the startup and operation process of the Android system (Figure 4) and the use of system capabilities by APP (Figure 5), it can be found that its various capabilities have been organized according to component standards and granularity (registration and discovery of capabilities, standardization of interfaces and communications, isolation of operating space, etc.), allowing the rapidly iterating mobile phone hardware and continuously upgraded system capabilities to be revealed at the lowest cost, and concretizing and maximizing the value of reuse on the mobile device system, thereby having higher flexibility and compatibility; the significance of the software engineering behind it is to build a bridge between software requirements and design, solving the problem of smooth transition from system structure to R&D needs.

Figure 4. Overview of the Android system process architecture

Figure 5. Typical call path for using device capabilities

Of course, other companies in history have had different ideas when facing such challenges. For example, Windows Phone 8.0 chose another path. Whether it is providing C# and VB.NET frameworks comparable to JAVA, or a UI system based on Silverlight Dependency Property + XAML, or even C++/CX and a set of runtimes developed to support C++, it seems to always flaunt the diversity and complexity of its system technology, which can be regarded as a technological feast.

Meego is another example. It was expected to save Nokia from danger and was jointly launched by Intel. It completed the system construction through a combination of various open source capabilities, such as Linux kernel + QEMU emulator + QT + QML interface, but in fact it was short-lived.

1.3 Application Foundation - Interface Layer

The system capabilities are basically ready, and how to attract more developers is crucial to the long-term development of Android. Choosing JAVA as the upper-level language requires courage and ambition; in order to cater to the biggest objective fact in the mobile field in the past, present and future, which is resource limitation, it designed the Dalvik virtual machine based on register architecture and with smaller executable files, and implemented it with high quality through clean room engineering. At the same time, it provides a smooth JAVA programming method with many tools, getting rid of the limitation that only small games can be written in KJava like MTK feature phones, so that Android development has both the convenience of JAVA and good performance.

As the saying goes, SUN was acquired by Oracle in April 2009, less than a year after the release of Android 1.0. Although it was a wise decision to initially choose Apache Harmony to provide the JAVA API, it encountered many challenges, such as technical lack of support for JAVA 7/8 and Oracle's copyright lawsuits. To deal with all this, Google changed JAVA support to OpenJDK starting with Android N. In addition, Kotlin also gained Google's favor and inclusion because of its similar features and the fact that it can be compiled into class or dx bytecode (Figure 6).

Figure 6. The past and future of Android interface layers

In fact, the reason why Android dares to do this is because it has the support of its design foundation. According to my personal rough understanding, we can find clues from the Android API call link (Figure 7): no matter how the underlying dependencies, implementations and processes change, the upper-level usage will not change.

Figure 7. Three implementations of call links in Android

This means that almost all the core capabilities of the system have been implemented in the native library, and combined with the upper layer to provide good shielding. This makes it possible for other languages ​​to implement the Framework, especially a language with features similar to JAVA. So what language is it, whether it is Kotlin or not, is only a suitable choice under the pre-designed specifications.

Figure 8. An extreme possibility of using Kotlin instead of Java in the future

2. Symbolism and practice for us

In summary, Android solves the key problems of its development from three aspects:

• Hardware-driven: Forms a basis for cooperation among manufacturers and in turn exerts influence on the entire industry.

• Componentization: Efficiently organize various internal capabilities and seek faster development.

• Interface layer: meets the various usage requirements of the upper layer for system and hardware capabilities.

The development of mobile Internet industry giants is different because of different starting points and execution concepts. Apple builds its entire system around its App Store and maintains it carefully, and is at the forefront in modern API programming, whole machine experience, vertical field technology such as network/algorithm and other in-depth fields; Google uses Android to lead the way, and needs to maintain and unite different forces at all levels to form its own development characteristics. Therefore, Android provides another answer to how the system develops: in addition to paying attention to the development of the system's own capabilities, how to maintain the foundation and premise of the system's continuous development, and how to better expose and allow the outside world to use the system's capabilities are also crucial (see Figure 9).

Figure 9. Android design inspiration for problem solving

Back to ourselves, in today's world where users and interactions are important and mobile phones are people, our products have reason and necessity to extend and amplify the value of services with their connotations. It is not easy to do this. First, business iterations are getting faster and faster, and the emergence of various applications means a wide range of demands for middleware; second, the environment is changing, whether it is the variety of running hardware and devices, or the complexity and diversity of docking clusters, all have a huge impact on Alibaba's original end-side middleware; third, today when the development of basic technology is slowing down, the value of technology needs to be continuously amplified. We hope to build a pan-connected foundation for services and businesses based on our own capabilities and take it as a development vision. This requires us to think about this development issue comprehensively based on the background of the group and the main goals of the development of the core APP (Figure 10).

Figure 10. Thoughts on building pan-connectivity capabilities

Inspired by Android, combined with the environment and current situation, we continuously evolve network capabilities from three levels while meeting business goals (Figure 11).

• First, by covering online and offline, various scenarios, and devices of various forms, we continue to create efficient private protocols that support universal standards, and provide some special capabilities that other end-side networks cannot or are extremely difficult to provide, to help us build a pan-connected foundation for devices and services, users and businesses.

• Secondly, through bottom-up abstraction, non-blocking IO multiplexing, user-mode network stack support, channel capability expansion, and a multi-instance architecture that can support hybrid clusters are efficiently organized, thereby ensuring the flow and management of data at different levels.

• Finally, based on the construction of the SDK matrix and access capabilities, we have achieved the goal of service access to business, business exposure to users, and brought more value by providing rich data.

  
  

Figure 11. Systematic construction of pan-connectivity capabilities

Based on the above continuous accumulation, we can now reach a large number of devices and users, become an interface to access various services and platforms inside and outside Alibaba, and shield the diversity of clusters and devices for terminals and services respectively, to achieve the pan-connection of new retail system capabilities and users (Figure 12).

Figure 12. The position of team capabilities in the group

3. Summary

Combined with the traditional C/S concept, the information obtained by the server comes from various network terminals. The network + protocol shields or standardizes the diversity of external input to the service, so that the server used to focus on clusters and high concurrency. But now, whether it is cloud computing or utilization, it is driven by business, cost scale and marginal effect. The generational theme of development is clear. But back to the client, due to the direct influence of diversity such as environment and interaction, even dynamic technology can hardly represent all or even the mainstream of the terminal side. Therefore, competing in some local technology has become an industry "trend" on the client in the past.

It is indeed meaningful to go deep into local technology and single points. The author has also made some advances in this area, such as obtaining the top Activity of the mobile phone stack in a non-polling manner, SDK multi-instance design for Alibaba's unique complex clusters, Sophix hot fixes and cloud products, etc. However, combined with past experience and Android design, we can look at this phenomenon more systematically: that is, in addition to meeting the core demands of the business (because a lot of resources are invested, it must and must succeed, at least to a small extent), we should pay more attention to how technology can better serve the business and how to continuously tap into the moat of capabilities. Therefore, to build and develop a system well, in addition to building the core capabilities of the system, it is also necessary to maintain the premise of system development, organize the cohesion of the capabilities of each system, and meet external demands on the system.

The above are my humble opinions on the development of Android system design and technical support system. I look forward to discussing with colleagues in the industry.

Add Geek Assistant WeChat and join the technical exchange group

Long press, scan the code, follow Official account