Flutter Technical Principles
Flutter is a mainstream cross-platform application development framework. It develops UI interfaces based on the Dart language. It directly compiles the Dart code describing the interface into machine code and uses the rendering engine to call GPU/CPU rendering.
Advantages of the rendering engine
Using its own rendering engine is the biggest difference between Flutter and other cross-platform frameworks.
Unlike frameworks such as React Native that are highly dependent on native components, Flutter is free from native component dependence, making the interface layout more flexible and the multi-terminal display effects highly consistent. Since the rendering engine is self-built, there is more room for performance optimization, which is why Flutter is known for its fluency.
Origin of Impeller rendering engine
Flutter's rendering engine has gone through many iterations. In the early days, Skia was used on all ends. Later, in order to solve the problem of shader compilation jamming on iOS, the Flutter team developed a new generation of rendering engine Impeller. Due to its excellent performance, Impeller has become the future development direction of Flutter.
The rendering engine compiles shaders into GPU instructions, that is, binary code.
A shader is a graphics drawing program that defines how to draw a graphic, such as color, shape, transformation, etc.
Flutter has made many efforts and attempts to solve the jamming problem based on Skia, but it has always been unsatisfactory, and finally Impeller was born.
Impeller's design goals include: eliminating the first freeze, reducing frame rendering driver overhead, and utilizing modern GPU parallel rendering capabilities
Impeller and Vulkan
Vulkan is the next-generation graphics API launched by OpenGL. On Android, Flutter Impeller calls Vulkan to implement interface rendering.
- Impeller implements commonly used shaders, supports parameterization, and avoids compilation jams by pre-compiling shaders
- The layered architecture simplifies the rendering process. Each layer performs specific functions based on the capabilities of the lower layer, which is efficient and easy to maintain and update
- Drawing commands are easy to aggregate, split and parallel
- Rendering design is decoupled from graphics API
Impeller Harmonyization
Impeller Harmonyization is based on Vulkan, and the XComponent component provided by ArkUI carries the Flutter view.
Like other platforms, the native capabilities are called through Method Channel and Dart calls ArkTS.
How does XComponent carry Flutter views
The OHNativeWindow instance at the bottom of the system is obtained through XComponet. This is the native window of Harmony. Through the extension VK_OHOS_surface provided by Harmony, this window is converted into a VKSurface in Vulkan, and then the window drawing is realized through VKSwapchain.
External texture: How to embed system-generated graphics and images into Flutter for display
With the help of XComponent, the interface drawn by Flutter can be displayed in Harmony. So how to embed the interface drawn by non-Flutter, such as system camera, video, etc., into the Flutter interface? This involves external textures.
Take the camera as an example. If you use the camera in Flutter, of course the most basic way is to communicate through Method Channel to pass the data, but this process is obviously very time-consuming and the performance is worrying. Another solution is to dig holes and overlay the system page and Flutter screen, but this may bring about inconsistent operations and animations of the two sets of interfaces. Harmony adopts a more difficult data import solution, that is, importing external data into Flutter and drawing these data in the form of texture components.
Harmony external texture involves encoded data transmission, how to solve the performance problem
NativeBuffer is provided by Harmony, through which memory sharing can be achieved. Then the externally imported data can be directly used by the GPU through NativeBuffer, avoiding the performance loss caused by data copying.
Whether it is VKImage or GLTexture, NativeBuffer can be used.
How to solve the screen distortion problem
Unlike Android, Impeller Harmony solution uses GPU hardware-level synchronization mechanism to protect data reading, prevent data competition, avoid image screen distortion, and reduce idle time and improve performance through CPU decoupling.
The Flutter engine does not need to wait for the Buffer to be read and written before calling the drawing capability. It only needs to ensure that the semaphore state involving the Buffer data is read when the GPU is in the drawing queue, which reduces the performance loss caused by the CPU waiting time.
Rendering pipeline preloading
Impeller avoids runtime compilation in Skia by precompiling shaders, but these shaders need to be loaded at startup (that is, loading the rendering pipeline), which will result in obvious white screen time. To solve this problem, you need to preload the rendering pipeline.
When running, Harmony Flutter first creates a Dart virtual machine to parse the UI. During this process, the rendering pipeline is loaded synchronously, so that the first frame can be quickly displayed on the screen, and the latency is reduced by 50ms.
This part does not require developer operation, and the SDK will complete it automatically.
Hybrid development loading optimization: page preloading
To jump from the native ArkUI page to the Fluter page, you need to initialize the Flutter engine first (this process takes a long time), and then render the first frame page, which will cause obvious lag.
Developers can manually call the Flutter engine initialization API in advance to solve this lag problem. For example, when the user touches, the Flutter engine is initialized synchronously, and when the user raises his hand, the Flutter page can be immediately jumped to, and the whole process will be more streamlined.
Performance test
By preloading, half of the jump time can be saved. Compared with the Skia solution, Impeller's transition smoothness is significantly improved.
Compared with the Skia solution, the Impeller solution performs better in performance.
The future of Harmony Flutter
The Harmony Flutter adaptation team is led by Huawei, and plans to launch 1-2 relatively large versions every year. These two major versions are achieved by forking the official main version at that time. As for whether the Harmony adaptation part can be merged into the official Flutter, it depends on Google's attitude.
Combined with the launch of Flock, the community version of Flutter some time ago, the author is optimistic. Although Google's progress in Flutter updates is slow, the community version brings a lot of imagination. The coexistence of multiple SDK versions does not mean division, and competition brings more vitality. This can be referred to Java. In addition to OracleJDK led by Oracle, there are more than a dozen community versions such as OpenJDK and AdoptOpenJDK.
In addition to the existing plug-in development method, Harmony Flutter plans to launch a lower-cost solution, that is, through a unified interface description, automatically generate calling codes for each end, saving developers the coding work.
Summary
As a popular cross-platform framework, Flutter is the general trend to support Harmony. Huawei's participation has brought strong momentum to the Flutter community.
Judging from the feedback from all parties, Flutter should be the best adaptation solution after ArkTs.
Whether it is the official, the open source community, or the vast number of developers, there is a strong demand for Flutter.
HarmonyOS hopes that more applications will be adapted to Harmony as soon as possible, and developers or manufacturers also hope to be available as quickly as possible at a lower cost; ArkTS still has a lot of room for improvement. Take hot reload as an example. Flutter is already far ahead in terms of ease of use, stability and maturity.For platform-specific features, such as various Picker components and permission-free buttons in Harmony, this is the unique advantage of native languages, just like on other platforms.
Although Flutter's Harmonyization has achieved initial results, and many third-party Flutter applications have also been quickly adapted and available, it still takes a lot of time to build an ecosystem, and third-party libraries need to be jointly built and shared.
Harmony has a promising future, and Flutter's Harmonyization has brought fresh blood to this long-dormant framework.
Top comments (0)