Hailstorm — the Netflix scaling solution over Android TV
by Rex Ching
Hailstorm is an Android TV scaling program designed by Netflix to reduce the Netflix integration effort for partners on the Android TV platform. It aims to shorten partner’s time to market and to unlock more pay TV operators’ device reach opportunities, making Netflix even more accessible to members around the world. As of today, we have observed an initial reduction of certification time from 3 months to 1 week from our pilot projects.
Understanding the hardware development process
As we want our certified devices (Netflix Ready Device) to deliver an optimal streaming and browsing experience to our members, we set high quality standards across key device metrics :
- Playback Quality : Present our content in optimal quality
- Graphics Performance: Deliver optimal UI browsing and discovery experience
- Network performance: Deliver highest bitrate/quality stream
- Security: Keep our content keys and members’ credentials safe
- System Stability: Optimize application uptime and context switch performance.
These metrics are highly dependent on the quality of the BSP (Board Support Package) which consists of both the hardware (electrical + mechanical) design and the system software (Middleware + Operating System + hardware drivers). In a typical development process, the SoC (Silicon-on-Chip) provider would release a reference BSP used by the ODM (Original Device Manufacturer) and (SI)System Integrator to build their own products. The product BSP should be locked down at the design verification phase where all certifications occurs, such as UL, CE, WIFI, Google CTS,GTS and Netflix NTS.
A world of fragmented hardware and software
Although Android has done a great job in consolidating the middleware stack, it is difficult to lock down every single component (BSP) through the traditional device development model. Through years of experience in integrating Netflix with our partners, we have learned a variety of lessons causing the product BSP fragmentation, making difficult to scale up from the baseline SoC BSP.
- Hardware components vary based upon each supplier’s inventory/cost
- Different ODMs or System Integrators may have fragmented software branches at different stages of the Android release
- End customer (Operator/OEM) may introduce its own software stack which could interfere with system performance.
- Operators / OEMs heavily customize Android framework, system applications and backend services
These are just a few examples that cause development overhead ,resulting in all parties over the development process (from Silicon vendor, ODM, System integrator to Netflix) having to pay the tax for it, simply because we cannot guarantee consistent certification results for all products derived from the same SoC BSP.
To defragmentize the BSP, we introduced a Netflix OTT BSP model to consolidate a set of features that could impact the Netflix quality standards described above. We started by investing with the SoC (Silicon-on-Chip) provider to qualify the SoC reference BSP which accounts for the majority of core system designs in the final products. The diagram above uses Amlogic s905x development board as an example, highlighting the key hardware components that could impact Netflix:
- Wifi/Ethernet : Networking
- SoC : all (Graphics, Playback, Security, Stability, Network)
- DRAM: System stability
- eMMC: Storage, system and stability performance
- HDMI: Video/Audio rendering and security
Notice that Wifi, DRAM and HDMI all carry high frequency signals and typically require lots of tuning in layout and software design. ODMs or System Integrators usually would reference to the SoC BSP to their product design.
Hailstorm BSP model also introduces certification checkpoints to qualify SoC and ODM BSP and the final Operator product. We collaborate with SoC partners to review their source code and ensure single source branch per silicon to eliminate software fragmentation. Furthermore, SoC would be responsible to audit the BSP of their customer to ensure the integrity and to understand the risk of changes. The goal is to reduce the integration effort required for ODM and Operators. Below you can find the workflow and the R&R (Roles & Responsibilities) of each party.
- SoC provides baseline SW/HW (CTS and GTS verified) to ODM
- ODM applies the additional HW/SW changes to build a ODM Reference which becomes the ODM BSP
- Netflix certifies the ODM BSP (Blue)
- Operator takes the ODM BSP and modifies (or requests ODM to modify) it to become a final product based on Netflix guidelines below
- Netflix certifies the final product (Blue + Yellow). We expect the final product certification will have very low level of effort based on the certified ODM BSP
ODM could customize the following for operators
- Launcher UI apk
- Pre-load 3rd party apps
Once ODM BSP got certified:
- Sideload 3rd party apps that conflict with Netflix application’s resources or cause security issues. E.g.
- Should not consume video/audio decoders or graphics (GL) memory while Netflix is foreground
- Should not conflict with access to Widevine L1 DRM resources
- Should not affect CPU bandwidth or any significant I/O throughput impact
- Should not allow any illegal content or services to the platform
2. modify hardware PCB/Component (e.g. DDR RAM) once device is certified by Netflix
3. modify any Android OS / Linux / Drivers / BSP code may affect CTS/GTS /NTS results
Once SoC BSP is certified with NTS, ODM(s) would have the option to apply the BSP directly or to customize to form their reference products. We place very strict change rules (see above) against operator products to scale the SoC/ODM BSP. The operators could take the reference as a turnkey product or could make minimal customization outside the OS framework and hardware.
Hailstorm Vision
The OTT BSP Model is the baseline that encapsulates the Netflix core features. This would offer a solution to lower our partners’ development cost and to reduce their product time-to-market. A direct benefit of this model is to enable more Android TV turnkey products. In addition, we do have a vision that could benefit not only a single BSP but could derive a family of BSP solutions based on a family of silicons.
The graphs above illustrate the potential to derive a family of BSP solutions using examples of the Amlogic S905x and the Hisilicon 98 families. Since the CPU, GPU and peripherals of all the SoCs in each family are similar, it is possible to scale BSP work from one to the others in the family without duplicating the effort. For instance:
- Reducing memory of S905x2 from 2G to 1G could mean less framework and driver optimization work on top of 905x2 2G BSP.
- Migrating BSP from x2 to y2 could be low to no effort as the dies are identical with differences limited to packaging only.
- Similarly, migrating a 1GB 98MV200 BSP to 2GB memory could be low effort, although adding CAS (Conditional Access) may raise the effort level as it could impact performance of the shared memory and security (Trust Execution Environment) resources.
And there are more BSPs that could be created based on the same mechanism such as a hybrid IP box with DVB tuners (DVB-S/T/C) or Conditional Access (CAS) on top of the OTT hardware. We will continue to invest with our partners to further scale this model so that more varieties of devices can benefit from the momentum of Hailstorm. If you would like to take full advantage of the Hailstorm program benefits, contact your business development representatives who could discuss further about the commercial requirements. Our team is looking forward to working with you!
