too many auto-generated attributes may slow down the context_struct_compute_av() in kernel #9

stesen · 2017-03-21T02:56:04Z

I found that if using the checkpolicy tool to generate sepolicy with cil mode, a lot of "base_typeattr_xx" will be auto-generated. And each of them contain a lot of types. So the "ebitmap_for_each_positive_bit" loop in context_struct_compute_av() function will run 10x times more than policy directly generated from policy.conf file.

if I comment the libsepol's set_to_cil_attr() function and re-generate cil. Kernel runs fast and happy as the non-cil mode

So, why we need so many "base_typeattr_xx" ? Is it a safe way to remove those auto-generated attributes?

jwcart2 · 2017-03-21T14:24:42Z

On 03/20/2017 10:56 PM, stesen wrote: I found that if using the checkpolicy tool to generate sepolicy with cil mode, a lot of "base_typeattr_xx" will be auto-generated. And each of them contain a lot of types. So the "ebitmap_for_each_positive_bit" loop in context_struct_compute_av() function will run 10x times more than policy directly generated from policy.conf file. if I comment the libsepol's set_to_cil_attr() function and re-generate cil. Kernel runs fast and happy as the non-cil mode So, why we need so many "base_typeattr_xx" ? Is it a safe way to remove those auto-generated attributes?

CIL does not allow type set expressions in av rules (allow, neverallow, etc), so an attribute needs to be created anywhere a type set expression occurs. Most of the type sets occur in neverallow rules, so this often does not cause a problem. It is true that too many attributes can slow things down. Originally, all attributes were expanded when the policy was created, but at some point the policy size started becoming too large and the kernel was modified to allow attributes to be preserved. What policy did you observe this for? Fedora's, Refpolicy, or something else?

…

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALOd-vOrFPgUzY65FjbrqSub7Y9-zVxaks5rnzxFgaJpZM4MjRUS>.

-- James Carter <[email protected]> National Security Agency

stesen · 2017-03-22T07:55:43Z

Thankyou for your reply :)

I am working on android. the latest code shows that android will using CIL in the next version.

It seems that base_typeattr_x can decrease the policy size, but use more cpu resource. So do you have some advices? Is that a good choice to set a limit number for one "base_typeattr_"?

This commit adds attribute expansion statements to the policy language allowing compiler defaults to be overridden. Always expands an attribute example: expandattribute { foo } true; CIL example: (expandtypeattribute (foo) true) Never expand an attribute example: expandattribute { bar } false; CIL example: (expandtypeattribute (bar) false) Adding the annotations directly to policy was chosen over other methods as it is consistent with how targeted runtime optimizations are specified in other languages. For example, in C the "inline" command. Motivation expandattribute true: Android has been moving away from a monolithic policy binary to a two part split policy representing the Android platform and the underlying vendor-provided hardware interface. The goal is a stable API allowing these two parts to be updated independently of each other. Attributes provide an important mechanism for compatibility. For example, when the vendor provides a HAL for the platform, permissions needed by clients of the HAL can be granted to an attribute. Clients need only be assigned the attribute and do not need to be aware of the underlying types and permissions being granted. Inheriting permissions via attribute creates a convenient mechanism for independence between vendor and platform policy, but results in the creation of many attributes, and the potential for performance issues when processes are clients of many HALs. [1] Annotating these attributes for expansion at compile time allows us to retain the compatibility benefits of using attributes without the performance costs. [2] expandattribute false: Commit 0be23c3 added the capability to aggresively remove unused attributes. This is generally useful as too many attributes assigned to a type results in lengthy policy look up times when there is a cache miss. However, removing attributes can also result in loss of information used in external tests. On Android, we're considering stripping neverallow rules from on-device policy. This is consistent with the kernel policy binary which also did not contain neverallows. Removing neverallow rules results in a 5-10% decrease in on-device policy build and load and a policy size decrease of ~250k. Neverallow rules are still asserted at build time and during device certification (CTS). If neverallow rules are absent when secilc is run, some attributes are being stripped from policy and neverallow tests in CTS may be violated. [3] This change retains the aggressive attribute stripping behavior but adds an override mechanism to preserve attributes marked as necessary. [1] SELinuxProject/cil#9 [2] Annotating all HAL client attributes for expansion resulted in system_server's dropping from 19 attributes to 8. Because these attributes were not widely applied to other types, the final policy size change was negligible. [3] data_file_type and service_manager_type are stripped from AOSP policy when using secilc's -G option. This impacts 11 neverallow tests in CTS. Test: Build and boot Marlin with all hal_*_client attributes marked for expansion. Verify (using seinfo and sesearch) that permissions are correctly expanded from attributes to types. Test: Mark types being stripped by secilc with "preserve" and verify that they are retained in policy and applied to the same types. Signed-off-by: Jeff Vander Stoep <[email protected]>

This commit adds attribute expansion statements to the policy language allowing compiler defaults to be overridden. Always expands an attribute example: expandattribute { foo } true; CIL example: (expandtypeattribute (foo) true) Never expand an attribute example: expandattribute { bar } false; CIL example: (expandtypeattribute (bar) false) Adding the annotations directly to policy was chosen over other methods as it is consistent with how targeted runtime optimizations are specified in other languages. For example, in C the "inline" command. Motivation expandattribute true: Android has been moving away from a monolithic policy binary to a two part split policy representing the Android platform and the underlying vendor-provided hardware interface. The goal is a stable API allowing these two parts to be updated independently of each other. Attributes provide an important mechanism for compatibility. For example, when the vendor provides a HAL for the platform, permissions needed by clients of the HAL can be granted to an attribute. Clients need only be assigned the attribute and do not need to be aware of the underlying types and permissions being granted. Inheriting permissions via attribute creates a convenient mechanism for independence between vendor and platform policy, but results in the creation of many attributes, and the potential for performance issues when processes are clients of many HALs. [1] Annotating these attributes for expansion at compile time allows us to retain the compatibility benefits of using attributes without the performance costs. [2] expandattribute false: Commit 0be23c3 added the capability to aggresively remove unused attributes. This is generally useful as too many attributes assigned to a type results in lengthy policy look up times when there is a cache miss. However, removing attributes can also result in loss of information used in external tests. On Android, we're considering stripping neverallow rules from on-device policy. This is consistent with the kernel policy binary which also did not contain neverallows. Removing neverallow rules results in a 5-10% decrease in on-device policy build and load and a policy size decrease of ~250k. Neverallow rules are still asserted at build time and during device certification (CTS). If neverallow rules are absent when secilc is run, some attributes are being stripped from policy and neverallow tests in CTS may be violated. [3] This change retains the aggressive attribute stripping behavior but adds an override mechanism to preserve attributes marked as necessary. [1] SELinuxProject/cil#9 [2] Annotating all HAL client attributes for expansion resulted in system_server's dropping from 19 attributes to 8. Because these attributes were not widely applied to other types, the final policy size change was negligible. [3] data_file_type and service_manager_type are stripped from AOSP policy when using secilc's -G option. This impacts 11 neverallow tests in CTS. Test: Build and boot Marlin with all hal_*_client attributes marked for expansion. Verify (using seinfo and sesearch) that permissions are correctly expanded from attributes to types. Test: Mark types being stripped by secilc with "preserve" and verify that they are retained in policy and applied to the same types. Change-Id: I516cd6920fa34411a64cb321ed7fade8bb591a91 Signed-off-by: Jeff Vander Stoep <[email protected]>

This commit adds attribute expansion statements to the policy language allowing compiler defaults to be overridden. Always expands an attribute example: expandattribute { foo } true; CIL example: (expandtypeattribute (foo) true) Never expand an attribute example: expandattribute { bar } false; CIL example: (expandtypeattribute (bar) false) Adding the annotations directly to policy was chosen over other methods as it is consistent with how targeted runtime optimizations are specified in other languages. For example, in C the "inline" command. Motivation expandattribute true: Android has been moving away from a monolithic policy binary to a two part split policy representing the Android platform and the underlying vendor-provided hardware interface. The goal is a stable API allowing these two parts to be updated independently of each other. Attributes provide an important mechanism for compatibility. For example, when the vendor provides a HAL for the platform, permissions needed by clients of the HAL can be granted to an attribute. Clients need only be assigned the attribute and do not need to be aware of the underlying types and permissions being granted. Inheriting permissions via attribute creates a convenient mechanism for independence between vendor and platform policy, but results in the creation of many attributes, and the potential for performance issues when processes are clients of many HALs. [1] Annotating these attributes for expansion at compile time allows us to retain the compatibility benefits of using attributes without the performance costs. [2] expandattribute false: Commit 0be23c3 added the capability to aggresively remove unused attributes. This is generally useful as too many attributes assigned to a type results in lengthy policy look up times when there is a cache miss. However, removing attributes can also result in loss of information used in external tests. On Android, we're considering stripping neverallow rules from on-device policy. This is consistent with the kernel policy binary which also did not contain neverallows. Removing neverallow rules results in a 5-10% decrease in on-device policy build and load and a policy size decrease of ~250k. Neverallow rules are still asserted at build time and during device certification (CTS). If neverallow rules are absent when secilc is run, some attributes are being stripped from policy and neverallow tests in CTS may be violated. [3] This change retains the aggressive attribute stripping behavior but adds an override mechanism to preserve attributes marked as necessary. [1] SELinuxProject/cil#9 [2] Annotating all HAL client attributes for expansion resulted in system_server's dropping from 19 attributes to 8. Because these attributes were not widely applied to other types, the final policy size change was negligible. [3] data_file_type and service_manager_type are stripped from AOSP policy when using secilc's -G option. This impacts 11 neverallow tests in CTS. Test: Build and boot Marlin with all hal_*_client attributes marked for expansion. Verify (using seinfo and sesearch) that permissions are correctly expanded from attributes to types. Test: Mark types being stripped by secilc with "preserve" and verify that they are retained in policy and applied to the same types. Change-Id: I516cd6920fa34411a64cb321ed7fade8bb591a91 Signed-off-by: Jeff Vander Stoep <[email protected]> Signed-off-by: Marco Zanin (B--B) <[email protected]>

This commit adds attribute expansion statements to the policy language allowing compiler defaults to be overridden. Always expands an attribute example: expandattribute { foo } true; CIL example: (expandtypeattribute (foo) true) Never expand an attribute example: expandattribute { bar } false; CIL example: (expandtypeattribute (bar) false) Adding the annotations directly to policy was chosen over other methods as it is consistent with how targeted runtime optimizations are specified in other languages. For example, in C the "inline" command. Motivation expandattribute true: Android has been moving away from a monolithic policy binary to a two part split policy representing the Android platform and the underlying vendor-provided hardware interface. The goal is a stable API allowing these two parts to be updated independently of each other. Attributes provide an important mechanism for compatibility. For example, when the vendor provides a HAL for the platform, permissions needed by clients of the HAL can be granted to an attribute. Clients need only be assigned the attribute and do not need to be aware of the underlying types and permissions being granted. Inheriting permissions via attribute creates a convenient mechanism for independence between vendor and platform policy, but results in the creation of many attributes, and the potential for performance issues when processes are clients of many HALs. [1] Annotating these attributes for expansion at compile time allows us to retain the compatibility benefits of using attributes without the performance costs. [2] expandattribute false: Commit 0be23c3 added the capability to aggresively remove unused attributes. This is generally useful as too many attributes assigned to a type results in lengthy policy look up times when there is a cache miss. However, removing attributes can also result in loss of information used in external tests. On Android, we're considering stripping neverallow rules from on-device policy. This is consistent with the kernel policy binary which also did not contain neverallows. Removing neverallow rules results in a 5-10% decrease in on-device policy build and load and a policy size decrease of ~250k. Neverallow rules are still asserted at build time and during device certification (CTS). If neverallow rules are absent when secilc is run, some attributes are being stripped from policy and neverallow tests in CTS may be violated. [3] This change retains the aggressive attribute stripping behavior but adds an override mechanism to preserve attributes marked as necessary. [1] SELinuxProject/cil#9 [2] Annotating all HAL client attributes for expansion resulted in system_server's dropping from 19 attributes to 8. Because these attributes were not widely applied to other types, the final policy size change was negligible. [3] data_file_type and service_manager_type are stripped from AOSP policy when using secilc's -G option. This impacts 11 neverallow tests in CTS. Test: Build and boot Marlin with all hal_*_client attributes marked for expansion. Verify (using seinfo and sesearch) that permissions are correctly expanded from attributes to types. Test: Mark types being stripped by secilc with "preserve" and verify that they are retained in policy and applied to the same types. Change-Id: I516cd6920fa34411a64cb321ed7fade8bb591a91 Signed-off-by: Jeff Vander Stoep <[email protected]>

This commit adds attribute expansion statements to the policy language allowing compiler defaults to be overridden. Always expands an attribute example: expandattribute { foo } true; CIL example: (expandtypeattribute (foo) true) Never expand an attribute example: expandattribute { bar } false; CIL example: (expandtypeattribute (bar) false) Adding the annotations directly to policy was chosen over other methods as it is consistent with how targeted runtime optimizations are specified in other languages. For example, in C the "inline" command. Motivation expandattribute true: Android has been moving away from a monolithic policy binary to a two part split policy representing the Android platform and the underlying vendor-provided hardware interface. The goal is a stable API allowing these two parts to be updated independently of each other. Attributes provide an important mechanism for compatibility. For example, when the vendor provides a HAL for the platform, permissions needed by clients of the HAL can be granted to an attribute. Clients need only be assigned the attribute and do not need to be aware of the underlying types and permissions being granted. Inheriting permissions via attribute creates a convenient mechanism for independence between vendor and platform policy, but results in the creation of many attributes, and the potential for performance issues when processes are clients of many HALs. [1] Annotating these attributes for expansion at compile time allows us to retain the compatibility benefits of using attributes without the performance costs. [2] expandattribute false: Commit 0be23c3 added the capability to aggresively remove unused attributes. This is generally useful as too many attributes assigned to a type results in lengthy policy look up times when there is a cache miss. However, removing attributes can also result in loss of information used in external tests. On Android, we're considering stripping neverallow rules from on-device policy. This is consistent with the kernel policy binary which also did not contain neverallows. Removing neverallow rules results in a 5-10% decrease in on-device policy build and load and a policy size decrease of ~250k. Neverallow rules are still asserted at build time and during device certification (CTS). If neverallow rules are absent when secilc is run, some attributes are being stripped from policy and neverallow tests in CTS may be violated. [3] This change retains the aggressive attribute stripping behavior but adds an override mechanism to preserve attributes marked as necessary. [1] SELinuxProject/cil#9 [2] Annotating all HAL client attributes for expansion resulted in system_server's dropping from 19 attributes to 8. Because these attributes were not widely applied to other types, the final policy size change was negligible. [3] data_file_type and service_manager_type are stripped from AOSP policy when using secilc's -G option. This impacts 11 neverallow tests in CTS. Test: Build and boot Marlin with all hal_*_client attributes marked for expansion. Verify (using seinfo and sesearch) that permissions are correctly expanded from attributes to types. Test: Mark types being stripped by secilc with "preserve" and verify that they are retained in policy and applied to the same types. Signed-off-by: Jeff Vander Stoep <[email protected]>

stesen · 2018-09-03T08:34:06Z

Thank You :)

stesen closed this as completed Sep 3, 2018

Aug	SEP	Oct
	01
2021	2022	2023

too many auto-generated attributes may slow down the context_struct_compute_av() in kernel #9

too many auto-generated attributes may slow down the context_struct_compute_av() in kernel #9

stesen commented Mar 21, 2017

jwcart2 commented Mar 21, 2017

stesen commented Mar 22, 2017

stesen commented Sep 3, 2018

too many auto-generated attributes may slow down the context_struct_compute_av() in kernel #9

too many auto-generated attributes may slow down the context_struct_compute_av() in kernel #9

Comments

stesen commented Mar 21, 2017

jwcart2 commented Mar 21, 2017

stesen commented Mar 22, 2017

stesen commented Sep 3, 2018