(IAC-824) Ability to exclude Apache MOD tests from unsupported platforms #2036
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #2036 +/- ##
=======================================
Coverage 58.49% 58.49%
=======================================
Files 12 12
Lines 212 212
=======================================
Hits 124 124
Misses 88 88 Continue to review full report at Codecov.
|
sanfrancrisko
force-pushed
the
feat/IAC-824/mod_test_exclusion
branch
3 times, most recently
from
05224c6 to
79dbdaa
Compare
sanfrancrisko
force-pushed
the
feat/IAC-824/mod_test_exclusion
branch
3 times, most recently
from
7dbbf90 to
3c410f0
Compare
|
Test example: Using changes from #2046, cherry-picked to this branch, I then provisioned all systems in release_checks_6 and kicked off the following shell function to run run_test_against_all_systems() {
platforms=($(grep 'platform' inventory.yaml | cut -d ':' -f 2 | sed 's/ //g'))
hosts=($(grep -o '[a-z]\+.[a-z]\+.delivery.puppetlabs.net\|localhost:[0-9]\+' inventory.yaml))
i=1
for host in $hosts; do
echo "Executing on $host ($platforms[$i])"
TARGET_HOST=$host bundle exec rspec $1
i=$(($i + 1))
done
}This verified that the platforms defined in this tag were filtered from execution: |
(IAC-824) Added more tests, refactoring and fixes (IAC-824) Fix Rubocop violations (IAC-824) Add guard for no tag scenario. Move init to RSpec.configure. (IAC-824) Fixes from 1st integration test results (IAC-824) Invert query and response for mods. After discussion, it was determined that it read much better to have a method called 'mod_support_on_platform' than 'mod_unsupported_on_platform' (IAC-824) Update README with Apache MOD test excl details (IAC-824) Add ability to disable Apache MOD test exclusion functionality (IAC-824) Tweak headingin README (IAC-824) Some minor fixes discovered from testing (IAC-824) Remove accidentally checked in binding.pry
sanfrancrisko
force-pushed
the
feat/IAC-824/mod_test_exclusion
branch
from
3c410f0 to
7c0d21d
Compare
sheenaajay
approved these changes
Jul 3, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description / Context
We have been battling a long running issue with tests that use Apache MODs failing on certain Linux platforms whenever the package or package dependencies become available on a specific platform. This has been defined in IAC-801. A number of recently raised JIRA tickets share the this root cause:
Proposed Solution
Provide:
Implementation Details
The ApacheModPlatformCompatibility class is initialised from
spec_helper_acceptance_local.The
register_running_platformis called from within theRSpec.configureblock, afterPuppetLitmus.configure!has been called and theosvariable has been correctly set with the parameters of the platform we're running the test on.The
generate_mod_platform_exclusionsmethod is called after, in theRSpec.configureblock, which will parse the files undermanifests/mod, extracting all tags that follow the syntax:# @note Unsupported platforms: OS: ver, ver; OS: ver, ver, ver; OS: allThe class definition for the Apache MOD will be extracted too. A concrete example would look something like this:
The rules for defining OS/Version pairs as as follows:
@note Unsupported platforms:;),)RedHat:5-7), they should be explicitly declared (e.g.RedHat:5,6,7)all(e.g.SLES:all)Scientific: 11 SP1, 11 SP2, 12, 13). PLEASE NOTE: The "SP" value will be stripped and normalised to the major version (e.g.11 SP1->11). This is because Litmus (usingserverspecunderneath) will not report the SP level of Scientific Linux with the 'SP' tag. We could potentially map the SP level to theminorversion equivalent reported byserverspec, but that seems more like a stretch goal.operatingsystem_supportvalues in themetadata.jsonto find the acceptable OS name and version syntax:-- E.g.
OracleLinuxORoraclelinux, not:[Oracle, OraLinux]-- E.g.
RedHatORredhat, not:[Red Hat Enterprise Linux, RHEL, Red Hat]metadata.jsonmap to the names reported back by Litmus / serverspec in theosvalue.There is now a mapping of class names to unsupported OSs, and using RSpec's conditional filtering, we can now use the
mod_unsupported_on_platformmethod from within the tests to exclude it, if we're running against an unsupported platform:Please note: The onus is still on the IAC Team / Contributor to determine what tests use what Apache MOD. As described in this document, there are a few tests you'll encounter in the module:
spec/acceptance/mod_php_spec.rb, where the class under test is defined in the describe block of the testspec/acceptance/vhost_spec.rb)spec/acceptance/vhost_spec.rb)Still TODO
print_parsing_errorsmethodmod_unsupported_on_platformmethodManual Test Scenarios
Unsupported platformtagsUnsupported platformtag