1

I am preparing for lpic-1 exam and wanted to create a list of topics plus the weight of it with some adventurous curl.

the outcome I aim for would be something like

101.1 Determine_and_configure_hardware_settings 2
101.2 Boot_the_system 3
101.3 Change_runlevels_.2F_boot_targets_and_shutdown_or_reboot_system 3
...

the below gets me quite far

❯ curl -s https://www.lpi.org/our-certifications/exam-101-102-objectives/ | egrep -A7 -e 'id="10[0-9].[0-9]' | egrep '#eaeaea|^<h4>' | awk {'print $3'} | sed 's/id="//' | sed 's/"><span$//' | sed 's/_/ /' | head
101.1 Determine_and_configure_hardware_settings
2
101.2 Boot_the_system
3
101.3 Change_runlevels_.2F_boot_targets_and_shutdown_or_reboot_system
3
102.1 Design_hard_disk_layout
2
102.2 Install_a_boot_manager
2

left aside whether this is the most efficient or beautiful pipeline, I can't figure out how the get from

102.1 Design_hard_disk_layout
2
102.2 Install_a_boot_manager
2

to

102.1 Design_hard_disk_layout 2
102.2 Install_a_boot_manager 2

played a bit with 'tr' which only got me to loss lines completely

102.1 Design_hard_disk_layout 2 102.2 Install_a_boot_manager 2

any hint?

Improve this question
3
  • 1
    Note that egrep and fgrep are deprecated. Better to use grep -E and grep -F instead. They still work, but they can be removed at any time, so it's a good idea to get into the habit of using the options instead. Commented Jul 12 at 17:48
  • yes, I am aware of that. Despite the fact I am not seeing the benefit of grep -E over egrep. But when the gods of bash think so, I am sure they have their reasons. Commented Jul 13 at 19:50
  • 1
    See this Q&A for more details on that topic — it’s best to use grep -E but egrep isn’t likely to go away any time soon. Commented Jul 13 at 20:55

1 Answer 1

Reset to default
4

To merge two lines you can use the following AWK script:

awk 'NR % 2 != 0 { printf "%s ", $0; next } 1'

Add that to the end of your pipeline, before the head command, and you’ll get the desired output. This works by printing odd-numbered lines with a space at the end and no newline, and all other lines as usual with a newline; thus odd-numbered lines are concatenated with the following line.

Note that AWK can do anything that grep and sed can, so you can reduce the number of commands involved; for example

curl -s https://www.lpi.org/our-certifications/exam-101-102-objectives/ |
awk '/id="10[0-9].[0-9]/ { n=7 } n-- >= 0 && /#eaeaea|^<h4>/ {print $3}' |
awk '{ gsub("id=\"", ""); gsub("\"><span$", ""); gsub("_", " ") } NR % 2 != 0 { printf "%s ", $0; next } 1' |
head

(This is why I would go for an AWK-based answer to your “merging lines” problem; see Text processing - join every two lines with commas for other approaches. Bear in mind that parsing HTML with a non-HTML-oriented parser is error-prone.)

It is possible to simplify the extraction by only looking for lines containing <h4> or <b>Weight</b>:

curl -s https://www.lpi.org/our-certifications/exam-101-102-objectives/ |
awk '/<h4>/ { gsub("<[^>]*>", ""); printf "%s", $0 } /<b>Weight<\/b>/ { weight = 1 } weight && /#eaeaea/ { weight = 0; gsub("<[^>]*>", ""); print }'
Improve this answer
1
  • thx @Stephan Kitt , really helpful. And the pointer to awk does not come unexpected. I am just not there really yet. Commented Jul 13 at 19:48

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.