[SOLVED] How to substitute spaces with %20 in a substring of a line across multiple files using sed, awk, grep etc

Issue

In a recent update neomutt changed how it handles regexp matching and it's breaking my notmuch URI's in my config. The solution seems to be replacing the spaces in the URI with %20. This wouldn't be a huge deal except that I have a lot of virtual mailboxes defined across multiple config files. So here is a sample of one config:

"Inbox"                 "notmuch://?query=folder:gmail/INBOX and tag:inbox" \
"Drafts"                "notmuch://?query=folder:gmail/Drafts" \
"Sent Mail"             "notmuch://?query=folder:gmail/Sent%20Mail" \
"Trash"                 "notmuch://?query=folder:gmail/Trash" \
"Today"                 "notmuch://?query=to:[email protected] and date:today" \
"Yesterday"             "notmuch://?query=to:[email protected] and date:yesterday" \
"This Week"             "notmuch://?query=to:[email protected] and date:this_week" \
"Todo"                  "notmuch://?query=to:[email protected] and tag:todo" \
"Starred"               "notmuch://?query=to:[email protected] and tag:star" \
"Burning Man"           'notmuch://?query=folder:"gmail/Burning Man"' \
"  Work List"           'notmuch://?query=folder:"gmail/Burning Man/Work List"' \
"ATXHS"                 'notmuch://?query=folder:"gmail/ATX Hackerspace" and not tag:archive' \
"  ATXHS Members"       'notmuch://?query=folder:"gmail/ATX Hackerspace/Members" and not tag:archive' \
"  ATXHS Discuss"       'notmuch://?query=folder:"gmail/ATX Hackerspace/Discuss" and not tag:archive' \
"  ATXHS Announce"      'notmuch://?query=folder:"gmail/ATX Hackerspace/Announce" and not tag:archive'

Using sed, awk, grep, or whatever, how do I change "gmail/ATX Hackerspace" to "gmail/ATX%20Hackerspace" without effecting " and not tag:archive"?

I know that other changes need to be made, but this is the only one that I'm stuck on. Basically, I need to change the spaces between folder:" and the next instance of a double quote. I don't know if this can even be done reasonably.

Solution

Based on I need to change the spaces between folder:" and the next instance of a double quote, the following seems to be a very easy and fairly readable solution:

sed -E ':a;s/(folder:"[^ "]*) /\1%20/;ta' yourinput

It is basically a while loop where

• the body s/(folder:"[^ "]*) /\1%20/ tries to pick the first, if any, space that follows folder:" before the closing ",
• the condition to repeat the loop is that the attempt was successful (i.e. the substitution was done indeed); ta indeed tests if any s command was successful on the current line and, if this is the case, it transfer the control to the line labelled :a.

Update

Concerning the -E option, I have tested the answer above only on GNU sed. Ed Morton has tested it on OSX/BSD and the command I provided gives an unchanged output.

I thought the reason could be -E, or maybe a missing ; after ta, but this does not seem to be the case, based on Ed Morton's attempts.

I initially thought the command was POSIX-compliant, based on a the following excerpt from GNU sed's man page:

-E, -r, --regexp-extended

              use extended regular expressions in the script (for portability
              use POSIX -E).

Furhtermore on this GNU page, I read

Historically this was a GNU extension, but the -E extension has since been added to the POSIX standard (http://austingroupbugs.net/view.php?id=528), so use -E for portability.

Up to this point, however, this is what GNU says of POSIX.

If you go to that link, the last line in the Issue history section is dated 2020-03-18 15:37 and reads Resolved => Applied, but I don't know how that sites relates to POSIX.

The bottom line is: I don't know if -E is POSIX-compliant.

Answered By - Enlico

Answer Checked By - Gilberto Lyons (WPSolving Admin)