Apache Commons Exec Problems Handling Regex Argument?

Question

I'm trying to use Apache Commons Exec to run a git command which uses a regex.

When I form my CommandLine and print it out it looks like this:

[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testFindByAdAccount).*", --, *Test.java]

However when I execute this, git returns no results, resulting in an exit code 1.

When I run this command manually though, it returns plenty of results and succeeds. Changing the --extended-regexp argument to just a string like testFindByAdAccount does yield results when run via Exec, so I think Apache Commons is doing something to the regexp argument making it invalid. Any ideas what is going on?

EDIT: Adding a reproducible example

Clone https://github.com/ragurney/min-example
Run gradlew shadowJar to produce jar file for project
Run the app with java -jar app/build/libs/app-all.jar
Note the output which shows the command printed fails with an exit code 1 (because there are no results returned by the git command)

$ java -jar app/build/libs/app-all.jar
HELLOOOOOO
WD:::  null
[git, --no-pager, grep, --line-number, --untracked, --extended-regexp, "^\s*public void\s+(testAppHasAGreeting)\(\).*", --, *Test.java]
WD:::  /Users/rgurney/Src/personal/min-example

Exception in thread "main" java.lang.RuntimeException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
    at min.example.App.lambda$runCommand$1(App.java:74)
    at io.vavr.control.Try.getOrElseThrow(Try.java:748)

Running the command manually does produce expected results:

$ git --no-pager grep --line-number --untracked --extended-regexp "^\s*public void\s+(testAppHasAGreeting)\(\).*" -- "*Test.java"
app/src/test/java/min/example/AppTest.java:11:    public void testAppHasAGreeting() {

Can you share a minimal reproducible example? Perhaps the way you're putting the CommandLine or Executor together are the problem — Mureinik
– Mureinik, Commented Dec 30, 2022 at 18:25
To follow up on my previous comment - I tested, and this most definitely works with Apache Commons Exec. I.e., the issue isn't the regex, but something else (perhaps the cwd? Perhaps handling the output?) We need to see a full example and the output that doesn't work for your usecase. — Mureinik
– Mureinik, Commented Dec 30, 2022 at 18:30
Thanks for taking a look @Mureinik. I added a rough MRE, which I think captures the problem. — ragurney
– ragurney, Commented Dec 30, 2022 at 19:18
No luck - I run the main in the sample you provided (using IntelliJ) and it just works. We're missing some environmental difference here. — Mureinik
– Mureinik, Commented Dec 30, 2022 at 19:44
Found it! I was testing on my Windows laptop. Once I switched over to my Linux desktop I was able to trivially reproduce the error with the instructions you provided. See my answer below for the details and the solution. — Mureinik
– Mureinik, Commented Dec 30, 2022 at 22:42

Mureinik · Accepted Answer · 2022-12-30 22:41:34Z

I got a clue as to what's going on here when the sample you provided worked just fine on my Windows laptop but failed on my Linux desktop.

Once I made sure the git version wasn't the culprit (tested several versions between 2.17 and 2.39 on both machines), I figured the difference must be in the way different shells handle quoting. Specifically, the only argument here that has any potential quoting issues is the regex ("^\s*public void\s+(testFindByAdAccount).*"), which is added to the command line by commandLine.addArgument(regex);.

addArgument may look innocuous, but under the hood, it allows the CommandLine to handle the quoting itself (i.e., addArgument(String argument) calls addArgument(String argument, true). Since you've handled the quoting yourself, you should not allow the CommandLine to handle the quoting, and should explicitly call it with the second argument false. i.e.:

public static List<String> grep(String regex, String filePattern, String wd) {
    CommandLine commandLine = CommandLine.parse("git");
    commandLine.addArgument("--no-pager");
    commandLine.addArgument("grep");
    commandLine.addArgument("--line-number");
    commandLine.addArgument("--untracked");
    commandLine.addArgument("--extended-regexp");
    commandLine.addArgument(regex, false); 
    // Here -----------------------^
    commandLine.addArgument("--");
    commandLine.addArgument(filePattern);

    System.out.println(commandLine);

    return List.of(runCommand(commandLine, wd).split("\n"));
}

This takes the quote-handling logic away and ensures the same code runs smoothly both on Windows and Linux (at least those I've tested).

My god. I tried so many permutations of the regex including manually adding quotes but never tried no quotes at all and adding false for handleQuoting. Confirmed working in both my MRE and my actual project. Thanks so much for this! Will award the bounty when the system allows in 17 hours.

Collectives™ on Stack Overflow

Apache Commons Exec Problems Handling Regex Argument?

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related