It's not so much "the compiler chose not to dig through function bodies" so much as "digging through function bodies doesn't make sense here". The function body does not have a say in what the generic parameter ends up being. The caller decides that.

You were passing a specific type, a &Path, to the closure within the function body. That's a "choice" the function body made. The caller has no say in that, so it didn't make sense for the closure argument to be represented by a generic.

The closure was a solution to a follow-up problem (which was not an inference problem), and not the original (inference) problem. By that point I had removed the S parameter from find_matches.

It's still instructive, so I'll work through it. But it's not a problem about inference, so it's somewhat off-topic.

Anyway, let's start from here. The problem is that F: Fn(&Path, &parser::Grep) -> io::Result<()> means that there must be a single type F which implements Fn(&'lt Path, ...) for all lifetimes 'lt. But the notional implementation of Fn by find_match_in_file<S> cannot meet that bound. The notional implementation looks something like this:

// (Simplified for the sake of presentation, lmk if you want playground code)
impl<'grep, S: AsRef<Path>> Fn(S, &'grep parser::Grep) for find_match_in_file<S> {
    type Output = io::Result<()>;
    fn call(&self, path: S, _args: &'grep parser::Grep) -> Self::Output {
        ...
    }
}

You're trying to pass a &Path, and the bounds that are satisfied by the function are

find_match_in_file<&'a Path>: for<'x> Fn(&'a Path, &'x parser::Grep) -> io::Result<()>
// same thing:
find_match_in_file<&'a Path>: Fn(&'a Path, &parser::Grep) -> io::Result<()>

find_match_in_file<&'b Path>: Fn(&'b Path, &parser::Grep) -> io::Result<()>
find_match_in_file<&'c Path>: Fn(&'c Path, &parser::Grep) -> io::Result<()>
find_match_in_file<&'d Path>: Fn(&'d Path, &parser::Grep) -> io::Result<()>
// ...
find_match_in_file<&'static Path>: Fn(&'static Path, &parser::Grep) -> io::Result<()>

Like Vec<T> and Vec<U> are different types (unless T = U), the function items find_match_in_file<&'a Path> and find_match_in_file<&'b Path> are different types (unless 'a = 'b). So there is no single find_match_in_file<_> type which satisfies the bound on the callback -- no single type which meets the bound on the closure parameter F.

In an impl, if a generic lifetime shows up in the parameters and in the implementing type, then the impl is not creating an implementation that meets a F: Fn(&'_ ...)[1] bound. The lifetime has to show up in the parameters only, and not in the implementing type.

Or in terms of turbofish: it's impossible to write out a find_match_in_file::<X>(...) that can take a &'path Path with any lifetime, it can only take the lifetime in X.

Okay, so that's the problem. Now let's look at the solutions. You asked about the closure solution specifically:

    find_matches(args.recursive, &args.path, &args, |p, g| {
        find_match_in_file(p, g)
    })?;

Here, the closure meets the bound by having an implementation like this:

impl<'path, 'grep> Fn(&'path Path, &'grep parser::Grep) for Closure {
    type Output = io::Result<()>;
    fn call(&self, p: &'path Path, g: &'grep parser::Grep) -> Self::Output {
        // Here within the function body, the lifetimes have been "reified"
        // (to make up a word on the spot) to some concrete lifetimes,
        // similar to how a generic parameter `T` would represent a single
        // type *within* the function body.  (`T` is chosen by the caller.)
        //
        // So there's no requirement for `find_match_in_file` to meet the
        // higher-ranked `F: Fn(&Path, ...)` bound in this context.
        find_match_in_file/* ::<&'path Path> */(p, g)
    }
}

(You could also do this with another function that calls find_match_in_file, it's not closure-specific magic or anything.) Hopefully that explains the fix: the closure handles every possible input lifetime by calling a "different function type" file_match_in_file::<&'path Path> for every lifetime.

And though you didn't ask, the other two fixes work by changing how find_match_in_file<_>'s notional Fn implementations are defined.

// fn find_match_in_file(path: &Path, _args: &parser::Grep) -> io::Result<()>
//
// The function item type is no longer parameterized by a type.
impl<'path, 'grep> Fn(&'path Path, &'grep parser::Grep) for find_match_in_file {
    type Output = io::Result<()>;
    fn call(&self, path: &'path Path, _args: &'grep parser::Grep) -> Self::Output {
        ...
    }
}

// fn find_match_in_file<S>(path: &S, _args: &parser::Grep) -> io::Result<()>
// where S: AsRef<Path> + ?Sized,
//
// Now it's paramertized by a type again, but the lifetime of the `&Path`
// is not part of that type parameter.
impl<
    'path, 
    'grep, 
    S: AsRef<Path> + ?Sized,
> Fn(&'path S, &'grep parser::Grep) for find_match_in_file<S> {
    type Output = io::Result<()>;
    fn call(&self, path: &'path S, _args: &'grep parser::Grep) -> Self::Output {
        ...
    }
}

In both cases, the 'path lifetimes shows up in the parameters, but not in the implementing type.

In terms of turbofish for the generic version, you end up using find_matches_in_file::<Path>, which does not have the 'path lifetime in the <..>.

Finally, why did we need to handle every lifetime in the first place? That is, why doesn't this work.

//              vvvvv
fn find_matches<'path, F, P>(recursive: bool, path: P, args: &parser::Grep, cb: F) -> io::Result<()>
where
    P: AsRef<Path>,
    F: Fn(&'path Path, &parser::Grep) -> io::Result<()>,
    // ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We're only requiring F to work with a single 'path lifetime now, which is compatible with the original find_matches_in_file siguature. But now we get a new error:

error[E0597]: `path_source` does not live long enough
  --> src/main.rs:33:15
   |
27 | fn find_matches<'path, F, P>(recursive: bool, path: P, args: &parser::Grep, cb: F) -> io::Result<()>
   |                 ----- lifetime `'path` defined here
...
32 |     let path_source = PathBuf::new();
   |         ----------- binding `path_source` declared here
33 |     let p = &*path_source;
   |               ^^^^^^^^^^^ borrowed value does not live long enough
...
37 |         match cb(p, args) {
   |               ----------- argument requires that `path_source` is borrowed for `'path`
...
43 | }
   | - `path_source` dropped here while still borrowed

And this final part is a bit more related to the conversation about generic parameters. The caller chooses the 'path lifetime too, and again, the function body has to work with every possible choice that the caller can make -- the function body has no say in how long the lifetime is.

So the caller chould have chosen 'static for example.

In fact, it goes a bit further than that -- it's impossible for the caller to "name" or choose a lifetime that's shorter than the call to find_matches. Within the function body, generic lifetime parameters always represent some borrow duration longer than the function body. So it is never possible to borrow a local variable for as long as a lifetime parameter -- because all locals are moved or dropped by the time we exit the function body.

And we fix this problem by requiring the closure to work with all lifetimes -- including those shorter than the function body.

F: Fn(&Path, &parser::Grep) -> io::Result<()>
// AKA
F: for<'path,'grep> Fn(&'path Path, &'grep parser::Grep) -> io::Result<()>

These for<..>, "for all lifetime" bounds are called higher-ranked trait bounds (HRTBs).

1. aka F: for<'x> Fn(&'x ...) ↩︎