My understanding from the talk was that any non-const method on the vector would be considered invalidating the pointers into the vector by default, but if a different behavior is needed then some annotations should be applied to the method. But Herb didn't give any more details about the annotations.
Overall it seems that Herb had presented a much simplified version of the rules compared to the PDF on GitHub. E.g. PDF distinguishes "unique_owner" and "shared_owner" kinds of pointers, but he never mentioned that in the talk. Also, the scenarios he demonstrated were relatively simple, and AFAIK there was no follow up talk with more technical explanations. I suppose this is still very much work in progress. He mentioned Rust a couple of times, but in his description Rust requires more verbose lifetime annotation. I'm not sure that's the case, he never said which version of Rust he was referring to though...
I was thinking what might be a more complicated scenario to test that logic, maybe something like sorting a vector of smart pointers. Sorting operation has to obtain non-const iterators, and it changes the vector itself, but it doesn't change what the smart pointers point to, so if you keep a pointer to one of those sub-objects, it shouldn't be invalidated.
Also, it wasn't very clear to me what happens by default in case when a function takes several pointers as inputs and returns a pointer. Lifetime of that pointer is assumed to be tied to the lifetime of one of the input pointers, but which one?..
Update. The annotated version of vector::operator[] is shown in the paper, the annotation expresses that it doesn't affect the lifetimes of the pointers:
T& operator[](size_t n) [[lifetime(const)]]
{
return data[n];
}