This is definitely a huge factor I see in the mistakes. If I hand an LLM some other parts of the codebase along with my request so that it has more context, it makes less mistakes.

These problems are getting solved as LLMs improve in terms of context length and having the tools send the LLM all the information it needs.