You really need to take care of fallen leaves as they are the main way of walnut to combat a competition. Once you rake the leaves every year you should be quite fine growing most of the vegetables that didn't require a lot of sun.