Why use foods with fat but not fat itself?

I eat a low fat, no added fat diet for health. I do use wholesome ingredients that contain a little fat naturally, such as beans, wheat, cocoa and tofu, because the body does need some dietary fat to survive. (A wholly fat-free diet deprives the body of necessary nutrients). However, I don't add any additional high fat ingredients such as nuts, seeds, olives, coconut milk or avocados to my diet or recipes, with one or two exceptions. I also never use oil, butter or margarine in my recipes beyond the occasional spritz of cooking spray to prevent sticking during baking. Basically, I consider any food made with no added fats to be "fat-free" and I strive to eat (and cook) without added fats, especially oil.