Sitting here eating olives as I read your post! Never had a problem with them. In fact, they are on my easy to eat list. I can also eat cooked broccoli and spinach all day long, but iceberg lettuce and any fruit with a thin skin has been a big no. This is improving with med changes. My pouch is 29 years old and things definitely evolved over time. What I eat is based on what my pouch tolerates, which is pretty much how it rolls for all of us. It's very true that everyone is different.
I highly recommend the practice of trying one new food at a time.....and not just once.....but over an extended time, in small quantities. I've learned I can eat almost anything sometimes, but not all things all the time or with great frequency. I have my core go to foods, so when I want to test the waters with a new food, I am pretty certain if the new food is okay regularly, sometimes or not at all.