You're not going to get a good result with straight olive oil. Get a blender or a stick blender and try blending olive oil with jalapeno peppers (fresh or pickled) or blend the olive oil with oregano and canned fire roasted tomatoes and a garlic clove or two, you can even throw in a fresh tomato, or even try just plain spaghetti sauce. The chicken will come out moist and you'll also get a nice sauce you can use.