So I followed the trick of standing behind the pillar. The first time he got mad and became aggro but didnt attack because I had pillar between him and me. I plucked two more times and he complained but didn't attack.
But how do you escape? Calling Home is the only way I can see for most people.
I'm a necro so I was able to let pet take the aggro and then run to a safe distance and FD. Tip: send pet in first or his first hit may kill you.
But not everyone has this option.