Dram castle

Please tell me how to get into Deam Castle.
I have the pirates cloak, the pirate dagger, tinder and gunpowder, but i can not get through the hole. There still is someone invisable who attacks me. I can nor see or kill him!
I have the pirates cloak, the pirate dagger, tinder and gunpowder, but i can not get through the hole. There still is someone invisable who attacks me. I can nor see or kill him!