To drive someone mad / To exasperate
Literally 'to take out of the hinge'. Describes something or someone that irritates you intensely and pushes you to the limit of your patience.
That noise drives me mad.
—Why are you so worked up? —It drives me mad when he does that.
Her attitude completely exasperates me.