Question is is why do you want to disable it?
Only way to do this is to insert a text box and overlay the summary field:
This puts a text object on top of the summary field so now what you mouse over it what you see is a text object:
Only way to do this...
By Design the groups are always drillable, no way to disable it.
Don