Yeah, it's all in charinfopanel. You can download the file from Nulflo's basehud repo on GitHub. The field is conveniently named "BackButton" so you should find it with no trouble.
I've also found having a close button really useful and can't live without it now. You can code your own by copying everything under the BackButton field, pasting below it in the same file, changing both field name definitions to something like "CloseButton", changing the "command" value to "close" and changing the labeltext value to something like "Close (&E)".