Yes, there is a specific title for what your brother's father-in-law would be to you. He would be your "brother-in-law's father" or simply your "brother-in-law's dad."
In this context, the term "brother-in-law" refers to your brother's spouse, and "father-in-law" refers to the father of your brother's spouse. So, there is a family connection between you and your brother's father-in-law through your brother's marriage.