Brown, who joined Glasgow Warriors in the summer as breakdown and contact area coach, is to return to London after his family struggled to settle in Glasgow during the pandemic.
On his departure, the former back-rower said: "Relocating during the pandemic has been a real challenge. We moved north in the summer and since arriving my family have struggled to settle in Glasgow.
"An opportunity has come up down south and for the wellbeing of my family we have taken the decision to return.
"My family's happiness is my number one priority and we have taken the difficult decision to return to London to be closer to where my children grew up and where their friends are.
"Glasgow Warriors will always have a special place in my heart, and I'd like to thank Glasgow and Scottish Rugby for the opportunity and understanding they have shown towards my family and me. It's a great club to be have been involved with and I believe they have a strong future ahead.
"I'd like to wish Danny, and everyone at the Warriors, all the best for the rest of this season, and beyond."
Head Coach Danny Wilson said: "We are understanding of Kelly's situation and family has to come first.
"This pandemic has been challenging for everyone. It's a shame to see Kelly go, especially after such a short period of time with us, however, we are completely understanding of the position he is in and wish him well."