Rafter has been replaced by his deputy and former Davis Cup captain and coach Wally Masur.
After two years in the role, Rafter has been replaced by his deputy and former Davis Cup captain and coach Wally Masur.
Masur will have the task of bringing the Tomic family back into the fold after relations soured under Rafter.
Tomic made himself unavailable for a Davis Cup tie this month against the Czech Republic, citing scheduling issues.
Reports said Tomic's father John had a bitter ongoing feud over his son and daughter Sara involving funding from Tennis Australia.
Last month at the Australian Open Rafter, a former world number one and two-time US Open champion, admitted he had little input into the direction of the country's top two players.
Rafter stepped down as Davis Cup captain in early 2015 following a turbulent time dealing with Tomic, while Kyrgios once tweeted: "Another negative comment out of Rafters mouth. Does this guy ever stop #everyoneisaworkinprogress."
Rafter said recently that he preferred to leave Tomic and Kyrgios alone.
"I haven't really been speaking to them very much and I don't really know where they're at," Rafter said.
"I just sit back and watch from a distance. I don't really have a lot to offer them."
"If they ever want to talk to me, I'm happy to talk."
Kyrgios and Tomic have frequently fallen foul of tennis authorities and have a love-hate relationship with the public.
Kyrgios, 21, was fined for swearing and throwing his racquet in a spectacular meltdown at last month's Australian Open, when he threw away a two-set lead to bow out in the second round against Andreas Seppi.
Tomic, 24, has also been plagued by controversy, making headlines last year when he turned his racquet the wrong way to face a match point against Fabio Fognini in Madrid.
Masur said he was already in regular contact with John Tomic and confirmed that Sara Tomic had been offered funding under a restructuring by TA late last year.
"They've got to hit certain benchmarks to get X amount of funding but I feel like that's a positive because we're not saying, it's our way or the highway," Masur said.
Rafter said his time as the Davis Cup captain and TA performance director had been an "eye-opener".
"The time is right for me, I came in to Davis Cup and then in this role because I wanted to make a difference and I feel we have taken some major strides," he said Friday.
"This journey for me –- through the Davis Cup years and now as performance director –- has been an eye-opener to say the least."
"The restructuring process that we went through in September last year was intense for everyone."