The news was first reported by Brett McMurphy of FanHouse.
Holtz, 45, had been the Pirate coach since 2005, going 38-27 with four bowl appearances.
Holtz told the Daily Reflector he was leaving Greenville for Tampa because of family reasons.
"With my parents living an hour from there, with my wife's parents being in that area and my wife being from that area [Port Charlotte], and looking at the decision I had to make, you don't get many opportunities to be around family," he said.
Holtz was Connecticut's coach from 1994-98, going 38-23, and left to join the staff of his father, Lou, who had been named coach at South Carolina. Lou resigned after the 2004 season and Skip was not among the assistants retained by Steve Spurrier.
Holtz had been courted in recent seasons by Cincinnati, Syracuse and Virginia.
"This has nothing to do with me not wanting to be in Greenville," Holtz said. "I have not tried to run out of here on the first boat that came by. I was not looking to leave. There are probably about 120 other jobs in the country that I would not have considered."