Deciding that he wouldn't be able to forge a career in Los Angeles, Cale returned to Tulsa in 1967. Upon his return, he set about playing local clubs. Within a year, he had recorded a set of demos. Radle obtained a copy of the demos and forwarded it to Denny Cordell, who was founding a record label called Shelter with Leon Russell. Shelter signed Cale in 1969. The following year, Eric Clapton recorded "After Midnight," taking it to the American Top 20 and thereby providing Cale with needed exposure and royalties. In December 1971, Cale released his debut album, Naturally, on Shelter Records; the album featured the Top 40 hit "Crazy Mama," as well as a re-recorded version of "After Midnight," which nearly reached the Top 40, and "Call Me the Breeze," which Lynyrd Skynyrd later covered. Cale followed Naturally with Really, which featured the minor hit "Lies," later that same year.

Following the release of Really, J.J. Cale adopted a slow work schedule, releasing an album every other year or so. Okie, his third album, appeared in 1974. Two years later, he released Troubadour, which yielded "Hey Baby," his last minor hit, as well as the original version of "Cocaine," a song that Clapton would later cover. By this point, Cale had settled into a comfortable career as a cult artist and he rarely made any attempt to break into the mainstream. One more album on Shelter Records, 5, appeared in 1979 and then he switched labels, signing with MCA in 1981. MCA only released one album (1981's Shades) and Cale moved to Mercury Records the following year, releasing Grasshopper.

In 1983, Cale released his eighth album, 8. The album became his first not to chart. Following its release, Cale left Mercury and entered a long period of seclusion, reappearing in late 1990 with Travel Log, which was released on the British independent label Silvertone; the album appeared in America the following year. 10 was released in 1992. The album failed to chart, but it re-established his power as a cult artist. He moved to the major label Virgin in 1994, releasing Close to You the same year. It was followed by Guitar Man in 1996. Cale returned to recording in 2003, releasing To Tulsa and Back in 2004 on the Sanctuary label and The Road to Escondido, a collaborative effort with Clapton, in 2006 on Reprise. Roll On appeared in 2009 on Rounder Records.