In 1956 Crowe joined the Sunny Mountain Boys, a band headed by Jimmy Martin. With this group Crowe was not only expected to pick but also to sing, in a style somewhat different from the one he'd developed. "Jimmy's timing was more outgoing," he said. "He wanted all the backup instruments going real hard all the time to match his singing, so I had to simplify my style.... It was straighter playing and hard driving. Jimmy always stressed rhythm and timing, and playing what fit the song."

Martin's was a traditional bluegrass band in many respects, and it served as an excellent apprenticeship for the more progressive Crowe. The 1960s found Crowe listening to rock 'n' roll, folk, and jazz, and although he adhered to the bluegrass tradition onstage, he was beginning to experiment on his own. Finally he formed his own group, the Kentucky Mountain Boys, a trio completed by the talented pair Doyle Lawson and Red Allen. Powerhouse instrumentals and fine vocal harmonies quickly made the Kentucky Mountain Boys a favorite on the bluegrass scene.

The group disbanded in the early 1970s and Crowe formed a new band, J. D. Crowe and the New South. The original New South included Tony Rice, Ricky Skaggs, and Jerry Douglas--all young, wide-ranging musicians like Crowe; the band's debut album caused a sensation. The New South, released in 1975, offered a revolutionary blend of the contemporary and the traditional, with electrifying instrumental riffs in a series of up-tempo songs. In Country Music U.S.A., Bill C. Malone wrote: "J. D. Crowe's New South never lost their identity as bluegrass musicians, nor their rapport with older fans, even though they roamed far and wide for material and pushed their instruments to limits generally not sought by other performers. Superb musicians, such as Larry Rice, Tony Rice, Doyle Lawson, and Ricky Skaggs, moved in and out of the New South, but all of them learned something from the timing, precision, and clarity exhibited by Crowe himself."

The popular New South served as a training ground for a number of superstars in country and bluegrass, especially Skaggs, Lawson, and Keith Whitley, who joined in the early 1980s. Each time the personnel changed, the band's sound changed too, much to the chagrin of bluegrass purists. Crowe's albums have become slightly more country-oriented over time, blending drums and electric instruments with the bluegrass strings. Freeman noted that for Crowe, "change is a matter of expanding rather than abandoning his bluegrass base.... It is a posture that is uniquely suited for a man with the myriad talents of J. D. Crowe."

Recent years have seen further changes in the New South lineup, and Crowe has begun to produce recordings in Nashville and been known to work solo as well. "I don't want to continue doing what I did in 1970," he said. "It's like riding a dead horse.... That's where progress comes from, how new groups evolve." Crowe was quick to add, however, that he has nothing but pleasant memories of his accomplishments with the New South, a band that many have considered the "outlaws" of bluegrass. "Anybody that's been a part of the New South, I don't feel they can say anything but that we enjoyed it, burnt hell out of a lot of songs, and put it down like it should have been done," he said. "Everybody had a good time." He added that at present, "we're looking forward to playing new places and for new audiences. Basically, it's a matter of playing what you like and what you feel, and hopefully, the people will like it too."

With Jimmy Martin and The Sunny Mountain Boys:
-Good 'n' Country Decca.
-Country Music Time Decca.
-Widow Maker Decca.
-Big 'n' Country Instrumentals Decca.

With The Kentucky Mountain Boys:
-Bluegrass Holiday Rebel.
-Blackjack Rebel.
-Model Church Rebel.

With The New South:
-J. D. Crowe and the New South Starday.
-The New South Rounder.
-You Can Share My Blanket Rounder.
-My Home Ain't in the Hall of Fame Rounder.
-Somewhere Between Rounder.
-Straight Ahead Rounder.
-J. D. Crowe Live in Japan Rounder, 1987.