AUGUSTA, Ga. -- Fifty years ago, when Lee Elder became the first Black golfer to qualify for the Masters, he wasn't much in a mood to discuss it.
"I'm not talking," Elder told reporters on April 7, 1975. "Every time I talk, I get into trouble."
His agent, who was also his wife at the time, said he was serious about it. "Lee feels that this is the only fair way," Rose Elder said. "He's here to play golf, and he wants to be left alone."
But Elder's caddie never had a problem talking. And Henry J. Brown, who moonlighted as a taxi cab driver in Augusta, Georgia, let it be known that if Elder didn't contend in a field that included defending champion Gary Player, Jack Nicklaus, Arnold Palmer and Johnny Miller among others, it wouldn't be because of him.
"I'm No. 1," Brown said. "I can walk this course backwards. I know every blade of grass on it. All Lee's got to do is stay cool. All I can promise is that he won't be out-caddied."
While Elder would make history that week as the first Black golfer to play in the most famous tournament in the world at one of the most restricted golf clubs in America, his historic achievement also introduced the sport to one of its most colorful characters, whose story remains mostly unknown a half-century later.
"Henry Brown was a hell of a character -- and a hell of a golfer," said Dave Wilmes, who befriended the caddie after he moved to South Bend, Indiana, in the early 1980s. "He was a wizard on the course with a super short game. He was a damn big showoff."
FOR THE FIRST 40 years of the Masters, golfers had been exclusively white and caddies were Black. By the early 1970s, Brown was hopeful Elder could change that. For four years, he requested to carry Elder's bag if he qualified for the Masters.
Elder finally did when he captured his first PGA Tour victory at the Monsanto Open on April 21, 1974, defeating England's Peter Oosterhuis with a birdie on the fourth hole of a sudden-death playoff. A short time later, Masters chair Clifford Roberts issued a statement saying Elder would be invited to play in 1975.
Elder wasn't the first African American golfer to win on the PGA Tour -- Pete Brown and Charlie Sifford had done it before him -- but he was the first to do it after the Masters revised its qualifying criteria to allow Black golfers in the tournament.
When Elder arrived on Magnolia Lane in a red limousine, Brown was the first person to greet him. Despite his confidence, Brown wasn't regarded as Augusta National's best caddie. That distinction belonged to Willie Peterson, who carried Nicklaus' bag in his first four Masters victories; Willie "Pappy" Stokes, who helped four golfers win five times; or Nathaniel "Iron Man" Avery, who helped Arnold Palmer slip on a green jacket four times. But Brown had already witnessed history at Augusta.
In the 1968 Masters, Brown caddied for Argentina's Roberto De Vicenzo, who famously knocked himself out of a playoff by signing an incorrect scorecard after the final round in one of the biggest blunders in sports history.
De Vicenzo, the reigning Open Championship winner, made a birdie on the par-4 17th hole to tie Bob Goalby for the lead, but his playing partner, Tommy Aaron, who was keeping De Vicenzo's scorecard, wrote down a par.
When De Vicenzo failed to correct the error when he signed his scorecard after the round, the higher score counted under United States Golf Association rules, and he finished a shot behind Goalby, who won a green jacket with a 72-hole total of 11-under 277. If De Vicenzo had recognized the scoring error, he would have faced Goalby in an 18-hole playoff the next day.
After tossing away a chance to win the Masters, De Vicenzo could only utter, "What a stupid I am." Years later, Brown told South Bend Tribune golf writer John Fineran that he tried to chase down Aaron in the parking lot of Augusta National, believing he had wronged De Vicenzo.
In subsequent years, Brown carried Goalby's bag, as well as 1959 Masters champion Art Wall Jr.'s.
When Elder stepped to the first tee in the opening round of the Masters on April 10, 1975, he was understandably nervous. He joked with reporters that a week earlier, he had tried to light a scoring pencil, instead of a cigarette. To break the silence, he asked patrons whether it was going to rain -- it was already drizzling on an overcast day.
Jim Brown, who had retired from the NFL nine years earlier and was working as an actor, stood near the No. 1 tee when Elder hit his tee shot about 250 yards down the fairway. Many of Elder's friends and family also looked on; he had requested 69 tournament badges and received 25, more than Nicklaus and Palmer.
The jitters faded, but he and Henry J. Brown didn't hit it off immediately. The caddie liked to judge distances by sight, rather than walking it off and measuring.
"Henry was a very personable guy, but his idea of caddying was the wrong way for me," Elder said when he was an honorary starter at the 2021 Masters, according to Ward Clayton's book, "The Legendary Caddies of Augusta National." "I usually played by yardage. He wanted to hand me a club instead of figuring out how far we were away."
Elder calmed down after making a birdie on the par-5 second hole, but then he carded bogeys on the next two par-5s. He posted a 2-over 74 in the opening 18 holes, which left him seven shots behind leader Bobby Nichols but only three shots out of 10th place. The top 44 golfers and ties made the 36-hole cut in 1975.
Elder battled windy conditions in the second round and struggled to post a 6-over 78. He missed the cut by four strokes.
"I did myself in," Elder said afterward. "I got out of the box bad. I was all over the place. I really did the beautification bit. I saw all the flowers and trees."
But Brown knew he was seeing a professional in action.
"I can see when a golfer gets nervous -- he gets crabbish," Brown told reporters after the first round. "I didn't see that in this man."
And Brown would know. He was one of the best golfers to ever wear a green and white caddie jumpsuit at Augusta National. De Vicenzo was asked about Brown when he was on Elder's bag. He praised him as a "fine caddie," but the man who narrowly missed being a Masters champion also offered a surprising admission:
"I think he can beat me," he said in 1975.
IKE "STABBER" CHOICE, who started caddying at Augusta National in the late 1960s, remembers Brown playing golf with his older brother, Charlie Choice. Each spring, a group of Black golfers would come to hardscrabble Augusta Municipal Golf Course, fondly known as the Patch, and Brown and others would usually take their money.
"He could shoot low numbers at the Patch because he was used to it," Ike Choice told ESPN. "He knew all the little nicks and knots, so he knew which way the ball was going to kick."
Brown played with a cross-handed grip and could hit the ball both right- and left-handed. He spent much of his time playing at the Patch with other Augusta National caddies such as Tommy "Burnt Biscuits" Bennett, Jariah "Bubba" Beard, "Cigarette" Bobby Jones and Jim Dent, who caddied for Goalby and won 12 times on the Senior PGA Tour in the 1980s and '90s.
In May 1980, Larry Adamson, the USGA's director of championship administration, received a letter he'll never forget. It was written on stationery from the Richmond County Jail in Augusta. The letter was from Brown, who had been accused of raping his ex-wife in the back of his taxi cab on April 12, 1979 (a Richmond County grand jury declined to indict Brown on the charges, according to a report in The Augusta Chronicle on May 21, 1980).
In the letter to Adamson, a copy of which was obtained by ESPN, Brown wrote, "I was not able to try to qualify for the U.S. Open, so if you would let me try wherever [when] I get out, I will be pleased. I was supposed to go to Atlanta [for a qualifier] at Horseshoe Bend. But I could not because I am in the county jail and cannot get a bond out, so I hope that I can get another chance."
Adamson, who retired from the USGA in 2002, informed Brown that he had already missed the deadline to register for a local qualifier for the 1980 U.S. Open at Baltusrol Golf Club in Springfield, New Jersey. Undeterred, Brown asked Adamson to send USGA executive director of rules and competitions P.J. Boatwright Jr. to Augusta to help spring him from jail.
"He wanted P.J. Boatwright to be his marker and keep his score, and then we could match his score with the day of qualifying," Adamson said. "I explained to Henry that's not how golf worked."
In the USGA's Golf Journal in June 1996, Brown said that he told Adamson: "If you set up this [qualifier] for me, you can even handcuff me between shots."
When Brown was released from jail, he called Adamson. "I'm out," he told him. "I told you I didn't do it."
He wanted to register for a local qualifier for the 1981 U.S. Open. Adamson said he'd rush him an application.
Brown gave him the address, which turned out to be at a salvage yard in South Bend. Brown returned the application, but he had missed the deadline again and failed to include his entry fee.
Robert Lee, a longtime attorney who oversaw the U.S. Open qualifiers in northern Indiana, remembers Brown walking into the pro shop at South Bend Country Club in the early spring of 1981. There was still snow on the ground at the course. Brown informed the golf pro, Banks Guyton, that he had worked as a caddie at the Masters. Guyton asked Brown for proof, and he presented his Augusta National identification card.
"Henry told the pro he was going to win the U.S. Open," Lee told ESPN. "He really believed that he could."
BROWN USED THE salvage yard address because that's where he spent Friday afternoons to make a little money. After employees received their paycheck, Brown would hang around and regale them with his golf tales.
"Those guys would be working near quitting time, and Henry would open up his truck and get out a few clubs," Lee said. "He'd get those guys gathered around, and he'd show them his grip and all that."
Then Brown would say, "You see that green Ford on top of that pile of cars over there? I bet I can hit it." He'd do the same with a red Chevrolet in the distance and a black Dodge that would require a sweeping hook.
Lee said Brown played with Slazenger irons and could hit an iron from anywhere. He could also "literally eat chicken soup with a wedge."
"He was that good," Lee said. "He could go either way. To bait guys, he would hit it in regular fashion, then he'd bet them five bucks he could do it cross-handed. Most people would take that bet."
Finally, Brown submitted his USGA application on time for a U.S. Open local qualifier at South Bend Country Club in the spring of 1982. On the day of the 36-hole qualifier, Brown was driving his 1965 Pontiac Catalina -- with half a bowling trophy as a hood ornament -- into the clubhouse parking lot when one of the car's tires blew. It sounded like a gunshot and sent people scurrying for cover.
"[Guyton] told him, 'You move that car, or I'll get somebody to move it for you," Fineran said.
After 36 holes, Brown was tied with Tom Stevens at the top of the leaderboard at 1-under 141 and had qualified to move on to sectional qualifying.
"One of the local TV stations was interviewing Henry after he won, and a car pulled up," Fineran said. "Two women jumped out and pushed a tire up the driveway. They changed the tire while the interview was going on and then drove off."
In a sectional qualifier at La Grange Country Club outside Chicago, Brown missed qualifying for the 1982 U.S. Open at Pebble Beach Golf Links by one stroke. He advanced to sectional qualifying each of the next three years but never made it to the major championship.
"He never was angry, he was never bitter," Adamson said. "I deal with all the players who missed the deadline, about 12,000 of them. I heard every story going, and people would get mad at you. They'd be waiting in the parking lot for you; they hated you.
"Henry was a guy that we declined three times because he was late and he still wasn't mad."
In April 1992, Fineran pulled into a parking lot of a hospital in South Bend to see his newborn son. He noticed a car with Georgia plates and an Augusta National caddie hat in the back window. Fineran knew Brown had been sick with cancer and figured his friend had taken a turn for the worse.
Brown died on July 22, 1992. He was 53. He was survived by six daughters and six sons, as well as a life of what-ifs.
Legendary Los Angeles Times sports columnist Jim Murray told Brown's story in 1975, about how he grew up motherless in Augusta, one of 10 children in his home. About how he started caddying at Augusta Municipal Golf Course when he was 9 and moved to Augusta National five years later. About how he caddied during the daytime and drove a cab at night.
"Although Henry J. was a better-than-fair country golfer, he could not exactly get to the point where he could tee it up in the U.S. Open, like another celebrated contemporary caddie, Johnny Miller," Murray wrote. "If Henry J. Brown had been born blond-haired and blue-eyed, life might have been spectacularly different."