Brown entered the game 0-for-15 this spring, but his chances of becoming the Phils' Opening Day right fielder ended in his first at-bat of the day.
Facing left-hander Paul Maholm with one out in the second inning, he fractured the hook of the hamate bone while swinging at the first pitch.
Brown remained in the batter's box and singled up the middle but went to the clubhouse at the end of the inning and did not take the field in the third.
Initial X-rays pointed to a possible fracture and Phillies general manager Ruben Amaro Jr. told MLB.com surgery may be necessary.
"We won't know until it's definitive," Amaro said. "What the X-ray reveals right now is what is believed to be a fracture.
"[Brown] didn't sound all that great, but it's part of the game."
Brown, MLB.com's No. 4 prospect, will return to Philadelphia next week for an evaluation by a hand specialist.
If the results of the initial X-ray are confirmed, Brown likely will be sidelined three to six weeks.
The 2006 20th-round Draft pick arrived at camp with Ben Francisco and John Mayberry Jr. as the frontrunners for the right field job vacated by Jayson Werth. He hit .327 with 20 homers, 68 RBIs and 17 stolen bases in 93 games between Double-A Reading and Triple-A Lehigh Valley in 2010 before earning a call-up to Philadelphia in July.
In 35 big league games, Brown batted .210 with a pair of homers and 12 RBIs.