What concerned the pediatrician is that she has steadily dropped percentiles as time has passed, instead of staying within the same range. For example, it would be different if she started and stayed within the 0% to 10% range. Looking at the chart below for approximate values, at about 2 weeks she was at 70%, 4 months 65%, 6 months 60%, 9 months 30%, 12 months 17%, 18 months 17%, 24 months 9%, 36 months 4%.
So, he'd like us to go and see an endocrinologist to rule out any problems. Personally, I'm not too worried considering the rest of her development and behavior. If anything, she might just not be producing (or absorbing) enough growth hormone. The fix for that is pretty simple...injections of a synthetic growth hormone or other drug (that's my understanding). There could be something more going on, but it's very unlikely. Not to worry.